Difference between revisions of "Documentation/Labs/NrrdReading Writing Optimizations"

From Slicer Wiki
Jump to: navigation, search
(Created page with "= Overview = Here we summarize some efforts to optimize the reading and writing of nrrd files (currently done by using the teem library). == Use Case == When reading or wri...")
 
 
Line 11: Line 11:
 
The idea is to stream the data between the Teem library and Slicer:
 
The idea is to stream the data between the Teem library and Slicer:
  
A) For Teem: last four commits in https://github.com/ProxSim/teem/commits/slicer-2015-04-14-r6245
+
* For Teem: last four commits in https://github.com/ProxSim/teem/commits/slicer-2015-04-14-r6245
B) For Slicer: https://www.slicer.org/w/images/b/b2/Reading_Writing.txt
+
* For Slicer: https://www.slicer.org/w/images/b/b2/Reading_Writing.txt
  
 
== Issue ==
 
== Issue ==
 
Before merging the solution, we still need to address:
 
Before merging the solution, we still need to address:
1) random access for compressed files to allow streaming with good performances in TEEM
+
* random access for compressed files to allow streaming with good performances in TEEM  
2) temporal filter as new interface to read/write data in the nrrdReader in SLICER
+
* temporal filter as new interface to read/write data in the nrrdReader in SLICER
  
 
See https://github.com/SlicerRt/Sequences/pull/59 for more info.
 
See https://github.com/SlicerRt/Sequences/pull/59 for more info.
  
 
== Current contributors ==
 
== Current contributors ==
-Davide Punzo (punzodavide@hotmail.it)
+
* Davide Punzo (punzodavide@hotmail.it)
-Andras Lasso (lasso@queensu.ca)
+
* Andras Lasso (lasso@queensu.ca)
  
 
Anyone is welcome to contribute.
 
Anyone is welcome to contribute.

Latest revision as of 09:10, 12 November 2018

Home < Documentation < Labs < NrrdReading Writing Optimizations

Overview

Here we summarize some efforts to optimize the reading and writing of nrrd files (currently done by using the teem library).

Use Case

When reading or writing nrrd files currently Slicer uses a factor x2 of the dimension of the datasets in RAM. This can already be an issue when loading datasets rather large (~2-3 GB) on systems with limited ram (already 8 GB). The use of virtual memory can slow down (loading time increases up to 30 minutes instead of few seconds) or freeze completely Slicer (e.g., if using HDD).

Solution

The idea is to stream the data between the Teem library and Slicer:

Issue

Before merging the solution, we still need to address:

  • random access for compressed files to allow streaming with good performances in TEEM
  • temporal filter as new interface to read/write data in the nrrdReader in SLICER

See https://github.com/SlicerRt/Sequences/pull/59 for more info.

Current contributors

  • Davide Punzo (punzodavide@hotmail.it)
  • Andras Lasso (lasso@queensu.ca)

Anyone is welcome to contribute.