View source

= Note =

'''This project is no longer active. Nobuhiko Hata, Dec 16, 2009'''


'''This project is no longer active. Nobuhiko Hata, Dec 16, 2009'''


'''This project is no longer active. Nobuhiko Hata, Dec 16, 2009'''

= Overview =
On this page a description of the ''Volume Rendering algorithm with CUDA For Slicer3'' is presented.
A link to the IGT project page can be found on [http://wiki.na-mic.org/Wiki/index.php/Slicer-IGT/GPU-IGT Slicer-IGT/GPU-IGT].

Involved in this projects are '''Benjamin Grauer (integration into vtk/Slicer3)''', Nicholas Harlambang (cuda algorithms) and Nobuhiko Hata (project supervisor).

The Volume Renderer has been tested on GeForce 8800GTX and Windows so far.
The latest CUDA compatible drivers can be found [http://www.nvidia.com/object/cuda_get.html here]. 

== Objective ==
The goal of this project is to implement a CUDA based Volume Rendering Technique inside of [http://www.vtk.org VTK] and Slicer3.

== Project Outline ==
Several steps have to be made to integrating a CUDA based volume rendering technique into vtk.
#Implement and Integrate a '''CUDA Volume Rendering algorithm'''.
#Integrate '''CUDA runtime API''' support into VTK using a class based approach.
#Integrate the algorithm into a '''VTK pipeline''' (described below)
#Create a '''user interface''' to control the flow of the Volume Rendering within Slicer3.

<br>
'''Module VolumeRenderingCuda has been moved out of Slicer3 trunk and saved in NAMICSandBox/CUDAForVolumeRendering, and is not actively being developed. Now, we should use VolumeRendering module for volume rendering in Slicer3. Updated by Haiying Liu on Oct 30, 2009'''
<br>

= Cuda Volume Rendering Algorithm =
The Code that we use so far has been implemented by Nicholas Harlambang and can be viewed [http://svn.orxonox.net/subprojects/volrenSample here].

== Feature Wishlist ==
{| border=1 cellspacing
|- bgcolor="#ccccff"
! Relevance || Feature || Status || Dependency || Description
|-
| +++       || Z-Buffer || 60% || Current Z-buffer to CUDA || Render and stop at the Z-buffer information
|-
| ++ || Different Data Types || 100% || || uchar, char, float, double, short, ushort, int
|-
| +++ || Sample Distance || 60% || || The sampling step distance along the ray.
|-
| ++ ||  Transfer Function || 100% || ||
|-
| + || Lighting Model || 30% || || 
|-
| = || Multiple Lights || 0% || || 
|-
| - || Shading Model || 0% || Lighting Model || Use Shaders to render 
|-
| +++ || Perspective and Planar Projection || 100% || || Perspective Projection using vtkCamera Projection Matrix
|-
| = || Memory To Texture || 100% ||  || Render CUDA directly onto Texture (for GeForce Cards only)
|-
| - || Partial Memory Update || 0% || || Partial Updates of the Input data for animation
|-
| -- || vtkCudaVolumeProperty || 100% || Transfer Function, || Implement all the features of [http://public.kitware.com/VTK/doc/release/4.0/html/classvtkVolumeProperty.html vtkVolumeProperty] 
|-
| - || Animation || 60% || Partial Memory Update || 4D Animated Image Data
|-
| + || No more static Symbols || 80% || || Remove the need of all c-style static Symbols in the CUDA code
|-
| --- || Multicore/GPU implementation || 0% || Everything else || Multicore implementation of the Cuda Rendering
|-
| -- || Gradients || ?% || || Not sure yet!  
|}

= VTK Cuda Runtime API support =
The c-interface to the CUDA objects and functions will be wrapped in a safe object oriented VTK-C++ class hierarchy.
*[[VTK Cuda Wrapper Classes]]
*[http://wiki.na-mic.org/Wiki/index.php/VTK_Cuda_Memory_Management_Classes VTK Cuda Memory Management Classes]

== Cuda Memory ==
Cuda memory comes in different shapes and forms:
*'''Host Memory''': Non-Paged Memory on the host side for fast interaction with the CUDA devices. see ''vtkCudaHostMemory.h''
*'''Linear Memory''': Linerely arranged memory on the CUDA device. see ''vtkCudaMemory.h''
*'''Pitch Memory''': 2D memory arrays that are aligned in the correct fashion for a fast interaction on the cuda devices. (Arranged over the memory banks for quick access). see ''vtkCudaMemoryPitch.h''
*'''Array Memory''': Arrays allocated over the CUDA API. see ''vtkCudaMemoryArray.h''
*'''Texture Memory''': Not discussed here. see [[VTK Cuda Wrapper Classes]]
All these memories are derived from a base class (see ''vtkCudaMemoryBase.h'') as shown in the following picture:

[[Image:vtkCudaMemoryManagementClassDiagram.png|frame|none|Cuda Memory Class Diagram]]

= VTK Modules and Pipeline =
These are the modules that will be created in order to integrate a Cuda based Volume Rendering into the Slicer3 pipeline
# '''A [http://www.vtk.org/doc/release/5.0/html/a01572.html vtkImageReader]:''' to read Volume data as any kind of Volume Data
# '''A [[vtkCudaImageFilter]]:''' A Filter to convert from the reader output to a cuda-able DataSet
# '''A [http://wiki.na-mic.org/Wiki/index.php/VtkCudaDataSet VtkCudaDataSet]:''' A new vtkDataSet where that holds and handles the data transfer from and to one or multiple CUDA devices.
# '''A [[vtkCudaVolumeMapper]]:''' A new vtkVolumeMapper that renders the scene using the 
## lighting model
## a prepared Z buffer
## a camera position
## A [http://www.vtk.org/doc/release/5.0/html/a02096.html vtkTexture] and a Plane to render the result to
# '''An actor''' that places the volume into the scene
# Chain this '''pipeline''' together and attach it to a rendering window.

In the following image the pipeline is displayed together with the connections from the VTK, Slicer3 and CUDA frameworks.
[[Image:VtkCudaVolumeRenderingPipeline.png|frame|none|500px|CUDA Volume Rendering Pipeline]]

Description:
* The '''Fat''' line displays the pipeline flow.
* Green colored fields are classes from the Slicer3 or vtk implementation
* Red colored fields are CUDA and hardware specific parts
* Blue colored fields are specially implemented parts for the communication of cuda and vtk and the volume rendering algorithms.

= A Slicer3 User Interface =
The GUI will be designed using KWWidgets. With this the user will be able to change the rendering resolution, step size, control the color sceme and load and store further information.

= Tasks and Timeline =
== Tasks and Completion ==
{| border=1 cellspacing
|- bgcolor="#ccccff"
! Task
! colspan=3 | Status
|- bgcolor="#bbddff"
|colspan=3 | Overall Status
| rowspan=20 valign="top" | 75%
|-

|- bgcolor="#ddffff"
|colspan=2  | Integrate into VTK 
| rowspan=4 valign="top"  | 85% 
|-
|VTK Memory Class Definitions || 90%
|-
|VTK Access Classes || 80%
|-
|VTK Algorithm || 90%
|-

|-bgcolor="#ddffff"
|colspan=2 bgcolor="#ddffff" | CUDA Volume Rendering Algorithm 
| rowspan=4 valign="top" bgcolor="#ddffff" | 80%
|-
|Adapt to Z-buffer || 70%
|-
|Adapt lighting model || 0%
|-
|Adapt to input Image || 100%
|-

|- bgcolor="#ddffff"
|colspan=2 | Integration into Slicer 
| rowspan=4 valign="top" | 80%
|-
|Integrate as a Module || 100%
|-
| GUI || 90% 
|}

== Timeline ==
{| border="1"
|- bgcolor="#ccccff"
! From !! To !! Task
|-
| 12/14/07 || 05/31/08 || Master Thesis Duration
|-
| 12/14/07 || 12/31/07 || Reading into the Project, Defining Basic Classes
|-
| 01/01/08 || 01/06/08 || Basic Class Definition Ready for Hands on Meeting
|-
| 01/07/08 || 01/12/08 || Hands on meeting in Salt Lake City
|-
| 01/14/08 || 02/01/08 || Further Implementation
|-
| 05/01/08 || 29/05/08 || Write Master Thesis
|-
| 06/14/08 ||          || Hand in Masters Thesis in Zurich
|}

== Progress ==
See the daily progress [http://www.google.com/notebook/public/08097279861700679221/BDQdBIgoQ1czeruoi?hl=en here].
* (01/03/08): Successfully proved that CUDA runs inside of Slicer3 with the code from Nicholas Harlambang
* (02/01/08): Cuda inside of Slicer3 and external Application, 4D imaging
* (02/05/08): Integration Rendering with Cuda Code and Transfer Functions
* (02/11/08): Integration with Network Module [http://wiki.na-mic.org/Wiki/index.php/OpenIGTLink OpenIGTLink] See [http://people.ee.ethz.ch/~grauerb/Slicer3/openIGTLinkCuda02_02.htm here]
* (03/06/08): Model is aligned correctly in Slicer RAS space.

=Install Cuda into Slicer3=
* Download Slicer3 with Cuda: 
 svn co http://www.na-mic.org/svn/Slicer3/branches/igt_cuda_merge Slicer3
* Install Cuda following the instructions in [[Media:Cuda2slicer3.pdf]]

=Resources=
#For testing purposes: [[Image:Heart256.raw]] Store this file in Slicer3/Modules/VolumeRenderingCuda/Testing/heart256.raw
#Also checkout the main project page at [http://wiki.na-mic.org/Wiki/index.php/Slicer-IGT/GPU-IGT Slicer-IGT/GPU-IGT]
#Currently I am working in this [http://www.na-mic.org/svn/Slicer3/branches/cuda branch] 
##on CudaSupport: [http://www.na-mic.org/svn/Slicer3/branches/cuda/Modules/VolumeRenderingCuda Libs/CudaSupport]
##and CudaVolumeRendering: [http://www.na-mic.org/svn/Slicer3/branches/cuda/Modules/VolumeRenderingCuda Modules/VolumeRenderingCuda]
#Slicer Version using CUDA: [[media:SlicerWithCuda.zip]] [[media:SlicerWithCuda_Check.zip]] [[media:SlicerWithCudaAndUltrasoundSimulator.zip]] [[media:SlicerWithCUDAUS_New.gz]] 

==People Involved==
* Nobuhiko Hata: Supervisor
* Nicholas Harlambang: Implementer of the Cuda Code and Algorithms
* Benjamin Grauer: Implementer of the VTK/Cuda integration and CudaSupport Library
* Andreas Freudling: First Implementation of Volume Rendering in Slicer3 and great supporter

==Screen Captures==
<gallery>
Image:FirstCudaVolumeRenderInSlicer3.png|The First Volume Render using CUDA within the Slicer Window
Image:First4DVolumeRenderInSlicer3.png|The First 4D Volume Rendering in CUDA using VTK 
Image:CudaIntegrationMethodHead.png|The First Volume Render using CUDA within the Slicer Window
Image:CudaIntegrationMethodInterpolationHead.png|Volume Rendered Head with a Interpolation Smoother
</gallery>