Difference between revisions of "Documentation/Nightly/Developers/CompressedVideo"

From Slicer Wiki
Jump to: navigation, search
m
(Add streaming section)
Tag: 2017 source edit
 
(3 intermediate revisions by the same user not shown)
Line 20: Line 20:
 
;[{{doxygen-class-url|vtkStreamingVolumeFrame}} vtkStreamingVolumeFrame]
 
;[{{doxygen-class-url|vtkStreamingVolumeFrame}} vtkStreamingVolumeFrame]
 
:Contains the frame data and metadata information.
 
:Contains the frame data and metadata information.
:Represents the [https://en.wikipedia.org/wiki/Video_compression_picture_types frame type], codec, decoded frame size, and a vtkUnsignedCharArray representing the encoded frame data.
+
:Represents the [https://en.wikipedia.org/wiki/Video_compression_picture_types frame type], codec, decoded frame dimensions, and a vtkUnsignedCharArray representing the encoded frame data.
 
:The frame also contains a pointer to the previous frame that must be decoded before the current frame can be decoded.
 
:The frame also contains a pointer to the previous frame that must be decoded before the current frame can be decoded.
 
;[{{doxygen-class-url|vtkStreamingVolumeCodec}} vtkStreamingVolumeCodec]
 
;[{{doxygen-class-url|vtkStreamingVolumeCodec}} vtkStreamingVolumeCodec]
Line 34: Line 34:
 
:Represents a volume created from a compressed video frame.
 
:Represents a volume created from a compressed video frame.
 
:Automatically decodes the frame when it is set and GetImageData() is called, or if there are observers on the vtkImageData.
 
:Automatically decodes the frame when it is set and GetImageData() is called, or if there are observers on the vtkImageData.
 +
 +
= Streaming =
 +
Compressed video frames can be streamed from external applications using the SlicerOpenIGTLink extension.
 +
The [https://plustoolkit.github.io/ PLUS Server] application can stream compressed video to Slicer from hardware devices such as webcam and ultrasound.
  
 
=Available codecs and video containers=
 
=Available codecs and video containers=
The following codecs and video container formats are available in Slicer.
+
The following codecs and video container formats are available in Slicer:
 
<br />
 
<br />
  
 +
<br />
 
<b>Codecs</b>
 
<b>Codecs</b>
  
Line 45: Line 50:
  
 
<br />
 
<br />
 
 
<b>Video containers</b>
 
<b>Video containers</b>
  

Latest revision as of 20:11, 14 November 2019

Home < Documentation < Nightly < Developers < CompressedVideo


For the latest Slicer documentation, visit the read-the-docs.


Slicer compressed video overview and acknowledgements

Slicer provides infrastructure for visualizing, and managing compressed video frames. Compressed video data is stored in a volume node that performs automatic encoding and decoding as needed. In conjunction with the Sequences and SlicerIGSIO modules, it is also possible to playback and record video sequences in a compressed format. These compressed video sequences can be read and written to a video container format that can be played back in any supporting media player.

Authors:
Kyle Sunderland (PerkLab, Queen's University)
Andras Lasso (PerkLab, Queen's University)

Acknowledgements:
Development was funded in part by CANARIE’s Research Software Program.

Architecture

Frame management

vtkStreamingVolumeFrame
Contains the frame data and metadata information.
Represents the frame type, codec, decoded frame dimensions, and a vtkUnsignedCharArray representing the encoded frame data.
The frame also contains a pointer to the previous frame that must be decoded before the current frame can be decoded.
vtkStreamingVolumeCodec
Decodes/Encodes the vtkStreamingVolumeFrame to convert the contents to/from vtkImageData.
When decoding a frame, it will compare against the last decoded frame and decode as many previous frames as necessary for the current frame.
vtkStreamingVolumeCodecFactory
Factory for registering availiable codecs.
Can create a new codec given a FOURCC.

MRML nodes

vtkMRMLStreamingVolumeNode
Represents a volume created from a compressed video frame.
Automatically decodes the frame when it is set and GetImageData() is called, or if there are observers on the vtkImageData.

Streaming

Compressed video frames can be streamed from external applications using the SlicerOpenIGTLink extension. The PLUS Server application can stream compressed video to Slicer from hardware devices such as webcam and ultrasound.

Available codecs and video containers

The following codecs and video container formats are available in Slicer:


Codecs

  • Uncompressed RGB: Included in Slicer
  • VP9: Included in the SlicerIGSIO extension


Video containers

  • MKV: Included in the SlicerIGSIO extension

Future work

  • Add support for additional codecs
  • Add support for additional video container formats