www.slicer.org Slicer Wiki pages 

We have a NEW web site, where you can find the newest relevant information on 3DSlicer. These pages are for archival purposes.

MRMLaid

Medical Reality Modeling Language

MRML is a format for describing 3D scenes that consist of various types of medical data sets collected in various geometric locations.

1. Motivations 2. XML Syntax
3. Design
    a. Basic Nodes
    b. Hierarchical Structure
    c. Current Details
    d. Proposed Details 		4. VTK Implementation

1. Motivations

Visualizing medical data involves combining various data sets into a single scene, and exploring the scene interactively. Using Slicer typically involves creating a scene from volume data sets, surface models derived from those volumes, and transformations derived from 3D registrations of both the volumes and models. We have found that the proper coordination of these items is easier to attain by using a hierarchical modeling paradigm, similar to the modeling systems and languages of graphics and CAD/CAM.

Toward this end, we created the Medical Reality Modeling Language (MRML) as a format for describing scenes that consist of various types of data sets collected in various geometric locations. These scenes can then be explored using programs that are essentially MRML browsers, such as Slicer.

MRML files describe three aspects of data:

top

2. XML Syntax

MRML is implemented as a type of XML document, with new tags defined to handle medical image data types such as volumes, models, and the coordinate transforms between them.

There are several advantages to building on the XML standard, as opposed to an original format. The World Wide Web has popularized markup languages so that the XML structure is immediately familiar to computer scientists everywhere. There are off-the-shelf XML parsers available in several programming languages. For example, double-clicking an XML file in Windows results in viewing it inside Microsoft Internet Explorer.

MRML files can be organized hierarchically to form a database. Scripts can parse the XML files to navigate the data automatically to perform large-scale batch operations, such as registering thousands of volumes together.

    

What is XML?

XML is the next generation of HTML for use as the document language of the world wide web. While HTML tags data with instructions on how a browser should interpret the data (including images, links, and tables), XML extends this functionality by allowing users to define their own tags.

Here are some helpful links we use in developing MRML:

top

3. Design

In MRML, a 3D scene is represented as a tree-like graph, where volumes, models, and other items are the nodes in the graph. Each node has attributes for specifying its data.
Basic Nodes Hierarchy Current Details Proposed Details

top

3a. Basic Nodes

We introduce the most important MRML nodes here, and more details can be found under the Current Details section.
Basic MRML Nodes
Volume
Volume nodes describe data sets that can be thought of as stacks of 2D images that form a 3D volume. Volume nodes describe where the images are stored on disk, how to render the data (window and level), and how to read the files. This information is extracted from the image headers (if they exist) of volume files at the time the MRML file is generated. Consequently, MRML files isolate MRML browsers from understanding how to read the myriad of file formats for medical data.
Model
Model nodes describe polygonal data. They indicate where the model is stored on disk, and how to render it (color, opacity, and so forth). Models (3D surfaces) are assumed to have been constructed with the orientation and voxel dimensions of the original segmented volume.
Matrix
The output of a rigid-body registration is a rotation and translation expressed mathematically as a transformation matrix. These transforms can be inserted into MRML files as Matrix nodes. Each matrix affects volumes and models that appear below it in the MRML file. Multiple matrices can be concatenated together.
Transform
A Transform is not a node with attributes, but a construct for building MRML files. A Transform encapsulates the Matrix nodes inside it, such that they are invisible to nodes outside the Transform.
Color
Color nodes define colors by describing not only the actual color value, but also its name and a list of label values. One attribute of model nodes is the name of the associated color. When Slicer displays label maps, it colors each voxel by looking up the color associated with that label value. Thus, when label maps appear on reformatted slices, their colors automatically match the corresponding surface models in the 3D view.
Option
Option nodes allow browser-specific information to be stored in a MRML file. For example, Slicer uses Option nodes to store the user's 3D viewpoint information, since there currently is no View node in MRML 2.0.

top

3b. Hierarchical Structure

Transforms along each branch are concatenated to form a composite transform applied to the leaf node.
The figure below displays an example MRML file and the tree-like graph it represents. The order of nodes in the file defines the tree structure; each transform is a parent node, and all the nodes following it (until the next transform) are its children. In the figure, the transform for the last volume in the file is the concatenation of all the transforms in the file. More complex trees, such as where multiple siblings have children, can be constructed.

In the figure above, pre-operative volumes named spgr and fmri were registered, and the fMRI transform was computed to align the fmri volume to the spgr. Then, the Pre-op transform was computed to align this fused, pre-operative data to the intra-operative volume called intra. Thus, the transform applied to the fmri volume is the concatenation of both transform nodes in the MRML file.

top

3c. Current Details

Browsers know how to interpret XML files from the file's Document Type Description (DTD). Our DTDs are: They are also in SLICER_HOME/slicer/dtd/.

A sample MRML file, of two volumes (registered using a transform) and a couple of surface models, is shown here.

In this section, we expand on the description of nodes given in the Basic Nodes section.

Volume node details

Key Value
name Name displayed on the user interface.
filePattern The pattern for naming the files, expressed in C language syntax. For example, a file named skin.001 has a pattern of %s.%03d, and a file named skin.1 has %s.%d for a pattern.
filePrefix Prefix of the image files. For example, the prefix of /d/skin.001 is /d/skin .
rasToIjkMatrix The matrix that transforms a point in RAS space to IJK space. The 4x4 matrix is listed as a string of 16 numbers in row-major order.
rasToVtkMatrix Same as the rasToIjkMatrix, except that the y-axis is directed up instead of down, which is the convention in VTK.
positionMatrix A scaled IJK-to-RAS matrix, formed by accounting for the position and orientation of the volume, but not the voxel size.
colorLUT Numerical ID of the color lookup table to use for rendering the volume.
description Optional text description of the data.
scalarType The type of data in the file. Either Char, UnsignedChar, Short, UnsignedShort, Int, UnsignedInt, Long, UnsignedLong, Float, or Double.
numScalars The number of scalar components for each voxel. Gray-level data has 1. Color data has 3.
littleEndian Describes the order of bytes for each voxel. Little-endian positions the least-significant byte on the rightmost end, and is true of data generated on a PC or SGI.
interpolate Determines whether to interpolate when reformatting slices of this volume.
labelMap Indicates if this volume is a label map, which is the output of segmentation that labels each voxel according to its tissue type. The alternative is a gray-level or color image.
tilt The gantry tilt, in degrees, for CT slices.
autoWindowLevel Specifies whether windowing and leveling are to be performed automatically.
window The window value to use when autoWindowLevel is 'no'.
level The level value to use when autoWindowLevel is 'no'.
applyThreshold Specifies whether to apply the threshold.
autoThreshold Specifies whether the threshold should be set automatically.
lowerThreshold The lower threshold value to use when autoThreshold is 'no'.
upperThreshold The upper threshold value to use when autoThreshold is 'no'.
imageRange Two numbers: the first and last image numbers in the volume. For example, 1 124.
dimensions Two numbers: the number of columns and rows of pixels in each image.
spacing Three numbers for the dimensions of each voxel, in millimeters.
scanOrder The order of slices in the volume. Either LR (left-to-right), RL, AP, PA, IS, or SI. This information is encoded in rasToIjkMatrix, so it is only stated here as a convenience to people reading the file.

Model node details

Key Value
name Name displayed on the user interface.
fileName Path of the data file.
colorName Name of the model's color, which is defined by a Color node in a MRML file.
description Optional text description of the data.
opacity Opacity of the surface expressed as a number from 0 to 1.
visibility Indicates if the surface is visible.
clipping Specifies whether to clip the surface with the slice planes.
backfaceCulling Indicates whether to cull (not render) the backface of the surface.
scalarVisibility Indicates whether to render the scalar value associated with each polygon vertex.
scalarRange Range of scalar values to render rather than the single color designated by colorName.

Color node details

Key Value
name Required, the name displayed on the user interface.
description Optional text description of the color.
diffuseColor Diffuse component of color expressed as 3 RGB (Red Green Blue) values in the range of 0 to 1.
ambient Factor of the effect of ambient light from 0 to 1.
diffuse Factor of the effect of diffuse reflection from 0 to 1.
specular Factor of the effect of specular reflection from 0 to 1.
power Specular power in the range of 1 to 100.
labels List of label values associated with this color.

Matrix node details

Key Value
name Name displayed on the user interface.
matrix 16 numbers that form a 4x4 matrix. The matrix is multiplied by a point (M*P) to compute the transformed point.

Options node details

Key Value
program A required attribute, the name of the program that should recognize these options.
content Short description of the contents of this node. For example, 'presets'.
options List of options expressed in this form: key1='value1' key2='value2'.

top

3d. Proposed Details

The proposed MRML specification differs from the current by the addition of support for volume rendering and hierarchical models. Handling volume rendering involves transfer functions for each color component (R,G,B) and opacity. The window/level/threshold attributes of current Volume nodes will be removed and replaced with Transfer Function nodes. This architecture more closely resembles the model of a normalized database. There will be standard Window/Level nodes for viewing CT data of various anatomy such as brain, abdomen, and so forth.

Volume node

A proposed addition to the volume node is as follows:
Key Value
id Required, a unique id for this volume. Default is ID.
transferFunction An implied attribute.

Model node

A proposed addition to the model node is as follows:
Key Value
id Required, a unique id for this model.
displayProperties Moved the display properties from this node into a new DisplayProperties node.

DisplayProperties node

This node encapsulates information about how to display a model or a model group.
Key Value
id Unique identifier of the display properties node, required.
name Name displayed on the user interface. Default is "".
colorName Name of the model's color, which is defined by a Color node in a MRML file. Default is "".
opacity Opacity of the surface expressed as a number from 0 to 1. Default is 1.0.
visibility Indicates if the surface is visible. Can be true or false. Default is true.
clipping Is clipping active? Default is false.
backfaceCulling Is backface culling active? Default is true.
scalarVisibility Default is false.
scalarRange Default is "0 100".

Transform node

A Transform node can contain zero or more Matrix nodes, as well as zero or more of Transform or Volume or Model or ModelGroup nodes. This is a grouping node, and will be expanded to include other types of coordinate transformations than just Matrix.

ColorLUT node

Information about color look up table.
Key Value
name A required attribute.
numberOfColors Default is 256.
fileName Default is "".
hueRange A pair of numbers giving the hue range. Default is "0 0.8".
saturationRange A pair of numbers giving the saturation range. Default is "0 0".
valueRange Default is "0 1".
annoColor Default is "1 0 0".

TransferFunction node

Key Value
name A required attribute.

Hierarchy node

A Hierarchy node can hold zero or more of a ModelGroup or a ModelRef node.
Key Value
id If you have more than one hierarchy in the file, use this to differentiate them. Required, but default is programatically "H1".
type The type of the hierarchy. Can be MEDICAL or LANGUAGE. Default is MEDICAL.

ModelGroup node

A node of the hierarchical tree, not connected to a model.
Key Value
id Unique identifier of the model group, required.
name Name displayed on the user interface. Default is "".
description Optional text description of the data. Default is "".
displayProperties Moved the display properties from this node into a new DisplayProperties node.
overRideChildProperties Do the display properties of the group override the display properties of any child models? Default is true.

ModelRef node

A reference to a model.
Key Value
modelRefId Required, ID of the referenced model.

Scene node

A Scene node can hold a matrix, a volume state, a model state, scene options, options, a cross section, or a locator.
Key Value
lang Language, not used. Default is "".
name Name of the scene. Default is "".
description A short description of the scene. Default is "".

Matrix node

As previously defined

VolumeState node

State of a volume data set as visible in one scene.
Key Value
volumeRefId ID of the referenced volume. A required attribute.
colorLUT Color look up table. Default is 0.
foreground Is the volume showing in the foreground? A fading option. Default is false.
background Is the volume shown in the background? A fading option. Default is false.
fade Is the volume faded? A fading option. Default is false.
opacity Opacity of the volume. A fading option. Default is 1.0.

ModelState node

Key Value
modelRefID ID of the referenced model. A required attribute.
visible Visibility, an attribute of the model. Default is true.
opacity Opacity, an attribute of the model. Default is 1.0.
sliderVisible Visibility of the opacity slider. Used in SPLVis, not Slicer. Default is true.
childrenVisible Are the children of this node visible? Only used for model groups. Default is true.
clipping Is clipping active? Default is false.
backfaceCulling Is backface culling active? Default is true.

SceneOptions node

Key Value
showAxes Show the 3D axes? An annotation option. Can be true or false. Default is false.
showBox Show 3D cube? An annotation option. Can be true or false. Default is true.
showAnnotations Show slice window text? An annotation option. Can be true or false. Default is false.
showSliceBounds Show 3D outline around slices? An annotation option. Can be true or false. Default is false.
showLetters Show 3D Letters? Default is true.
showCross Show the crosshair? Default is true.
showHashes Show hash marks? Default is true.
showMouse Not used in Slicer. Only for SPLViz compatibility. Show the mouse pointer? Default is true.
viewUp View up direction for the camera. VTK camera option. Default is "0 1 0".
position The position of the camera in world coordinates. VTK camera option. Default is "0 0 1".
clippingRange 3D view, the near and far clipping planes. VTK camera option. Default is "0.1 1000".
focalPoint VTK camera option. The focal point of the camera in world coordinates. Default is "0.0 0.0 0.0".
viewAngle VTK camera option. Angular height of the camera view, in degrees. Default is "30".
viewMode The current Slicer viewing mode (3D, 4x512, 4x256, and so forth). Default is "".
viewBgColor Background color of the 3D view: blue, black, or midnight. Default is "".

Options node

As previously defined

CrossSection node

This node holds information about the state of the saved volume. It gives camera positioning information that allows the view to be reproduced the next time the volume is loaded.
To be finished.
Key Value
position Which cross section is referenced. A required attribute. Can be 0, 1, or 2. Default is programatically 0.
direction Can be axial, coronal, sagittal, or none. Default is none.
sliceSlider Position of the slider, showing which slice is currently displayed. Default is 0.
rotatorX Rotation X. Not used in Slicer. Only in SPLVis. Default is 0.
rotatorY Rotation Y. Not used in Slicer. Only in SPLVis. Default is 0.
inModel Visibility in 3D view. The slice is shown in the 3D view. Can be true or false. Default is false.
zoom Zoom. Default 1.0.
clipState Clipping state. Can be true or false. Default is true.
backVolRefId ID of background volume.
foreVolRefId ID of foreground volume.
labelVolRefID ID of label volume.

Locator node

Holds the options for the locator.
Key Value
driver Who or what is driving the locator? Default is "".
diffuseColor Locator color. Default is "".
visibility Is the locator visible? Can be true or false. Default is false.
transverseVisibility Is the transverse of the locator visible? Can be true or false. Default is true.
normalLen Length of the normal of the locator. Default is 100.
transverseLen Length of the transverse of the locator. Default is 25.
radius Radius of the locator. Default is 3.0.

top

4. VTK Implementation

We have implemented MRML as a set of VTK classes. There is a vtkMrmlNode class, which is the superclass of specific types of nodes, such as vtkMrmlVolumeNode. There is a vtkMrmlTree class, derived from vtkCollection, to be a collection of MRML nodes. This tree class contains member functions for traversing the tree, adding and deleting nodes, and writing the entire scene as a MRML file. This is achieved by calling the Write() member function of each node. (There should also be a Read() function, but we currently have the parsing implemented in Tcl, since that's so easy.)

Here is an example of creating and writing a MRML tree:

#Construct MRML nodes after parsing the file
vtkMrmlVolumeNode skin
   skin SetName Skin
   skin SetPrefix models/Skin.vtk
   skin SetOpacity 0.7
   skin SetColor Tan
vtkMrmlColorNode tan
   tan SetName Tan
   tan SetDiffuseColor 1.0 0.8 0.7
# Construct tree
vtkMrmlTree tree
   tree AddNode model
   tree AddNode tan
# Traverse tree
tree InitTraversal
set node [tree GetNextItem]
while {$node != ""} {
   puts "name='[$node GetName]'"
   set node [tree GetNextItem]
}
# Write MRML file
tree Write "scene.xml"

To assist developers who would like to use MRML, we offer our Tcl parser and VTK classes.

Download MRML Toolkit

top