Difference between revisions of "Slicer3:XCEDE use cases"

From Slicer Wiki
Jump to: navigation, search
Line 36: Line 36:
 
'''Step1.''' An investigator uploads all tagged data from a study (using an upload script, or a webGUI or something outside of Slicer) – for instance:
 
'''Step1.''' An investigator uploads all tagged data from a study (using an upload script, or a webGUI or something outside of Slicer) – for instance:
  
curl –X PUT localFile –u user:passwd http://central.xnat.org/CENTRAL_OASIS_CS/OAS1_0009_MR1/uploadFileName  
+
curl –X PUT localFile –u user:passwd http://central.xnat.org/CENTRAL_OASIS_CS/OAS1_0009_MR1/uploadFileName  
  
 
QUESTIONS:  
 
QUESTIONS:  
Line 46: Line 46:
 
'''Step2.''' Investigator queries for all acquisitions uploaded as part of the study, tagged with him as the investigator and with a particular protocol, which have not yet been assigned for segmentation. Maybe this is done via web form, maybe with a script or query tool inside Slicer.
 
'''Step2.''' Investigator queries for all acquisitions uploaded as part of the study, tagged with him as the investigator and with a particular protocol, which have not yet been assigned for segmentation. Maybe this is done via web form, maybe with a script or query tool inside Slicer.
  
curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&investigator=Marek%20Kubicki&protocol=fBIRN&technician=NULL
+
curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&investigator=Marek%20Kubicki&protocol=fBIRN&technician=NULL
  
  
Line 59: Line 59:
 
'''Step3.''' Investigator updates the metadata with some segmentation assignments.
 
'''Step3.''' Investigator updates the metadata with some segmentation assignments.
  
curl –X POST  http://central.xnat.org/acquisition/OAS1_0006_MR1?technician=Tech1&status=assigned  
+
curl –X POST  http://central.xnat.org/acquisition/OAS1_0006_MR1?technician=Tech1&status=assigned  
curl –X POST http://central.xnat.org/acquisition/OAS1_0009_MR1?technician=Tech2&status=assigned  
+
curl –X POST http://central.xnat.org/acquisition/OAS1_0009_MR1?technician=Tech2&status=assigned  
curl –X POST http://central.xnat.org/acquisition/OAS1_0054_MR1?technician=Tech2&status=assigned
+
curl –X POST http://central.xnat.org/acquisition/OAS1_0054_MR1?technician=Tech2&status=assigned
+
http://central.xnat.org/acquisition/OAS1_0450_MR1?technician=Tech2&status=assigned
+
curl -X POST http://central.xnat.org/acquisition/OAS1_0450_MR1?technician=Tech2&status=assigned
  
  
Line 75: Line 75:
 
'''Step5.''' Technician downloads an XCEDE catalog describing a single acquisition
 
'''Step5.''' Technician downloads an XCEDE catalog describing a single acquisition
  
curl –X GET http://central.xnat.org/acquisition/OAS1_0009_MR1?view=XCAT
+
curl –X GET http://central.xnat.org/acquisition/OAS1_0009_MR1?view=XCAT
  
 
Or, technician downloads all of their assigned acquisitions:
 
Or, technician downloads all of their assigned acquisitions:
  
curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&technician=Tech2&view=XCAT
+
curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&technician=Tech2&view=XCAT
  
 
QUESTION: are multiple XCEDE catalogs returned, or one composite catalog?  
 
QUESTION: are multiple XCEDE catalogs returned, or one composite catalog?  
Line 92: Line 92:
 
Post a file that already has a uri on the DB, retrieved from the XCAT:
 
Post a file that already has a uri on the DB, retrieved from the XCAT:
  
curl –X POST {uri}?status=in%20progress
+
curl –X POST {uri}?status=in%20progress
  
 
Put a newly created file:
 
Put a newly created file:
  
curl –X PUT http://central.xnat.org/acquisition/OAS1_0009_MR1/{path}/aseg.mgz?  {all tags from the acquisition?}  &status=completed
+
curl –X PUT http://central.xnat.org/acquisition/OAS1_0009_MR1/{path}/aseg.mgz?  {all tags from the acquisition?}  &status=completed
  
 
QUESTION: what’s a good way to describe with metadata, and to know the appropriate path?
 
QUESTION: what’s a good way to describe with metadata, and to know the appropriate path?

Revision as of 18:15, 21 July 2008

Home < Slicer3:XCEDE use cases

Slicer3 use cases for XCEDE

Slicer use-case #1: Managing image processing project in a Neuroscience Lab

Overview

  • Step1: Investigator uploads data from a study.
  • Step2: Later, the investigator queries for all “Acquisitions” uploaded as part of the study.
  • Step3: The investigator then assigns each acquisition to a researcher for a segmentation task.
  • Step4: A researcher queries for all acquisition segmentations assigned to them.
  • Step5: A researcher downloads some subset of those assignments in an XCEDE catalog, opens the catalog with Slicer and creates a label map.
  • Step6: When finished, a researcher uploads completed work or work in progress and tags the work appropriately.

Assumptions

For example, we’ll use the public OASIS brains dataset on XNAT, and let “composite ID” for an acquisition be OAS1_0006_MR1, or OAS1_0009_MR1, etc. QUESTION: Is it appropriate to assume that an “acquisition’s” compositeID is analygous to XNAT’s MR Session ID?

OAS1_0006_MR1 contains:

  • mpr-1 (mprage)
  • mpr-2 (mprage)
  • mpr-3 (mprage)
  • mpr-4 (mprage)
  • assessment Freesurfer APARC
  • assessment Freesurfer ASEG
  • assessment Atlas Scaling Factor
  • assessment Fast Segmentation
  • misc files
  • quality control images


Stepping thru details

Step1. An investigator uploads all tagged data from a study (using an upload script, or a webGUI or something outside of Slicer) – for instance:

curl –X PUT localFile –u user:passwd http://central.xnat.org/CENTRAL_OASIS_CS/OAS1_0009_MR1/uploadFileName 

QUESTIONS:

  • is uploadFileName the same as {id}?
  • need to bundle data file and XCEDE wrapper somehow?

Script looks for return value and posts error if upload failed.

Step2. Investigator queries for all acquisitions uploaded as part of the study, tagged with him as the investigator and with a particular protocol, which have not yet been assigned for segmentation. Maybe this is done via web form, maybe with a script or query tool inside Slicer.

curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&investigator=Marek%20Kubicki&protocol=fBIRN&technician=NULL


Receives back XCEDE XML describing list of uris for appropriate acquisitions and return code; If return code is good, parse XML to get list of uris and display them using some tool or another. Gets a list like:

http://central.xnat.org/acquisition/OAS1_0006_MR1 http://central.xnat.org/acquisition/OAS1_0009_MR1 http://central.xnat.org/acquisition/OAS1_0054_MR1http://central.xnat.org/acquisition/OAS1_0450_MR1

Step3. Investigator updates the metadata with some segmentation assignments.

curl –X POST  http://central.xnat.org/acquisition/OAS1_0006_MR1?technician=Tech1&status=assigned 
curl –X POST http://central.xnat.org/acquisition/OAS1_0009_MR1?technician=Tech2&status=assigned 
curl –X POST http://central.xnat.org/acquisition/OAS1_0054_MR1?technician=Tech2&status=assigned
…
curl -X POST http://central.xnat.org/acquisition/OAS1_0450_MR1?technician=Tech2&status=assigned


Step4. Technician “Tech2” queries for all of their assignments from this particular investigator.

curl –X GET http://central.xnat.org/acquisitions?technician=Tech2&investigator=Marek%20Kubicki&status=assigned 

Receives back XCEDE XML describing list of uris for acquisitions they need to segment. If return code is good, parse XML to get list of uris and display them using some tool or another.


Step5. Technician downloads an XCEDE catalog describing a single acquisition

curl –X GET http://central.xnat.org/acquisition/OAS1_0009_MR1?view=XCAT

Or, technician downloads all of their assigned acquisitions:

curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&technician=Tech2&view=XCAT

QUESTION: are multiple XCEDE catalogs returned, or one composite catalog?

Technician opens the catalog in Slicer and works on the segmentation.


Step6. At the day’s end, he wants to upload his work in progress with updated status

QUESTION: doing a ‘post’ will presumably overwrite the original data (if it were present in the DB). Does the back end do some automatic versioning? Or should we do this explicitly on the client side, and then perform a ‘put’ of the new version instead of a ‘post’.

Post a file that already has a uri on the DB, retrieved from the XCAT:

curl –X POST {uri}?status=in%20progress

Put a newly created file:

curl –X PUT http://central.xnat.org/acquisition/OAS1_0009_MR1/{path}/aseg.mgz?  {all tags from the acquisition?}  &status=completed

QUESTION: what’s a good way to describe with metadata, and to know the appropriate path?



Query Atlas Module

Contents:

This is a record of the way we're using the xcede2.0 catalog in order to load in a query atlas scene that contains:

  • One FreeSurfer "lh" model and one FreeSurfer "rh" model at most
  • the FreeSurfer structural brain.mgz image volume
  • the FreeSurfer example functional example_func.nii
  • some FIPS-generated Statistical volumes
  • the FreeSurfer aparc+aseg.mgz label map volume
  • the FreeSurfer lh.aparc.annot annotation file
  • the matrix to align the example_func + statistics to the brain.mgz: anat2exf.register.dat

Assumptions:

  • we are assuming a single lh or rh model (or one of each) is in the catalog. (Reason: Slicer will load the lh.aparc.annot file as a scalar overlay onto a model; so the overlay needs to be associated with a model automatically during load. Since the catalog attribute list contains no references to other datasets, the only way to associate the lh.aparc.annot to the lh.pial surface is by matching on the "lh." and "rh." in the model and annotation names.)
  • we are detecting whether a matrix with the name "anat2efx" is in the catalog (we should probably check the uri instead) -- and if so, we are wrapping the list of all FIPS-generated statistical volumes and the example_func(noted above) in a derived transform in the load routine. We recognize files with the string "brain.mgz" "example_func" and "anat2exf" in them, in order to do some automatic calculation of the Slicer transform that does the Slicer ras2ras registration. Then we stuff volumes with "example_func" or "stat" in their name into the derived transform.
    • the tests to detect these files are weak -- need a better test: now, a structural volume with the string 'stat' embedded in it will be stuffed into the transform by mistake.
    • no registration transform is computed if volumes matching the brain.mgz, example_func and an anat2exf.register.dat file are not detected.

New value inventions for attributes:

  • Freesurfer:matrix-1 (this gives us a clue about file format).
  • Freesurfer:overlay-1 (for files that need to be associated with a model in order to load in Slicer)
  • Freesurfer:label-1 (for mgz files that are label maps) This gives us a hint about what kind of color map to use to display the image data. The default colormap for volume data is greyscale.

Validation:

Qdec Module

Description needed...

Prototype XCAT Files

<XCEDE>
<catalog ID="ID0">

<catalogList>

<catalog ID="ID1">
<entryList>

<entry ID="ID2" name="anat2exf" description="matrix that registers anatomical information to functional datasets" format="FreeSurfer:matrix-1" content="anat2exf" cachePath="" modelID="ID2" uri="./anat2exf.register.dat"/>

<entry ID="ID3" name="lh.pial" description="pial surface of left hemisphere" format="FreeSurfer:surface-1" content="lh.pial" cachePath="" uri="./lh.pial"/>

<entry ID="ID4" name="rh.pial" description="pial surface of right hemisphere" format="FreeSurfer:surface-1" content="rh.pial" cachePath="" uri="./rh.pial"/>

<entry ID="ID5" name="brain" description="extracted brain mri" format="FreeSurfer:mgz-1" content="brain" cachePath="" uri="./brain.mgz"/>

<entry ID="ID6" name="exf" description="example functional dataset" format="nifti:nii-1" content="example_func" cachePath="" uri="./example_func.nii"/>

<entry ID="ID7" name="zstat7" description="7th zstatisic contrast E1+E3+E5>Fix" format="nifti:nii-1" content="zstat7" cachePath="" uri="./zstat7.nii"/>

<entry ID="ID8" name="zstat14" description="14th zstatisic contrast P1+P3+P5>Fix" format="nifti:nii-1" content="zstat14" cachePath="" uri="./zstat14.nii"/>

<entry ID="ID9" name="zstat17" description="17th zstatisic contrast Learn>Fix" format="nifti:nii-1" content="zstat17" cachePath="" uri="./zstat17.nii"/>

<entry ID="ID10" name="aparc+aseg" description="parcellation and segmentation label map" labelmap="true" format="FreeSurfer:mgz-1" content="aparc+aseg" cachePath="" uri="./aparc+aseg.mgz"/>

<entry ID="ID11" name="lh.aparc.annot" description="annotations for surface of left hemisphere" format="FreeSurfer:overlay-1" content="lh.aparc.annot" cachePath="" uri="./lh.aparc.annot"/>

<entry ID="ID12" name="rh.aparc.annot" description="annotations for surface of right hemisphere" format="FreeSurfer:overlay-1" content="rh.aparc.annot" cachePath="" uri="./rh.aparc.annot"/>

</entryList>
</catalog>

</catalogList>

</catalog>
</XCEDE>

XCEDE REST Interface

A straw man REST interface to XCEDE. The implementation is using Grails (Groovy on Rails). Grails is a rapid application development environment patterned after Rails (Ruby on Rails). Grails, however, is completely Java-based utilizing industry standard components (Spring, Hibernate, Groovy). In addition, Grails supports many of the common AJAX frameworks and can be compiled and deployed as a .war file.

REST Interaction

REST support four basic operations akin to CRUD, Create, Read, Update and Delete. These verbs operate on resources (nouns) exposed as URIs, with CRUD relating to HTTP operations:

HTTP CRUD
POST Create, Update, Delete
GET Read
PUT Create, Update
DELETE Delete

A server has been set up on slicerl using code in the NAMICSandbox (see next section for details). Two interfaces are available, a human-friendly web site and a machine-friendly REST interface. The command line program curl can interact with the REST interface, and is useful for learning and debugging.

Retrieve a list of available Projects

7: curl -X GET http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/
<?xml version="1.0" encoding="UTF-8"?><list>
  <project id="1">
    <createdDateTime>2008-06-18 21:36:50.291</createdDateTime>
    <description>Neurological implications of working at BWH</description>
    <name>BWH-0001</name>
    <subjectGroups>
      <subjectGroup id="1"/>
    </subjectGroups>
  </project>
  <project id="2">
    <createdDateTime>2008-06-18 21:36:50.332</createdDateTime>
    <description>Bar</description>
    <name>Foo</name>
    <subjectGroups>
      <subjectGroup id="2"/>
    </subjectGroups>
  </project>
</list>

Update an existing Project (Note how Project 2 has been changed)

8: curl -X GET http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/2 > Project2.xml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   242    0   242    0     0    802      0 --:--:-- --:--:-- --:--:--     0

9: cat Project2.xml 
<?xml version="1.0" encoding="UTF-8"?><project id="2">
  <createdDateTime>2008-06-18 21:36:50.332</createdDateTime>
  <description>Bar</description>
  <name>Foo</name>
  <subjectGroups>
    <subjectGroup id="2"/>
  </subjectGroups>

11: perl -e 's/Bar/Garf/g' -pi Project2.xml

12: curl -X PUT -H 'Content-Type: application/xml' -d @Project2.xml http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/2
<?xml version="1.0" encoding="UTF-8"?><project id="2">
  <createdDateTime>2008-06-18 21:36:50.332</createdDateTime>
  <description>Garf</description>
  <name>Foo</name>
  <subjectGroups>
    <null/>
  </subjectGroups>
</project>

Create a new project (Note using POST creates a new project, while PUT updates an existing project, this command created a project with id=3)

15:  curl -X POST -H 'Content-Type: application/xml' -d @Project2.xml http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/
<?xml version="1.0" encoding="UTF-8"?><project id="3">
  <createdDateTime>2008-06-18 21:43:04.790 EDT</createdDateTime>
  <description>Garf</description>
  <name>Foo</name>
  <subjectGroups>
    <null/>
  </subjectGroups>
</project>

Delete a project

17: curl -X DELETE http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/3
Grails/XCEDE development

So, you're ready to kick the tires? Great! The first step is to check out the code from Subversion (make sure you do the second step!).

svn co http://www.na-mic.org/svn/NAMICSandBox/trunk/XCEDE

Resources that come with the Grails binary installation are omitted from Subversion and must be generated.

cd XCEDE
grails upgrade

and answer "yes" to both questions.

The main code is in XCEDE/grails-app/controllers/. Grails uses an MVC model and "coding by convention". Thus, if code is put in the correct place, it works without configuration. This is handy if you know what you are doing, frustrating if not. I would suggest going through "Getting Started with Grails" as a first tutorial.