Difference between revisions of "Slicer3:UIDesign"

From Slicer Wiki
Jump to: navigation, search
m (Text replacement - "\[http:\/\/www\.slicer\.org\/slicerWiki\/index\.php\/([^ ]+) ([^]]+)]" to "$2")
 
(181 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Slicer3:Interface_Design|Return to Slicer3 Interface Design and Usability ]]
 
[[Slicer3:Interface_Design|Return to Slicer3 Interface Design and Usability ]]
  
 +
[[Slicer4:UIDesign | Go to Slicer4 UI Design page ]]
  
 +
<br>
  
−
= Embodiment =
+
<br>
−
 
+
'''This page contains a catalog of user interface design and development notes for various aspects of Slicer 3.4, 3.6 (based on KWWidgets) and links to new work on Slicer 4.0 (based on Qt).'''
−
* Current schematic of the main application GUI: ( higher resolution image: [[Media:GUIMap.png ]])
+
<br>
−
 
 
−
[[Image:SmallEmbodiment.png]]
 
−
 
 
−
* Current schematic of a Slice Viewer and Controller: (higher resolution image: [[ Media:SliceGUI.png ]])
 
−
 
 
−
[[Image:SmallSliceGUI.png]]
 
−
 
 
−
* [[Media:NavZoomWidgetSketch.png|Design sketch for the 3D navigation & Slice magnifier widget]]
 
−
* [[Media:SlicerLookFeelDraftSmall.jpg|Early look and feel draft]]
 
−
* [[Media:LayoutSketch.gif|First sketch of the GUI layout design]]
 
−
* Link to a running [[Slicer3:VisualBlog|visual blog ]] highlighting progress in the Slicer3 embodiment.
 
  
 
= Slicer users =
 
= Slicer users =
Line 33: Line 24:
 
* Application scientists (who use 3DSlicer as a desktop application and turnkey system)
 
* Application scientists (who use 3DSlicer as a desktop application and turnkey system)
  
 +
<br>
  
−
== Current status and TO-DO ==
+
= User Feedback: Feature, Resource, Application Convention Requests =
  
−
The existing base classes form a basic framework for the UI. For the Alpha/Beta releases, we provided a main application window with the general layout described in our current design. For the Release deadline, we plan to complete the overall application GUI. Things we are focusing on:
 
  
−
* choose radiobutton/checkbutton with/without icon convention
+
Please add usability feedback to Slicer3's [http://www.na-mic.org/Bug/my_view_page.php bug tracker]. If you are not familiar with the bug tracker:
−
* finish content for slicer human interface and style guidelines
+
* login or create an account
−
* theme font, match to screen resolution, set in application settings
+
* choose Slicer3 as your project (upper right corner)
−
* finish slicer mouse modes
+
* and select Report Issue.
−
* think through color palette to support semantic applications
+
* make sure to choose the correct category for your report (select "usability" for usability issues).
−
* update module GUI how-to
 
−
* refine ROI toolbox design, work with developer to implement
 
−
* refine lightbox viewer
 
−
* implement mechanism to specify oblique view
 
−
* build classes for pop-up information (help and context info)
 
−
* copyrights
 
  
−
= Working problem: Module Management Wizard Interface & Interaction Design Storyboard =
+
Early feature requests can also be found  '''[[Slicer3:FeatureRequests|here.]]'''
  
−
This wizard allows users to uninstall modules from their current build or to download and install new modules. Currently working on a usable design. Storyboard roughed out during IGT project week Dec8-12 '08, (Terry Lorber, Steve Pieper, Wendy Plesniak)
 
  
−
== Link to [[Slicer3:UIDesign:WorkingProblems:ModuleManagementWizard:Draft1|Draft 1]]==
+
<br>
  
  
−
== Link to [[Slicer3:UIDesign:WorkingProblems:ModuleManagementWizard:Draft1|Draft 2]](in progress)==
 
  
−
=Working problem: Save Data Widget=
+
= Codes to mark contributions =
  
−
# Save Data: how can one save a data file as a specific/different ITK file format. ITK uses a IO plugin framework to support new file formats: is there a way to find the list of file formats supported by ITK at run-time.
+
These wiki pages can be hard to parse. Here's an experiment for marking feedback to make it easier to parse; (copy, paste and customize from below).
−
#* I (Seb) to check with Luis
 
−
#* '''[UPDATE]''' see ImageFileReader<...>::GenerateOutputInformation, BUT every IO file/reader need to expose the list of file extensions it can read and write. Luis is on it, to be added 3.8 end of July. Luis is fast, it's [http://public.kitware.com/cgi-bin/viewcvs.cgi/Code/IO/itkImageIOBase.h?root=Insight&r1=1.48&r2=1.49 in the CVS], and examplified in [http://public.kitware.com/cgi-bin/viewcvs.cgi/Testing/Code/IO/itkImageIOFileNameExtensionsTests.cxx?root=Insight&sortby=date&view=log itkImageIOFileNameExtensionsTests.cxx]
 
−
# Save Data: the "Save" dialog in general is very confusing and buggy. The "Save All" and "Save None" button are mislabelled since they don't perform any save. Saving data to a new file format requires the user to double-click in the file name and manually change the file extension (provided that this extension is known to the user). Changing the data directory does not update the data locations in the multicolumn list (it does if you change the data directory twice). Changing the scene directory has no impact on the data directory.
 
−
#* I (Seb) suggest that the "Save All/None" should be something like "Mark all for save", "Mark none for save". A button should be placed next to each data filename/location to quickly switch to a new file type (depending on what ITK can save to at run-time). The dialog is large, we have room to put a help string/label that explains a little more how the whole dialog works. Changing the data directory should update the location of each data entry in the multicolumn list correctly. A button should be added to quickly set the data directory to match the scene directory (it should not be automatic though).
 
−
#* (wjp): How about something like one of the following designs (comments/revising welcome):
 
  
−
== Draft1 ==
+
[[Image:Opinion.png]] '''(wjp)''' Add a general comment or opinion.
  
−
[[image:SaveDesign1.png]]
+
[[Image:Usability.png]] Flag a usability problem.
  
−
[[image:SaveDesign2.png]]
+
[[Image:Idea.png]] Mark an idea.
  
 +
[[Image:Sketch.png]] [[Slicer3:UIDesign#Codes_to_mark_contributions | Mark a link to related info.]]
  
−
The designs include the following adjustments:
+
[[Image:Bug.png]] Report a bug.
−
* Save data options come first, scene options second
 
−
* Node type is the second column
 
−
* "Not modified" status is displayed in grey, so that "Modified" status stands out
 
−
* Format column is added, showing (optionally) the current format and an icon-button for switching (imagine clicking button, having pop-up widget for choosing among supported file formats. Once a format has been confirmed, the file name is changed to show the new file extension, and if required, "Not Modified" changes to "Modified". Should "save" also be automatically selected too? -- I think probably yes, but this may turn out to be annoying.
 
−
(SP) Yes, it would be useful to have 'Save' automatically selected after changing the file extension.
 
−
* buttons for "Check all" "Check none" are below multi-column
 
−
* Save scene checkbutton is gone -- instead user is given three options at the GUI bottom.
 
−
* Wonder if "Save Data" should be changed to "Save Data Only"?
 
−
(SP) Yes, that would make things clearer.
 
  
−
Icons are here, in case we want to use one of them...
+
[[Image:Fire.png]] Report a really bad bug that's blocking work progress, or causing a crash.
  
−
[[image:FormatSelectButtonIcon1.png]]
+
[[Image:MissingOrStaleDoc.png]] Report missing documentation.
  
−
[[image:FormatSelectButtonIcon2.png]]
+
[[Image:MissingOrStaleDocLink.png]] Report bad documentation link.
  
−
Here's another stab at this widget after playing with it for awhile:
+
[[Image:WeakDoc.png]] Report insufficient documentation.
−
* Controls that set parameters for all datasets (check all, uncheck all, set data dir for all) are clustered together.
 
−
* User has two ways to change format easily -- using the format icon button and editing the filename.
 
−
* User has two ways to specify the save data directory (the button for all files, and individual buttons for individual datasets.)
 
  
−
Previously, the path name had to be hand edited, which is an error-prone exercise. The filename (and extension) were often trailing off the end of the window, so one had to click and scroll in the filename column in order to see the entire path or discover the dataset format. (Image has been updated to reflect review with Yumin and Wendy).
+
= '''Slicer 4.0:''' Working problems in QT Interface Design and Implementation =
  
 +
[[Slicer3:Developers:Projects:QtSlicer|See this link for ongoing project]]
  
−
== Draft2 ==
 
  
−
[[image:SaveDesign3.png]]
+
= '''Slicer 3.6:''' Working problems in Interaction and interface design for KWWidgets-based Slicer interface. =
  
−
* '''[UPDATE]''' (Yumin)
+
= Working problem: Consolidated Load in Slicer including a Data Management System =
−
Here is an actual screen shot of the working-copy save dialog from my machine (it does not yet have the new design for options to save locally or remotely),
 
  
−
=== Implementation ===
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:XNAT_Desktop_GUI_Design | Link to design sketches extending XNAT Desktop to have OsiriX-like UXP & functionality]] ==
  
−
[[Image:WorkCopySaveDialog.png]]
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:Slicer_Consolidated_Load_GUI_Design | Link to info for Slicer Consolidated Load Design integrating a local file management system]] ==
  
−
It has the following changes:
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:XNAT_Enterprise_QueryWidget | Link to user specifications and design sketches for a Qt-based Query Widget for XNAT Enterprise]] ==
−
* The "Select/Save" column is moved to be the first column.
 
−
* The "File Format" column is using a dropdown list, instead of using an icon to invoke another widget. The actual displayed texts for file formats in the dropdown lists are still being worked on as we are adding necessary APIs into itkImageIO.
 
  
 +
= Working problem: Mouse Modes in Slicer3 =
  
−
== Draft3 ==
+
Since a left-mouse-button click in Slicer's 3D or Slice Viewers can mean different things at different times; this can be confusing to users, and can lead to mistakes. We are currently deciding a good paradigm for defining and controlling mouse modes in Slicer's 3D viewer and 2D slice windows. We rely on mouse interaction to transform the view, and to perform explicit actions like placing a fiducial, selecting and moving a fiducial, collecting voxel-value samples, and so on. The challenge is to make all actions possible with just the left mouse button (some mice have only one button), and to disambiguate what a left-mouse-button click in the 3D and Slice viewers means.
  
−
Below is a sketch for changes to the widget to expose functionality for saving to either local disk or remote repositories. This sketch is NOT ready to be acted upon -- just put here for discussion for now.
+
'''Goals:''' We'd like the interaction paradigm to have the following features:
  
−
[[image:SaveData4.png]]
+
* Consistent in all slice windows and 3D viewer
 +
* Reflect common software conventions
 +
* Intuitive and easy for beginners
 +
* Fast for intermediate and expert users
 +
* Rely on only one mouse button (but accommodate multiple mouse buttons)
 +
* Support direct manipulation as much as possible
 +
* Provide clear visual feedback about current mouse mode
 +
* Support both '''view transformation''' (rotate, pan, zoom) as the primary mode, and a set of '''mouse actions''' (pick, manipulate, place, etc..., and '''window/level adjustments''' according to convention in medical imaging software packages.
 +
* Consistent classification of all direct-manipulable entities in slicer into a set of groups, and have consistent behavior for 'selecting' all entities within each group.
  
  
−
== Draft4 ==
+
==Current issues we want to address in existing Slicer3 implementation:==
  
−
This new version (local data handling only) contains the following changes to current implementation:
+
* Mouse motion should '''default''' to "transform view" mode
−
* the word "Node" is dropped from column headings as much as possible since users don't need to think about "nodes"
+
* Interaction relies on three mouse buttons (for rotate, pan, zoom, etc.) which aren't always present on all computers
−
* the Scene is listed along with data -- always at the top, and maybe in a different text color to highlight it.
+
* It's currently easy to forget about persisting non-transform mouse modes (like pick or place) and assume that the view can be transformed -- leading to annoying errors.  We want to avoid these errors
−
* Changes to how File formats are handled for those without writing support are reflected (sketch included below widget mockups).
+
* Would like to free up right mouse button to invoke the display of a 'context menu', but well-established convention uses right mouse button for zooming in all viewers.
−
* "Select all" and "Select none" are above the multicolumn, over the Select column. Icons are used instead of text -- their balloon help should say "Select all" "Select none"
+
* Would like BOTH intuitive one-handed mouse-only interaction to be possible AND hot-key assisted shortcuts to be available.
−
* The Data Directory File browse button is moved to top, above the Directory column.
+
* We need to combine the concepts of '''active''' (volume or model, etc.), '''selected''' (volume or model...) and '''picked''' (volume or model...)
−
* There's an icon-button included in the "Format" column which should raise a pop-up window that lists the available format options and what they are. For instance, Ron notes that a novice user may not know the difference between NRRD (.nrrd) and NRRD (.nhdr). So the pop-up help should clarify this. (There's a vtkSlicerPopUpHelpWidget in Slicer/Base/GUI that might be useful...)
+
* We need to design presentation treatments for all groups of selectable entities within Slicer, and a consistent presentation treatment for all selected entities within each group.
−
* Bottom button text is shortened to "Save Selected" and "Cancel".
 
−
* Note: icon image data and icon resources (zip files) are linked below.
 
−
 
 
−
[[image:SaveData6A.png]]
 
  
−
Modification '''for later''' when we add remote data handling includes:
+
<br>
−
* radiobuttons to switch between Remote and Local save destination for selected guys
 
−
* Local saving exposes an option for saving selected datasets relative to the scene file (Ron's request)
 
  
−
[[image:SaveData6B.png]]
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:MouseModes:OtherSoftware | Link to Other software conventions...]] ==
  
−
How the widget should handle data loaded in a format for which writing is not supported:
 
  
−
[[image:formatFlagsV2.png]]
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:MouseModes:SelectionAndManipulation | Link to Notes on Selection and Direct Manipulation ]] ==
  
−
Icon image data and .h resources for the mockups above:
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:MouseModes:WhatToPick | Link to Developer & user suggestions for what we need to pick]]==
  
−
[[media:IconImageData.zip | icon image data zip file ]]
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:MouseModes:Draft1 | Link to User suggestions and Early draft ]] ==
  
−
[[media:IconResources.zip | icon resources zip file ]]
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:MouseModes:SelectedVersusActive | Link to Notes on State=Selected vs. State=Active in MRML ]] ==
  
−
==Draft5==
 
  
−
A user selects File->Save
 
−
* widget raises with all modified data checked (as in current implementation).
 
−
User has four convenient select options:
 
−
* [[image:CheckAll.png]] Select scene and all data for saving
 
−
* [[image:UncheckAll.png]] Deselect scene and all data
 
−
* [[image:CheckModifiedData.png]] Select all modified data (not scene file)
 
−
* [[image:CheckModified.png ]] Select scene and all modified data.
 
  
−
plus option to choose destination for all selected, and individual dataset options (as in current implementation).
 
  
−
* if user deselects ALL, then selects the a scene file, logic shouldn't automatically re-check all modified data. We don't do this now either -- currently if a user tries to save the scene and unchecked modified data exists, we just pop up a message telling the user that the save requires them to go back and check unchecked stuff. We can keep this same behavior and adapt the Save button logic a little:
 
−
* When save is clicked, logic checks to see if scene saving is requested
 
−
** if so, checks to see if all modified data is selected.
 
−
*** If yes, save.
 
−
*** If not, pop up the message.
 
  
−
Links to image data and resources:
+
<br>
−
* [[media:SaveDataWidget_Resources.zip | Link to image data resources ]]
 
−
* [[media:SaveDataWidget_Images.zip | Link to image data ]]
 
  
−
'''Question:''' Is it confusing to have the scene bundled in with the data? (wjp) I think that it's reasonable to ask users to understand the concepts of datasets and scene descriptions at a high level, and reasonable also to treat them collectively as writable data. Unless it's important to clarify their difference to the user, it's simpler and more compact to treat their saving consistently. Anything I'm missing -- other opinions?
+
= Working problem: CTSC: Facilitating Tumor Analysis Workflow in Slicer =
  
−
[[image:SaveData7.png]]
+
'''Contributors:''' Ron Kikinis, Jeff Yapp, Randy Gollub, Wendy Plesniak
  
 +
'''Goals:''' Perform the following analyses in Slicer and create presentation & tutorial materials
  
−
'''Note: Might be nice feature to support someday:''' should a user be able to save a scene description that contains only the selected subset of data in a scene? I can imagine this to be useful:
+
* '''A. Single Volumetric Dataset''' Basic Volumetric Measurements for Tumor Assessement
 +
** Linear
 +
** Volumetric
 +
* '''B. Dynamic Volumetric Data''' Visualize & Analyze Perfusion Data
 +
** Define ROI
 +
** Interactive Plot of Contrast Dilution as Function of Time
 +
** Measure Peak (first priority) & Transit (second priority) times in Blood Pool & Tumor
 +
** Save timepoints
 +
*** Intensity(t) for ROI1
 +
*** Intensity(t) for ROI2
 +
* '''C. PET Dataset''' Basic Standardized Uptake Value (SUV) Analysis
 +
** Segment Tumor by Hand (ROI or VOI, circle or sphere will work)
 +
** Normalize by SUV (use maximum value)
 +
** Extract Parameters:
 +
*** For single tumor: SUVmax_Baseline, SUVmax_Timepoint, %change
 +
*** For multi tumor: SUM(SUVmax_Baseline) - SUM(SUVmax_Timepoint), %change
 +
* '''D. Visualization Requirements''' Displaying Static, Dynamic and PET data
 +
* '''E. Presentation Materials''' Powerpoint Slides for CTSA IRAT
 +
** '''Deadline: September (for October meeting)'''
 +
*** Short slideset showing Slicer overview & BC tumor analysis
 +
** '''Deadline: November (for RSNA in December)'''
 +
*** Full slidesets including Basic Training and Specific hands-on BC tumor analysis
  
−
My scene contains a volume V, transform T and model M, and two other "working" volumes. I want to save those three nodes only and a scene description that includes them but NOT my working volumes -- but I don't want to delete them yet.
+
[[media:RSNASlidesUnderDevelopment.ppt | Tutorial slides currently under development ]]
−
* I select volume V, transform T, model M and the scene.
 
−
* I click Save
 
−
* Since the scene file is selected, but not all modified data is selected, the logic checks to see if the scene filename already exists:
 
−
** If the scene.mrml file exists, popup message says ''"You have chosen to overwrite the scene file without saving all modified data. MRML file will not describe this data. Ok to proceed? OK/Cancel''
 
−
** If the scene.mrml file doesn't exist, popup message says ''"You have chosen to save the scene file without saving all modified data in the scene. MRML file will not describe this data. OK to proceed? OK/Cancel''
 
  
  
 +
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:BCAnalysis:Volumetric| Link to info for A. Volumetric Analysis]] ===
  
  
−
= Working problem: spot GUI fixing =
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:BCAnalysis:Dynamic| Link to info for B. Dynamic analysis]] ===
−
== Scene snapshot panel ==
 
−
This GUI panel allows the scene to be recorded at any point and restored later; it's particularly useful for sharing relevant views of a scene with collaborators or for instruction. The layout can be made more compact and improved in general. Also, colors on the 'restore' button are overly bold, and the red/green imply more than they intend.
 
  
−
'''Current layout:'''
 
  
−
[[image: oldSceneSnapshot1.png]]
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:BCAnalysis:PET| Link to info for C. PET Analysis]] ===
  
−
[[image: oldSceneSnapshot0.png]]
+
= Working problem: Customizable layouts for Slicer3 =
  
−
'''New design:'''
+
Instead of a finite set of layout options, we would like to provide users the ability to select among a richer set of layouts, some created to be appropriate for analyzing certain kinds of studies, and the ability to customize their application layout as well. We have begun working on ideas at the NA-MIC January 2009 AHM.
  
−
The restore button needs to convey three states:
+
Contributors: Jim Miller, Steve Pieper, Ron Kikinis, Julien Finet, Jean Christophe Fillion-Robin, Wendy Plesniak
  
−
* no scene snapshots exist to restore
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerFlexibleLayout:UXP| Link to first draft of user experience design]] ===
−
[[image: SceneSnapshot2.png]]
 
−
* a scene snapshot is selected to be restored, but is not yet visible.
 
−
[[image: SceneSnapshot0.png]]
 
−
* a selected scene snapshot has been restored, and is displayed.
 
−
[[image: SceneSnapshot1.png]]
 
  
−
= Working problem: Application-wide support for ROI definition =
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerFlexibleLayout:UXP2| Link to second draft of user experience design]] ===
−
 
 
−
==GOALS:==
 
−
* determine relationships between label maps, implicit regions (spheres, circles, ellipses), and vector-format region definitions.
 
−
* determine what kinds of ROIs we will allow users to create/edit at application level
 
−
* determine how to expose ROI functionality application-wide (without "wormholing" to an ROI module)
 
−
* provide tools for defining, editing ROIs
 
−
* provide tools for converting ROIs into label maps (?)
 
−
* provide good mouse+hotkey design
 
−
* generalize implementation to eventually accommodate DICOM-format ROIs
 
−
* determine relationship between fiducial lists and ROIs (from implementation pov).
 
  
−
== Draft design: ==
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerFlexibleLayout:UXP3| Link to tabbed-slicer layout]] ===
  
−
Sketch of the draft interface for the "ROI toolbox" is below:
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerFlexibleLayout:Architecture| Link to first ideas about layout architecture ]] ===
  
−
[[Image:ROIsketch.png ]]
+
= Working problem: Slicer3 workflow for Tractography: usability issues, design & planning =
  
−
Collecting feedback on the [[ Slicer3:EditorUsabilitySessions#Editor_design_sketch_3_.28ROI_toolbox.29: | complete Interface & interaction design]] drafted at June NA-MIC Project Week meeting / MIT:
+
Working with Dr. Jean-Jacques Lemaire, we are understanding the use of Slicer in a complicated tractography analysis, within the context of a particular workflow. This work is identifying usability issues, software bugs, and ideas for improvement. These pages are a touchstone for the project.  
  
−
= Working problem: Consistent checkbuttons and radiobuttons across platforms (Issue Resolved) =
+
Contributors: Jean-Jacques Lemaire, Nicole Aucoin, Wendy Plesniak, Steve Pieper, Alex Yarmarkovich, Andriy Fedorov, Haiying Liu
  
−
== GOALS ==
+
== General issues, comments and feature requests==
−
Currently, radiobutton and checkbutton indicators display their state differently on different platforms Slicer supports. For instance, visually similar indicators on linux and windows actually represent different states. This can be confusing for people who work on both platforms. So instead of using the default Tk bitmaps for the indicators, we are proposing to use our own image data which will display the same on all platforms.
 
  
−
* develop a custom slicer widget as an extension of vtkKWCheckButton
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:Organization| Link to Organization of Presentation Layer]] ===
−
* disambiguate current "deslected" radiobutton icon state -- it can be read as "disabled"
 
−
* override method for turning the old indicator visible/invisible (always keep it off)
 
−
* develop visually parseable convention for icons to represent:
 
−
** selected
 
−
** deselected
 
−
** disabled
 
−
* use methods SetImageToIcon and SetSelectImageToIcon to display selected/deselected icons, selected/deselected icons+indicators
 
−
* provide blank icons with and without indicators for people to download.
 
  
−
Various experiments below for quickly readable icon state: either "selected" "deselected" or "disabled". Varying color, contrast, texture, etc. to find a design solution that reads out state quickly, at a glance. Drop shadow makes icons pop, but requires more pixels in each icon and we might have to re-do all existing icons to ensure good packing behavior. Also, we might get tired of this effect. Contrast differences add more visual "pop" than color differences. The greyed out dotted lines  provide a good solution for "deselected" state, with good visual contrast to the black outlined "selected" state. Both are clearly distinguished from the "disabled" state. Options P&Q are preferred below.
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:Semantics| Link to Naming conventions]] ===
  
−
[[Image:RadioIcons.png]]
+
== Workflow overview and details on sub-steps ==
  
−
How do various icon configurations appear with and without icons, text, and indicators? The preferred convention would be to use either '''indicators + text or icons alone''' -- rather than indicators + icons together, or indicators, icons + text together. Sometimes this extra information may be necessary, so the last two embodiments are not ruled out.
+
Below is a rough outline of the tractography analysis workflow currently in use (pending a proofreading check by J-J). The workflow can be broken down into general stages:
  
−
[[Image:RadioCheckButtons.png]]
+
* setting up: including loading and saving and adjusting data display properties
  
−
Best without indicator, but ok with too.
+
* registration of target dataset to a reference dataset
  
−
== Applying new design to Mouse Mode Toolbar ==
+
* exploratory and analytic tractography
  
−
Easier to comprehend at a glance, and no ambiguity between "disabled" and "deselected"
+
[[Image:DTIworkflowOverview.png]]
  
−
[[Image:MouseModeToolbar.png ]]
+
(if the figure should be corrected, please let me know how to fix -- maybe deformation should be part of the registration step).Data loading happens up front, but data and the scene may be saved frequently throught the workflow. Highlighted are some areas where serious usability issues have been flagged. 
  
−
= Working problem: Navigating with fiducials (Issue resolved: implemented in Editor module and Edit Box)=
+
A plan to address these issues will be discussed on Dec 29/30 2008 at the next Slicer Developers meeting.  Notes from that meeting and a proposed development plan will be posted here.
  
−
==GOALS:==
+
The sections below contain collected comments and suggestions for all workflow stages: '''Setting up''', '''Registration''', and '''Tractography'''. Markers on the figures indicate places where problems are reported.
−
* use fiducials as markers for features of interest
 
−
* allow navigation to fiducial points within a slice window and
 
−
* allow navigation to same fiducial point in all slice windows (a feature we're calling 'jumpslice').
 
−
* want to read-out which fiducial is currently being viewed while navigating
 
−
* want to expose these features in top level in each slice controller widget
 
−
* want also to keep the slice scrollbar from getting too small to use.
 
−
* want to indicate which slice scrollbar direction is R,L, S,I, A,P, depending on orientation
 
  
  
−
==Proposed modification of vtkSliceControllerWidget:==
+
[[Image:DTIworkflowSetup.png]]
  
−
The widget below shows a read-out/entry for current fiducial; buttons for navigate-to-next-fiducial and navigate-to-previous-fiducial. Slice scrollbar is longer, and contains labels at each endpoint.
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:Loading | Link to Comments on Data/Scene Loading]] ===
−
 
−
[[image:FiducialNavigationSliceWindow.png | icons ]]
 
  
−
* user can navigate forward, backward using fiducial jump buttons
+
=== [[Image:Sketch.png]]  [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:Saving | Link to Comments on Saving]] ===
−
* user can enter an ordinal number (or name?) to jump to a fiducial (above, third fiducial of 12 total in the list)
 
−
* if viewers are unlinked, only the active slice viewer is updated
 
−
* if viewers are linked, all viewers are updated to show the slice containing the same fiducial.
 
−
* if a viewer's slice scrollbar is adjusted, its fiducial read-out is reset to ""
 
−
* any time a new fiducial is placed in a viewer, the fiducial read-out is updated to that (current) fiducial.
 
  
−
==Questions about placing and moving new fiducials:==
+
=== [[Image:Sketch.png]]  [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:PreRegistration| Link to Comments on Quality check & Deformation (Pre-registration)]] ===
−
* should placing a new fiducial in the viewer cause a jumpslice event if all viewers are linked?
 
−
Seems we may not always want to trigger this by default. A reasonable solution for this is to unlink viewers when placing fiducials if you don't want the jumpslice behavior...
 
  
−
* should moving a newly placed fiducial in the viewer cause a jumpslice event if all viewers are linked? Again, seems we may not want this default behavior. And again, it seems easy enough to unlink viewers if you don't want the jumpslice behavior.
+
=== [[Image:Sketch.png]]  [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:Registration| Link to Comments on Registration]] ===
  
−
* can we undo the creation and placement of a fiducal easily, so that while exploring the data, user can click, look, delete, and click again -- as an alternative to grabbing and moving an existing fiducial?
+
[[Image:DTIworkflowRegistration.png]]
  
−
==Questions about navigation:==
+
=== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerUsabilityInTractography:Tractography| Link to Comments on Tractography]] ===
−
* first time navigating, we start on fiducial#1. maybe we go forward to
 
−
fiducial 10 of 20. Next time navigating, do we continue on to 11, or restart at 1?
 
−
* Will there ever be too many fiducials to navigate?
 
−
* should the fiducial navigation entry (read-out) also take a fiducial NAME instead of number?
 
−
* Do we want to include other navigation options?
 
−
* With numerous fiducials, it may be hard to tell which fiducial is the corresponding one, when navigating them. Can we color the jump-fiducial temporarily to help integrate all displays?
 
  
 +
[[Image:DTIworkflowTractography.png]]
  
 +
<br>
  
−
= Working problem: Cine display for viewing and analyzing multi-volume data in Slicer3 =
+
= Working problem: Welcome Module for Slicer3.4 release =
  
−
Design for a basic cine controller:
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:WelcomeModule:Draft | Link to Mockup]]==
  
−
[[Image:CineBasicController.png ]]
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:WelcomeModule:VersionA | Link to First Draft Implementation]]==
  
−
Design studies for a cine display that allows viewing either corresponding slices in multiple volumes over time, or consecutive slices in a single volume over time. Version 4 is the most recent iteration, containing elements for specifiying the following animation parameters:
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:WelcomeModule:VersionB | Link to Second Draft Implementation]]==
 +
<br>
  
−
* volumes (select a volume collection to animate)
+
= Working problem: Design revision on Fiducials GUI for Slicer3.4 release =
−
* volume (select a single volume to animate)
 
−
* display (select Green, Yellow or Red slice viewer)
 
−
* step (set slice increment within a volume, or volume increment within a collection)
 
−
* speed (set animation speed from slow to fast)
 
−
* range ( choose range of slices within a volume, or range of volumes within a group. In the former case, we assume the animation displays consecutive slices in the specified range, incrementing according to the step size. In this latter case, we assume you select the slice of interest via the usual slice viewer gui, and animation displays the corresponding slice in each volume within the collection, incrementing according to the step size. )
 
−
* current ( show or choose current slice or current volume ).
 
  
 +
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:FiducialsGUIDesignRevisions:Draft | Link to Draft1]]==
  
−
And controller icons:
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:FiducialsGUIDesignRevisions:Draft2 | Link to Draft2]]==
−
* go to first frame
 
−
* play in reverse
 
−
* pause
 
−
* stop
 
−
* record a movie
 
−
* play forward
 
−
* go to last frame
 
−
* loop
 
−
* ping-pong
 
  
 +
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:FiducialsGUIDesignRevisions:Draft3 | Link to Draft3]]==
  
 +
= Working problem: Module Management Wizard Interface & Interaction Design Storyboard =
  
−
[[Image:CineDisplayDesigns1-4.png ]]
+
This wizard allows users to uninstall modules from their current build or to download and install new modules. Currently working on a usable design. Storyboard roughed out during IGT project week Dec8-12 '08, (Terry Lorber, Steve Pieper, Wendy Plesniak)
  
−
Slicer 2.6's Interval Browser controller for viewing and organizing timeseries data:
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:ModuleManagementWizard:Draft1|Link to Draft 1]]==
  
−
[[Image:ibrowser.png ]]
 
  
−
= Working problem: Mouse Modes in Slicer3 =
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:ModuleManagementWizard:Draft1|Link to Draft 2]] (not yet ready)==
  
−
Currently, a left-mouse-button click in Slicer's 3D or Slice Viewers can mean different things at different times; this can be confusing to users, and can lead to mistakes. We are currently deciding a good paradigm for defining and controlling mouse modes in Slicer's 3D viewer and 2D slice windows. We rely on mouse interaction to transform the view, and to perform explicit actions like placing a fiducial, selecting and moving a fiducial, collecting voxel-value samples, and so on. The challenge is to make all actions possible with just the left mouse button (some mice have only one button), and to disambiguate what a left-mouse-button click in the 3D and Slice viewers means.
+
<br>
  
−
'''Goals:''' We'd like the interaction paradigm to have the following features:
+
= Working problem: Consolidating Load functionality =
  
−
* Consistent in all slice windows and 3D viewer
+
Functionality to load data is spread all over the application... we want to begin consolidating it in some simple fashion and to gradually implement a more integrated and appropriate solution.  
−
* Reflect common software conventions
 
−
* Intuitive and easy for beginners
 
−
* Fast for intermediate and expert users
 
−
* Rely on only one mouse button (but accommodate multiple mouse buttons)
 
−
* Support direct manipulation as much as possible
 
−
* Provide clear visual feedback about current mouse mode
 
−
* Support both '''view transformation''' (rotate, pan, zoom) as the primary mode, and '''mouse actions''' (pick, manipulate, place, etc...) as additional modes.
 
  
 +
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:ConsolidatedLoad:Draft1|Link to Draft 1]]==
  
−
==Current issues we want to address in existing Slicer3 implementation:==
+
<br>
  
−
* Mouse motion should default to "transform view" mode
+
=Working problem: Save Data Widget=
−
* Interaction relies on three mouse buttons (for rotate, pan, zoom, etc.) which aren't always present on all computers
 
−
* It's currently easy to forget about persisting non-transform mouse modes (like pick or place) and assume that the view can be transformed -- leading to annoying errors. 
 
−
* Would like to free up right mouse button to invoke the display of a 'context menu', but well-established convention uses right mouse button for zooming in all viewers.
 
−
* Would like BOTH intuitive one-handed mouse-only interaction to be possible AND hot-key assisted shortcuts to be available.
 
−
* We need to combine the concepts of '''active''' (volume or model, etc.), '''selected''' (volume or model...) and '''picked''' (volume or model...)
 
−
* We need to design a consistent rendering treatment for all selected objects (model, volume, fiducial, vertex, label).
 
  
−
=== Developers /user suggestions: What do we need to pick? ===
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SaveDataWidget:Draft1|Link to Draft 1 and notes]]==
  
−
At the June 2007 project week at MIT, we determined that we'll need to be able to pick several distinct things in Slicer, and the single "pick" mouse mode currently available was not going to provide the unambiguous picking functionality we need. Please add your picking needs to the list developing below. Thanks!
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SaveDataWidget:Draft2|Link to Draft 2 and notes]]==
  
−
* single/multiple vertices on models (for plotting, we need to be able to get the closest vertex ID)
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SaveDataWidget:Draft3|Link to Draft 3 and notes]]==
−
* single/multiple cells on models ( need to calcuate the vertex usually)
 
−
* single/multiple points on surfaces (for placing fiducials, get RAS)
 
−
* single/multiple models
 
−
* model groups
 
−
* single/multiple "nodes" in general
 
−
* "node" groups
 
−
* single/multiple volumes
 
−
* single/multiple fiducials
 
−
* single/multiple ROIs
 
−
* tensors
 
−
* voxels in volumes
 
−
* slices in volumes
 
−
* subvolumes
 
−
* general polygons?
 
−
* annotations?
 
−
* two points to make a line? (this is a ''place'' operation, rather than a ''pick'')
 
−
* for tractography, we need to pick:
 
−
** cell in a model (to get one fiber trajectory)
 
−
** entire models (to pick a bundle)
 
−
** ideally, a model group (to pick an entire structure like the corpus callosum that includes several bundles)
 
−
* It would also be very cool to have a plane widget for 3D intersection tract selection.
 
−
* for tractography and other things with an intermediate processing step before visualization (generation of glyphs or whatever)
 
−
** it may be useful for picking to indicate what was picked, if several models are displayed per node
 
−
** like: model #2 was picked, then my code can know that is a glyph model, rather than a line model, if that is ever needed.
 
−
** this depends on the final implementation of Alex's new cool display API, whether there is ever >1 model allowed
 
  
 +
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SaveDataWidget:Draft4|Link to Draft 4 and notes]]==
  
 +
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SaveDataWidget:Draft5|Link to Draft 5 and notes]]==
  
−
===User-suggestions for interaction patterns===
+
<br>
  
−
* various lassos would be nice to select multiple of the thing current mode indicates
+
=Working problem: Slicer & XNAT Informatics =
−
* single click to select one of the thing the current mode indicates
 
−
* control-click to select multiple of the thing the current mode indicates
 
−
* a shift-click to select ALL of the thing the current mode indicates
 
  
−
===Proposed mouse mode design===
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerInformatics:Draft1|Link to Fetch Medical Informatics (FetchMI) implementation 1 notes]]==
  
−
The following four mouse modes are proposed:  
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SlicerInformatics:Profiling_and_Refining|Link to Fetch Medical Informatics (FetchMI) and RemoteIO profiling/refining notes]]==
−
* '''Pick''' (or ''select''),
 
−
* '''Manipulate''' (or ''transform selected''),
 
−
* '''Place''' ( or ''create new object''), and  
 
−
* '''Transform View'''.
 
  
−
Transform View is the default mode. All other modes can be invoked in a non-persistent manner from this mode with hot-keys. (which keys to choose?) The proposed design attempts to accommodate user requests listed above, and to provide keyboard+mouse commands for transform view and transform object that are consistent across modes.
+
== [[Image:Sketch.png]] [http://www.na-mic.org/Wiki/index.php/CTSC_Ellen_Grant,_CHB  Link to CTSC Use-case: Grant & Pienaar at Children's Hospital Boston] ==
  
−
[[image:MouseModes.png]]
+
== [[Image:Sketch.png]] [http://www.na-mic.org/Wiki/index.php/CTSC_Brad_Dickerson,_MGH  Link to CTSC Use-case: Brad Dickerson at Mass General NMR] ==
  
−
===Current bindings:===
+
== [[Image:Sketch.png]] [http://www.na-mic.org/Wiki/index.php/CTSC_Simon_Warfield,_CHB  Link to CTSC Use-case: Warfield & Weisen at Children's] ==
  
−
* 3D viewer: left mouse button >> rotate
+
== [[Image:Sketch.png]] [http://www.na-mic.org/Wiki/index.php/CTSC_IGT,_BWH  Link to CTSC Use-case: Managing Image Guided Therapy (NCIGT) Restrospective Data at Brigham & Women's Hospital] ==
−
* 3D viewer: middle mouse button >> pan
 
−
* 3D viewer: mouse wheel button >> zoom in/out
 
−
* 3D viewer: right mouse button, pull drag >> zoom in
 
−
* 3D viewer: right mouse button, push drag >> zoom out
 
  
−
* Slice viewer: left mouse button >> select/move a fiducial point
+
== [[Image:Sketch.png]] [http://www.na-mic.org/Wiki/index.php/CTSC_Slicer_XNE Link to FetchMI XNE extension planning (work with Curtis Lisle)] ==
−
* Slice viewer: middle mouse button >> pan
 
−
* Slice viewer: mouse wheel button >> scroll slices
 
−
* Slice viewer: right mouse button, pull drag >> zoom in
 
−
* Slice viewer: right mouse button, push drag >> zoom out
 
−
* special key-bindings enable placing of fiducials
 
  
−
===Proposed bindings: ===
+
<br>
  
−
'''Pick mode:'''
+
= Working problem: Scene Snapshot Interface =
−
* left-mouse-button click (mouse up) for single selection
 
−
* ctrl+left-mouse-button click (mouse up) for multiple selection
 
−
* shift+left-mouse-button click (mouse up) for selecting all of a datatype, where appropriate
 
  
−
'''Manipulate mode:'''
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:SceneSnapshots | Link to Scene Snapshot UI Design]]==
−
* left-mouse-button press+drag to rotate
 
−
* middle-mouse-button press+drag to translate
 
−
* right-mouse-button press+drag to scale
 
−
* shift+left-mouse-button press+drag to translate
 
−
* ctrl+shift+left-mouse-button press+drag to scale
 
  
−
'''Place mode:'''
+
<br>
−
* left-mouse-button-down creates and initially places an object
 
−
* drag translates the object in X,Y
 
−
* left-mouse-button-up finalizes place operation.
 
  
−
'''Transform view mode:'''
+
= Working problem: Application-wide support for ROI definition =
−
* left-mouse-button press+drag to rotate
 
−
* middle-mouse-button press+drag to pan
 
−
* right-mouse-button press+drag to zoom
 
−
* shift+left-mouse-button press+drag to pan
 
−
* ctrl+shift+left-mouse-button press+drag to zoom
 
  
−
===Selection behavior===
+
==GOALS:==
−
Once the concepts of 'active' and 'selected' are folded together, then all transformable data types will be either selected or unselected. When selected they should display a consistent visual treatment that makes this state clear. This presents a number of challenges that we need to address.
+
* determine relationships between label maps, implicit regions (spheres, circles, ellipses), and vector-format region definitions.
 +
* determine what kinds of ROIs we will allow users to create/edit at application level
 +
* determine how to expose ROI functionality application-wide (without "wormholing" to an ROI module)
 +
* provide tools for defining, editing ROIs
 +
* provide tools for converting ROIs into label maps (?)
 +
* provide good mouse+hotkey design
 +
* generalize implementation to eventually accommodate DICOM-format ROIs
 +
* determine relationship between fiducial lists and ROIs (from implementation pov).
  
−
'''Challenge 1: Consistent mouse-over behavior for widgets and selectable data objects.''' Ideally, when the application is in "Pick" mode, there will be consistent mouse-over and selection behavior between VTK widgets and VTK objects representing datasets in the scene. How does an application currently know whether the mouse is over a VTK widget -- is a pick constantly running against a list of instanced widgets? In order to have datasets or sub-sets highlight on mouseover (highlight-rendering aside), the mouse would continuously  have to pick, which is too computationally expensive to be practical.
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:EditorToolbox | Link to Draft ROI design]] ==
  
−
'''Approach:''' One approach is to attach VTK widgets to each object in the scene, as is implemented in the fiducials module. This could be visually noisy, and it may be hard to tell which object a widget is attached to. Maybe better: we could adapt the QueryAtlas picker to display a model name, label name, cell ID (whatever is appropriate for the datatype being picked) on mouse-over in pick mode.
+
== [[Image:Sketch.png]] [[ Slicer3:EditorUsabilitySessions#Editor_design_sketch_3_.28ROI_toolbox.29: | Link to Interface & interaction design]] drafted at June NA-MIC Project Week ==
  
−
'''Challenge2: Clear and clean select feedback for data objects in the scene, compatible with rendering of active vtk widgets.'''
+
== [[Image:Sketch.png]] [[Slicer3:EditorUsabilitySessions | Link to Earlier notes from Editor module usability sessions]] ==
  
−
'''Approach:'''
+
<br>
  
−
'''Challenge3: Ability to easily clear selections.''' Currently, 'active' models and volumes etc. are persistent, but since their 'active' state isn't visually indicated in the scene, this state remains kind of hidden -- and is only revealed in the vtkSlicerNodeSelectorWidgets in the GUI panel. But once 'active' and 'selected' represent the same concept and the visual presentation of any/all selected datatypes is displayed in Slicer viewers, this change of interactive behavior will be very noticeable to users.
+
= Working problem: Consistent checkbuttons and radiobuttons across platforms (Issue Resolved) =
  
−
'''Approach:''' Once the scene display becomes affected by any object's selected state, users are likely going to want to have a QUICK & EASY option to '''deselect all''' after they perform some processing on a selected dataset. Candidate approaches would be to inclue a "deselect all" option in the drop-down pick menu, or an explicit "deselect all" icon in the toolbar.
+
== GOALS ==
 +
Currently, radiobutton and checkbutton indicators display their state differently on different platforms Slicer supports. For instance, visually similar indicators on linux and windows actually represent different states. This can be confusing for people who work on both platforms. So instead of using the default Tk bitmaps for the indicators, we are proposing to use our own image data which will display the same on all platforms.
  
−
===Manipulation behavior===
+
* develop a custom slicer widget as an extension of vtkKWCheckButton
−
A ''vtkSphereWidget'' could be created and displayed surrounding a single or a set of selected objects when Manipulate mode is invoked. The widget would remain active until Manipulate mode was exited or all objects were deselected.
+
* disambiguate current "deslected" radiobutton icon state -- it can be read as "disabled"
 +
* override method for turning the old indicator visible/invisible (always keep it off)
 +
* develop visually parseable convention for icons to represent:
 +
** selected
 +
** deselected
 +
** disabled
 +
* use methods SetImageToIcon and SetSelectImageToIcon to display selected/deselected icons, selected/deselected icons+indicators
 +
* provide blank icons with and without indicators for people to download.
  
−
===Tests: visually rendering selected objects ===
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:RadioCheckButtons | Link to Radio and check button tests]] ==
  
−
What values of ambient, specular and color are used in the first image?
+
<br>
  
−
<gallery perrow="2" widths="500px" heights="300px">
+
= Working problem: Navigating with fiducials (Issue resolved: implemented in Editor module and Edit Box)=
−
Image:Ambient and color.png|Changes to color, ambient, specular and ambient ( all muscles are the same color in the original)
 
−
Image:Slicer3ModelsSelectedAmbientRed.jpg| Using higher ambient setting and red colour to show selected state (inherited from Slicer2)
 
−
Image:Slicer3ModelsSelectedAmbient.jpg| Using ambient setting to show selected state
 
−
Image:Slicer3ModelsSelectedBoundingBox.jpg|Using a bounding box (vtkBoxWidget2) to show selected, bonus of allowing interaction with the model (one off test via the tkcon, need to design the callbacks to allow moving the selected model)
 
−
Image:Slicer3ModelsSelectedSphere.jpg|Using a sphere widget (vtkSphereWidget2) to show selected. Not recommended, as this is more used to position a point on the sphere, and probably won't work as intuitively as the bounding box
 
−
</gallery>
 
  
−
===State in MRML===
+
==GOALS:==
 +
* use fiducials as markers for features of interest
 +
* allow navigation to fiducial points within a slice window and
 +
* allow navigation to same fiducial point in all slice windows (a feature we're calling 'jumpslice').
 +
* want to read-out which fiducial is currently being viewed while navigating
 +
* want to expose these features in top level in each slice controller widget
 +
* want also to keep the slice scrollbar from getting too small to use.
 +
* want to indicate which slice scrollbar direction is R,L, S,I, A,P, depending on orientation
  
−
Currently, Slicer uses the concept of '''active''' datasets (the active volume, an active model, and active fiducial list, etc.) Application-wide, only a single datatype can be active at one time. This set of active datatypes is represented in the '''vtkMRMLSelectionNode'''.
 
  
−
The fiducials module also uses the concept of '''selected''' fiducials within any fiducial list. Many command line modules use only selected fiducials in their processing.
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:NavigatingWithFiducials | Link to Fiducial navigation UI design sketches]] ==
  
−
We propose to collapse these ("active" and "selected") into one state variable and move that concept consistently into the codebase, which includes the following work:
+
<br>
  
−
* defining '''Selected=0 or 1''' on a vtkMRMLTransformableNode.
+
= Working problem: Cine display for viewing and analyzing multi-volume data in Slicer3 =
−
* modifying the ''vtkMRMLSelectionNode'' to include multiple selected objects within any datatype
 
−
* extending the ''vtkSlicerNodeSelectorWidget'' to include checkbuttons rather than radiobuttons for nodes that can have multiple selections
 
−
* Changing the term '''active''' to '''selected''' in the codebase, (Fiducials module, ROI module, SlicesControlGUI, ViewControlGUI, ApplicationLogic and MRML XML parsing). MRML XML parsing should continue to support the "active" tag to remain backward compatible.
 
−
* Building the selection logic that appropriately marks objects as selected or deselected.
 
−
* Encourage developers to conform to this convention of setting and using the '''Pick''' mode for selection instead of overloading the '''Transform View''' mode with hot-keys (provide example of how to get and set the mouse mode, and how to use the selection logic.
 
  
−
== Other software conventions... ==
+
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:CineDisplay | Link to Drafts of UI for Cine Display ]]==
  
−
===How Actions and View Transformations are done in Maya===
+
<br>
  
−
Maya is a very heavily used professional 3D animation package.  Basic mouse bindings are:
+
= Working problem: Linked controls in Slicer3 =
  
−
* Rotate: alt-left mouse button
+
== Goals ==
−
* Pan: alt-middle mouse button
 
−
* Zoom: alt-right or alt-right-and-middle mouse button
 
−
* Actions: always left mouse button, specific action depends on current mode
 
−
* Context menu: space bar
 
  
−
The advantage of this approach is that the left mouse button is freed up for many possible tasks such as selection, grabbing widget handles, etc.  Disadvantage is that it requires two hands to operate. (A free copy of an almost-fully functional version of Maya is [http://www.autodesk.com/maya-trial available] for testing ).
+
Slice viewing can operate in the context of a single slice reformation or in a ''linked'' state where multiple slice viewers respond to interactions with any slice viewer. When Slice Viewers are linked, a number of viewing geometry and view representation parameters are linked across viewers.
  
 +
== [[Image:Sketch.png]] [[Slicer3:UIDesign:WorkingProblems:LinkedControls | Link to Drafts of UI for Linked Controls ]]==
  
 +
<br>
  
−
===How Actions and View Transformations are done in Blender===
+
= Background: Slicer 3.2 usability & ideas for Slicer 3.3=
  
−
[http://www.blender.org Blender] is an Open Source modeling and animation package similar in many ways to Maya (almost like GIMP is to Photoshop).  Basic mouse bindings are:
+
* [[ Slicer3:ApplicationUsabilitySession | Application GUI sessions ]]
 +
* [[ Slicer3:EditorUsabilitySessions | Editor Module sessions ]]
 +
* [[ Slicer3:FiducialsUsabilitySessions | Fiducials functionality ]]
  
−
* Rotate: alt-left mouse button OR middle mouse button
+
= Some Embodiment History=
−
* Pan: shift-middle mouse button
 
−
* Zoom: control-middle mouse button
 
−
* Actions: mostly left mouse button
 
−
* Context menu: space bar
 
  
 +
* [[Media:GUIMap.png| Earlier embodiment of main application GUI]]
 +
* [[Media:SmallSliceGUI.png| Earlier design for Slice Viewer and Slice Controller Widgets]]
 +
* [[Media:NavZoomWidgetSketch.png|Design sketch for the 3D navigation & Slice magnifier widget]]
 +
* [[Media:SlicerLookFeelDraftSmall.jpg|Early look and feel draft]]
 +
* [[Media:LayoutSketch.gif|First sketch of the GUI layout design]]
 +
* Link to a running [[Slicer3:VisualBlog|visual blog ]] highlighting progress in the Slicer3 embodiment.
  
−
===Note: Mac bindings===
+
<br>
−
 
 
−
On a one-button mac, the following translations work by default in Mac OSX:
 
−
* single button -> left mouse button
 
−
* alt (option) + button -> middle mouse button
 
−
* apple (command) + button -> right mouse button
 
−
 
 
−
---
 
−
 
 
−
 
 
−
 
 
−
 
 
−
 
 
−
 
 
−
 
 
−
== GUI screenshots from other packages: ==
 
−
Below are snippets of SketchUp's interface, and Maya's Interface. Maya's mouse modes switch between object transforms (translate, rotate and scale), object or region select, and an extrude function. SketchUp's mouse-mode icons switch between view transforms (translate, rotate, zoom) and an object extrude function.
 
−
 
 
−
[[image:sketchupInterface.png | SketchUp Example ]]
 
−
 
 
−
[[image:mayaInterface.png | Maya Example ]]
 
  
 
[[Slicer3:Interface_Design|Return to Slicer3 Interface Design and Usability ]]
 
[[Slicer3:Interface_Design|Return to Slicer3 Interface Design and Usability ]]

Latest revision as of 02:26, 27 November 2019

Home < Slicer3:UIDesign

Return to Slicer3 Interface Design and Usability

Go to Slicer4 UI Design page



This page contains a catalog of user interface design and development notes for various aspects of Slicer 3.4, 3.6 (based on KWWidgets) and links to new work on Slicer 4.0 (based on Qt).

Contents

Slicer users

Core user communities:

  • (NAC) Longitudinal and multi-channel dataset analysis
  • (BIRN/NA-MIC) Individual and group analysis
  • (IGT/AMIGO) Real-time control and tracking in the operating theater
  • (IGT) Neurosurgical planning and guidance

Types of users:

  • Algorithm researchers (who work within 3DSlicer's development environment and with associated toolkits)
  • Biomedical engineers (who rely on 3DSlicer's interactive enironment and scripting capabilities)
  • Application scientists (who use 3DSlicer as a desktop application and turnkey system)


User Feedback: Feature, Resource, Application Convention Requests

Please add usability feedback to Slicer3's bug tracker. If you are not familiar with the bug tracker:

  • login or create an account
  • choose Slicer3 as your project (upper right corner)
  • and select Report Issue.
  • make sure to choose the correct category for your report (select "usability" for usability issues).

Early feature requests can also be found here.




Codes to mark contributions

These wiki pages can be hard to parse. Here's an experiment for marking feedback to make it easier to parse; (copy, paste and customize from below).

Opinion.png (wjp) Add a general comment or opinion.

Usability.png Flag a usability problem.

Idea.png Mark an idea.

Sketch.png Mark a link to related info.

Bug.png Report a bug.

Fire.png Report a really bad bug that's blocking work progress, or causing a crash.

MissingOrStaleDoc.png Report missing documentation.

MissingOrStaleDocLink.png Report bad documentation link.

WeakDoc.png Report insufficient documentation.

Slicer 4.0: Working problems in QT Interface Design and Implementation

See this link for ongoing project


Slicer 3.6: Working problems in Interaction and interface design for KWWidgets-based Slicer interface.

Working problem: Consolidated Load in Slicer including a Data Management System

Sketch.png Link to design sketches extending XNAT Desktop to have OsiriX-like UXP & functionality

Sketch.png Link to info for Slicer Consolidated Load Design integrating a local file management system

Sketch.png Link to user specifications and design sketches for a Qt-based Query Widget for XNAT Enterprise

Working problem: Mouse Modes in Slicer3

Since a left-mouse-button click in Slicer's 3D or Slice Viewers can mean different things at different times; this can be confusing to users, and can lead to mistakes. We are currently deciding a good paradigm for defining and controlling mouse modes in Slicer's 3D viewer and 2D slice windows. We rely on mouse interaction to transform the view, and to perform explicit actions like placing a fiducial, selecting and moving a fiducial, collecting voxel-value samples, and so on. The challenge is to make all actions possible with just the left mouse button (some mice have only one button), and to disambiguate what a left-mouse-button click in the 3D and Slice viewers means.

Goals: We'd like the interaction paradigm to have the following features:

  • Consistent in all slice windows and 3D viewer
  • Reflect common software conventions
  • Intuitive and easy for beginners
  • Fast for intermediate and expert users
  • Rely on only one mouse button (but accommodate multiple mouse buttons)
  • Support direct manipulation as much as possible
  • Provide clear visual feedback about current mouse mode
  • Support both view transformation (rotate, pan, zoom) as the primary mode, and a set of mouse actions (pick, manipulate, place, etc..., and window/level adjustments according to convention in medical imaging software packages.
  • Consistent classification of all direct-manipulable entities in slicer into a set of groups, and have consistent behavior for 'selecting' all entities within each group.


Current issues we want to address in existing Slicer3 implementation:

  • Mouse motion should default to "transform view" mode
  • Interaction relies on three mouse buttons (for rotate, pan, zoom, etc.) which aren't always present on all computers
  • It's currently easy to forget about persisting non-transform mouse modes (like pick or place) and assume that the view can be transformed -- leading to annoying errors. We want to avoid these errors
  • Would like to free up right mouse button to invoke the display of a 'context menu', but well-established convention uses right mouse button for zooming in all viewers.
  • Would like BOTH intuitive one-handed mouse-only interaction to be possible AND hot-key assisted shortcuts to be available.
  • We need to combine the concepts of active (volume or model, etc.), selected (volume or model...) and picked (volume or model...)
  • We need to design presentation treatments for all groups of selectable entities within Slicer, and a consistent presentation treatment for all selected entities within each group.


Sketch.png Link to Other software conventions...

Sketch.png Link to Notes on Selection and Direct Manipulation

Sketch.png Link to Developer & user suggestions for what we need to pick

Sketch.png Link to User suggestions and Early draft

Sketch.png Link to Notes on State=Selected vs. State=Active in MRML


Working problem: CTSC: Facilitating Tumor Analysis Workflow in Slicer

Contributors: Ron Kikinis, Jeff Yapp, Randy Gollub, Wendy Plesniak

Goals: Perform the following analyses in Slicer and create presentation & tutorial materials

  • A. Single Volumetric Dataset Basic Volumetric Measurements for Tumor Assessement
    • Linear
    • Volumetric
  • B. Dynamic Volumetric Data Visualize & Analyze Perfusion Data
    • Define ROI
    • Interactive Plot of Contrast Dilution as Function of Time
    • Measure Peak (first priority) & Transit (second priority) times in Blood Pool & Tumor
    • Save timepoints
      • Intensity(t) for ROI1
      • Intensity(t) for ROI2
  • C. PET Dataset Basic Standardized Uptake Value (SUV) Analysis
    • Segment Tumor by Hand (ROI or VOI, circle or sphere will work)
    • Normalize by SUV (use maximum value)
    • Extract Parameters:
      • For single tumor: SUVmax_Baseline, SUVmax_Timepoint, %change
      • For multi tumor: SUM(SUVmax_Baseline) - SUM(SUVmax_Timepoint), %change
  • D. Visualization Requirements Displaying Static, Dynamic and PET data
  • E. Presentation Materials Powerpoint Slides for CTSA IRAT
    • Deadline: September (for October meeting)
      • Short slideset showing Slicer overview & BC tumor analysis
    • Deadline: November (for RSNA in December)
      • Full slidesets including Basic Training and Specific hands-on BC tumor analysis

Tutorial slides currently under development


Sketch.png Link to info for A. Volumetric Analysis

Sketch.png Link to info for B. Dynamic analysis

Sketch.png Link to info for C. PET Analysis

Working problem: Customizable layouts for Slicer3

Instead of a finite set of layout options, we would like to provide users the ability to select among a richer set of layouts, some created to be appropriate for analyzing certain kinds of studies, and the ability to customize their application layout as well. We have begun working on ideas at the NA-MIC January 2009 AHM.

Contributors: Jim Miller, Steve Pieper, Ron Kikinis, Julien Finet, Jean Christophe Fillion-Robin, Wendy Plesniak

Sketch.png Link to first draft of user experience design

Sketch.png Link to second draft of user experience design

Sketch.png Link to tabbed-slicer layout

Sketch.png Link to first ideas about layout architecture

Working problem: Slicer3 workflow for Tractography: usability issues, design & planning

Working with Dr. Jean-Jacques Lemaire, we are understanding the use of Slicer in a complicated tractography analysis, within the context of a particular workflow. This work is identifying usability issues, software bugs, and ideas for improvement. These pages are a touchstone for the project.

Contributors: Jean-Jacques Lemaire, Nicole Aucoin, Wendy Plesniak, Steve Pieper, Alex Yarmarkovich, Andriy Fedorov, Haiying Liu

General issues, comments and feature requests

Sketch.png Link to Organization of Presentation Layer

Sketch.png Link to Naming conventions

Workflow overview and details on sub-steps

Below is a rough outline of the tractography analysis workflow currently in use (pending a proofreading check by J-J). The workflow can be broken down into general stages:

  • setting up: including loading and saving and adjusting data display properties
  • registration of target dataset to a reference dataset
  • exploratory and analytic tractography

DTIworkflowOverview.png

(if the figure should be corrected, please let me know how to fix -- maybe deformation should be part of the registration step).Data loading happens up front, but data and the scene may be saved frequently throught the workflow. Highlighted are some areas where serious usability issues have been flagged.

A plan to address these issues will be discussed on Dec 29/30 2008 at the next Slicer Developers meeting. Notes from that meeting and a proposed development plan will be posted here.

The sections below contain collected comments and suggestions for all workflow stages: Setting up, Registration, and Tractography. Markers on the figures indicate places where problems are reported.


DTIworkflowSetup.png

Sketch.png Link to Comments on Data/Scene Loading

Sketch.png Link to Comments on Saving

Sketch.png Link to Comments on Quality check & Deformation (Pre-registration)

Sketch.png Link to Comments on Registration

DTIworkflowRegistration.png

Sketch.png Link to Comments on Tractography

DTIworkflowTractography.png


Working problem: Welcome Module for Slicer3.4 release

Sketch.png Link to Mockup

Sketch.png Link to First Draft Implementation

Sketch.png Link to Second Draft Implementation


Working problem: Design revision on Fiducials GUI for Slicer3.4 release

Sketch.png Link to Draft1

Sketch.png Link to Draft2

Sketch.png Link to Draft3

Working problem: Module Management Wizard Interface & Interaction Design Storyboard

This wizard allows users to uninstall modules from their current build or to download and install new modules. Currently working on a usable design. Storyboard roughed out during IGT project week Dec8-12 '08, (Terry Lorber, Steve Pieper, Wendy Plesniak)

Sketch.png Link to Draft 1

Sketch.png Link to Draft 2 (not yet ready)


Working problem: Consolidating Load functionality

Functionality to load data is spread all over the application... we want to begin consolidating it in some simple fashion and to gradually implement a more integrated and appropriate solution.

Sketch.png Link to Draft 1


Working problem: Save Data Widget

Sketch.png Link to Draft 1 and notes

Sketch.png Link to Draft 2 and notes

Sketch.png Link to Draft 3 and notes

Sketch.png Link to Draft 4 and notes

Sketch.png Link to Draft 5 and notes


Working problem: Slicer & XNAT Informatics

Sketch.png Link to Fetch Medical Informatics (FetchMI) implementation 1 notes

Sketch.png Link to Fetch Medical Informatics (FetchMI) and RemoteIO profiling/refining notes

Sketch.png Link to CTSC Use-case: Grant & Pienaar at Children's Hospital Boston

Sketch.png Link to CTSC Use-case: Brad Dickerson at Mass General NMR

Sketch.png Link to CTSC Use-case: Warfield & Weisen at Children's

Sketch.png Link to CTSC Use-case: Managing Image Guided Therapy (NCIGT) Restrospective Data at Brigham & Women's Hospital

Sketch.png Link to FetchMI XNE extension planning (work with Curtis Lisle)


Working problem: Scene Snapshot Interface

Sketch.png Link to Scene Snapshot UI Design


Working problem: Application-wide support for ROI definition

GOALS:

  • determine relationships between label maps, implicit regions (spheres, circles, ellipses), and vector-format region definitions.
  • determine what kinds of ROIs we will allow users to create/edit at application level
  • determine how to expose ROI functionality application-wide (without "wormholing" to an ROI module)
  • provide tools for defining, editing ROIs
  • provide tools for converting ROIs into label maps (?)
  • provide good mouse+hotkey design
  • generalize implementation to eventually accommodate DICOM-format ROIs
  • determine relationship between fiducial lists and ROIs (from implementation pov).

Sketch.png Link to Draft ROI design

Sketch.png Link to Interface & interaction design drafted at June NA-MIC Project Week

Sketch.png Link to Earlier notes from Editor module usability sessions


Working problem: Consistent checkbuttons and radiobuttons across platforms (Issue Resolved)

GOALS

Currently, radiobutton and checkbutton indicators display their state differently on different platforms Slicer supports. For instance, visually similar indicators on linux and windows actually represent different states. This can be confusing for people who work on both platforms. So instead of using the default Tk bitmaps for the indicators, we are proposing to use our own image data which will display the same on all platforms.

  • develop a custom slicer widget as an extension of vtkKWCheckButton
  • disambiguate current "deslected" radiobutton icon state -- it can be read as "disabled"
  • override method for turning the old indicator visible/invisible (always keep it off)
  • develop visually parseable convention for icons to represent:
    • selected
    • deselected
    • disabled
  • use methods SetImageToIcon and SetSelectImageToIcon to display selected/deselected icons, selected/deselected icons+indicators
  • provide blank icons with and without indicators for people to download.

Sketch.png Link to Radio and check button tests


Working problem: Navigating with fiducials (Issue resolved: implemented in Editor module and Edit Box)

GOALS:

  • use fiducials as markers for features of interest
  • allow navigation to fiducial points within a slice window and
  • allow navigation to same fiducial point in all slice windows (a feature we're calling 'jumpslice').
  • want to read-out which fiducial is currently being viewed while navigating
  • want to expose these features in top level in each slice controller widget
  • want also to keep the slice scrollbar from getting too small to use.
  • want to indicate which slice scrollbar direction is R,L, S,I, A,P, depending on orientation


Sketch.png Link to Fiducial navigation UI design sketches


Working problem: Cine display for viewing and analyzing multi-volume data in Slicer3

Sketch.png Link to Drafts of UI for Cine Display


Working problem: Linked controls in Slicer3

Goals

Slice viewing can operate in the context of a single slice reformation or in a linked state where multiple slice viewers respond to interactions with any slice viewer. When Slice Viewers are linked, a number of viewing geometry and view representation parameters are linked across viewers.

Sketch.png Link to Drafts of UI for Linked Controls


Background: Slicer 3.2 usability & ideas for Slicer 3.3

Some Embodiment History


Return to Slicer3 Interface Design and Usability

Retrieved from "https://www.slicer.org/w/index.php?title=Slicer3:UIDesign&oldid=62247"