Documentation/Nightly/Developers/QtTesting

For the latest Slicer documentation, visit the read-the-docs.

Overview

Recording of macro using QtTesting should be considered experimental. We suggest to implement workflow automation through Python scripts operating at a lower level by changing properties of MRML nodes and calling module logic functions. See https://discourse.slicer.org/t/macro-xml-to-python/15803

QtTesting provides us a testing framework to test our application. It complements unit tests, by testing the application in its globality.

Following is a step by step guide to create slicer unit test, or module unit test, using this framework, which can be run as part of a nightly build automatic testing.

Rules for Developers

QtTesting records high level events, and try to not record mouse events as mouse press, release or move.
Furthermore, QtTesting is still under heavy development and some low level events won't be properly recorded.

Please be aware of those following warning :

• Use "activated" signal instead of "clicked", when connection Qt views

How QtTesting and CTK work

How it works : When pqTestUtility starts the playback, it calls pqEvenDispatcher to play a specific script. Then pqEventDispatcher finds an eventSource able to read the script. Then pqEventDispatcher forwards each event to the pqEventPlayer. Finally, this last class tries to find the best player to playback the action into the application. If the action has been done, we start over and over the same process until the end. If there is one error, or an action couldn’t be handled, the player fails with an error.

How it works : When pqTestUtility starts to record events, it finds the observer which will write the file, creates the file, and starts the pqEventRecorder. Then every time there is a user action in the application, a specific translator will catch it and transform it into an high level event. This high level event is sent to the pqEventTranslator, and then to the pqEventObserver which writes the action recorded into a specific language, XML for example.

Write a 3DSlicer test using QtTesting

QtTesting framework can also be use to create module test.
Use the description below, but instead of Slicer/Applications/SlicerApp/Testing all the path start by Slicer/Path/To/The/Modules/Testing

Following is the different step to create a test using QtTesting framework.

Create your XML script

See QtTesting user Wiki to know how to create this xml script.
Save the xml script to Slicer/Applications/SlicerApp/Data/Input/MYTUTORIALNAME.xml

Create a Python script

This Python test will start 3DSlicer and run the previously recorded xml script

Create a python file with the following lines

   import os
   import slicer
   import ApplicationsSlicerAppData as data
   filepath = data.input + '/MYTUTORIALNAME.xml')     ex : NeurosurgicalPlanningTutorial.xml
   testUtility = slicer.app.testingUtility()
   success = testUtility.playTests(filepath)
   if not success :
     raise Exception('Failed to finished properly the play back !')

Save to the directory Slicer/Applications/SlicerApp/Testing/Python/MYTUTORIALNAME.py

Add the test in CMake

Adding the test in CMake will allow the test to be run with CTest and will also be run by the dashboard machines every nights.

Edit the file Slicer/Applications/SlicerApp/Testing/Python/CMakeLists.txt
Add your test at the end of the Slicer_USE_QTTESTING condition and add the name in the set_tests_properties function :

   if(Slicer_USE_QTTESTING)
     {   
     ...
     slicer_add_python_test(
       SCRIPT MYTUTORIALNAME.py
       SLICER_ARGS --launcher-no-splash --qt-testing)
     set_tests_properties(
       py_NeurosurgicalPlanningTutorial
       py_DiffusionTensorImagingTutorial
       ....
       py_MYTUTORIALNAME
       ...
       PROPERTIES RUN_SERIAL ON
       )
     }

You just wrote a Slicer Unit Test using QtTesting framework !

Run your test

You can either use directly CTest or the pyhton script created.

Using CTest

Linux	Mac	Windows
You need to open a terminal. Then type in the terminal: $ cd path/to/Slicer-Superbuild/Slicer-build $ ctest -R MYTUTORIALNAME -VV	You need to open a terminal. Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt Then type in the terminal: $ cd path\to\Slicer-Superbuild\Slicer-build $ path\to\cmake\ctest.exe -R MYTUTORIALNAME -VV

Using the python script

Linux	Mac	Windows
You need to open a terminal. Then type in the terminal: $ cd path/to/Slicer/Application/SlicerApp/Testing/Python $ python MYTUTORIALNAME.py	You need to open a terminal. Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt Then type in the terminal: $ cd path\to\Slicer\Applicaiton\SlicerApp\Testing\Python $ path\to\Slicer-Superbuild\python-build\PCBuild\python.exe MYTUTORIALNAME.py

Create custom translator/player

After created a custom widget, it might be possible that all its actions are not properly recorded, or played.
QtTesting provides us basic translator and player template, to create our custom translator and player according to the widget.

You can use the following command to create those basic files.

Linux	Mac	Windows
You need to open a terminal. Then type in the terminal: $ cd path/to/Slicer-Superbuild/CTK-build/QtTesting/Utilities/Scripts $ python TranslatorPlayerWizard.py MY_WIDGET_NAME $ cd ../MY_WIDGET_NAME	You need to open a terminal. Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt Then type in the terminal: $ cd path\to\Slicer-Superbuild $ python-build\PCBuild\python.exe CTK-build\QtTesting\Utilities\Scripts\TranslatorPlayerWizard.py MY_WIDGET_NAME $ cd ../MY_WIDGET_NAME

Finally move the 4 files to the same directory as your widget

$ cd path/to/Slicer-Superbuild/CTK-build/QtTesting/Utilities/MY_WIDGET_NAME
$ mv MY_WIDGET_NAMETranslator.cpp MY_WIDGET_NAMETranslator.h MY_WIDGET_NAMEPlayer.cpp MY_WIDGET_NAMEPlayer.h Path/To/My/Widget/Directory

or just a simple copy to the same directory as your widget

$ cp MY_WIDGET_NAMETranslator.cpp MY_WIDGET_NAMETranslator.h MY_WIDGET_NAMEPlayer.cpp MY_WIDGET_NAMEPlayer.h Path/To/My/Widget/Directory

Open bugs

Work in progress, following the two bug trackers :

Slicer issues

QtTesting issues