Difference between revisions of "Documentation/Nightly/Developers/QtTesting"

From Slicer Wiki
Jump to: navigation, search
Tag: 2017 source edit
 
(12 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 +
<noinclude>{{documentation/versioncheck}}</noinclude>
 
{|width="100%"
 
{|width="100%"
 
|
 
|
Line 6: Line 7:
 
|}
 
|}
 
=Overview=
 
=Overview=
 +
 +
{{documentation/banner
 +
| text  = Recording of macro using QtTesting should be considered experimental. <br> We suggest to implement workflow automation through Python scripts operating at a lower level by changing properties of MRML nodes and calling module logic functions. See https://discourse.slicer.org/t/macro-xml-to-python/15803 }}
  
 
QtTesting provides us a testing framework to test our application. It complements unit tests, by testing the application in its globality.
 
QtTesting provides us a testing framework to test our application. It complements unit tests, by testing the application in its globality.
Line 14: Line 18:
  
 
QtTesting records high level events, and try to not record mouse events as mouse press, release or move.<br>
 
QtTesting records high level events, and try to not record mouse events as mouse press, release or move.<br>
Furthermore, QtTesting is still under heavy development and and some low level event won't be properly recorded.
+
Furthermore, QtTesting is still under heavy development and some low level events won't be properly recorded.<br>
 +
 
 +
'''Please be aware of those following warning :'''
 +
 
 +
* Use '''''"activated"''''' signal instead of '''''"clicked"''''', when connection Qt views
  
# use "activated" signal instead of "clicked"
+
=How QtTesting and CTK work=
 +
 
 +
{|width="100%"
 +
|align="left"|
 +
[[Image: ctkQtTestingSchemaPlay.png|550px|]]
 +
|align="left"|
 +
[[Image: ctkQtTestingSchemaRecord.png|550px]]
 +
|-
 +
|'''''How it works :'''''
 +
When pqTestUtility starts the playback, it calls pqEvenDispatcher to play a specific script. Then
 +
pqEventDispatcher finds an eventSource able to read the script. Then pqEventDispatcher forwards each
 +
event to the pqEventPlayer. Finally, this last class tries to find the best player to playback the action into the
 +
application. If the action has been done, we start over and over the same process until the end. If there is
 +
one error, or an action couldn’t be handled, the player fails with an error.
 +
|'''''How it works :'''''
 +
When pqTestUtility starts to record events, it finds the observer which will write the file, creates the file,
 +
and starts the pqEventRecorder.
 +
Then every time there is a user action in the application, a specific translator will catch it and transform it
 +
into an high level event. This high level event is sent to the pqEventTranslator, and then to the
 +
pqEventObserver which writes the action recorded into a specific language, XML for example.
 +
|}
  
 
=Write a 3DSlicer test using QtTesting=
 
=Write a 3DSlicer test using QtTesting=
  
QtTesting framework can also be use to create module test.<br> Use the description below, but instead of <code>Slicer4/Applications/SlicerApp/Testing</code> all the path start by <code>Slicer4/Path/To/The/Modules/Testing</code>
+
QtTesting framework can also be use to create module test.<br> Use the description below, but instead of <code>Slicer/Applications/SlicerApp/Testing</code> all the path start by <code>Slicer/Path/To/The/Modules/Testing</code>
  
 
Following is the different step to create a test using QtTesting framework.  
 
Following is the different step to create a test using QtTesting framework.  
  
 
== Create your XML script ==
 
== Create your XML script ==
See [http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/SlicerApplication/QtTesting QtTesting user Wiki] to know how to create this xml script.<br>
+
See [[Documentation/Nightly/SlicerApplication/QtTesting|QtTesting user Wiki]] to know how to create this xml script.<br>
'''Save the xml script to''' <code>Slicer4/Applications/SlicerApp/Data/Input/MYTUTORIALNAME.xml''</code><br>
+
'''Save the xml script to''' <code>Slicer/Applications/SlicerApp/Data/Input/MYTUTORIALNAME.xml''</code><br>
 
{{ambox  
 
{{ambox  
 
| type = content
 
| type = content
|text = If you needs data, please first add the data to the Sample Data module, and use this module during the test. See instruction [http://www.slicer.org/slicerWiki/index.php/SampleData here]<br>
+
|text = If you need data, please first add the data to the Sample Data module, and use this module during the test. See instruction [[SampleData|here]]<br>
 
[[Documentation/{{documentation/version}}/Modules/SampleData Sample Data]] module according to the current Slicer version
 
[[Documentation/{{documentation/version}}/Modules/SampleData Sample Data]] module according to the current Slicer version
 
}}
 
}}
Line 47: Line 75:
 
       raise Exception('Failed to finished properly the play back !')
 
       raise Exception('Failed to finished properly the play back !')
  
'''''Save to the directory ''''' <code>Slicer4/Applications/SlicerApp/Testing/Python/MYTUTORIALNAME.py </code>
+
'''''Save to the directory ''''' <code>Slicer/Applications/SlicerApp/Testing/Python/MYTUTORIALNAME.py </code>
  
 
== Add the test in CMake ==
 
== Add the test in CMake ==
 
Adding the test in CMake will allow the test to be run with CTest and will also be run by the dashboard machines every nights.
 
Adding the test in CMake will allow the test to be run with CTest and will also be run by the dashboard machines every nights.
  
Edit the file <code>Slicer4/Applications/SlicerApp/Testing/Python/CMakeLists.txt</code><br>
+
Edit the file <code>Slicer/Applications/SlicerApp/Testing/Python/CMakeLists.txt</code><br>
 
Add your test at the end of the Slicer_USE_QTTESTING condition and add the name in the <code>set_tests_properties</code> function :
 
Add your test at the end of the Slicer_USE_QTTESTING condition and add the name in the <code>set_tests_properties</code> function :
 
   
 
   
Line 86: Line 114:
 
| colspan=2; valign="top" |
 
| colspan=2; valign="top" |
 
You need to open a terminal.<br>Then type in the terminal:<br><br>
 
You need to open a terminal.<br>Then type in the terminal:<br><br>
  $ cd path/to/Slicer4-Superbuild/Slicer-build
+
  $ cd path/to/Slicer-Superbuild/Slicer-build
 
  $ ctest -R MYTUTORIALNAME -VV
 
  $ ctest -R MYTUTORIALNAME -VV
 
| valign="top" |
 
| valign="top" |
Line 92: Line 120:
 
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt<br>
 
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt<br>
 
Then type in the terminal:<br>
 
Then type in the terminal:<br>
  $ cd path\to\Slicer4-Superbuild\Slicer-build
+
  $ cd path\to\Slicer-Superbuild\Slicer-build
 
  $ path\to\cmake\ctest.exe -R MYTUTORIALNAME -VV
 
  $ path\to\cmake\ctest.exe -R MYTUTORIALNAME -VV
 
|}
 
|}
Line 104: Line 132:
 
| colspan=2; valign="top" |
 
| colspan=2; valign="top" |
 
You need to open a terminal.<br>Then type in the terminal:<br><br>
 
You need to open a terminal.<br>Then type in the terminal:<br><br>
  $ cd path/to/Slicer4/Application/SlicerApp/Testing/Python
+
  $ cd path/to/Slicer/Application/SlicerApp/Testing/Python
 
  $ python MYTUTORIALNAME.py
 
  $ python MYTUTORIALNAME.py
 
| valign="top" |
 
| valign="top" |
Line 110: Line 138:
 
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt<br>
 
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt<br>
 
Then type in the terminal:<br>
 
Then type in the terminal:<br>
  $ cd path\to\Slicer4\Applicaiton\SlicerApp\Testing\Python
+
  $ cd path\to\Slicer\Applicaiton\SlicerApp\Testing\Python
  $ path\to\Slicer4-Superbuild\python-build\PCBuild\python.exe MYTUTORIALNAME.py
+
  $ path\to\Slicer-Superbuild\python-build\PCBuild\python.exe MYTUTORIALNAME.py
 
|}
 
|}
  
Line 128: Line 156:
 
| colspan=2; valign="top" |
 
| colspan=2; valign="top" |
 
You need to open a terminal.<br>Then type in the terminal:<br><br>
 
You need to open a terminal.<br>Then type in the terminal:<br><br>
  $ cd path/to/Slicer4-Superbuild/CTK-build/QtTesting/Utilities/Scripts
+
  $ cd path/to/Slicer-Superbuild/CTK-build/QtTesting/Utilities/Scripts
 
  $ python TranslatorPlayerWizard.py MY_WIDGET_NAME
 
  $ python TranslatorPlayerWizard.py MY_WIDGET_NAME
 
  $ cd ../MY_WIDGET_NAME
 
  $ cd ../MY_WIDGET_NAME
Line 135: Line 163:
 
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt<br>
 
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt<br>
 
Then type in the terminal:<br>
 
Then type in the terminal:<br>
  $ cd path\to\Slicer4-Superbuild
+
  $ cd path\to\Slicer-Superbuild
 
  $ python-build\PCBuild\python.exe CTK-build\QtTesting\Utilities\Scripts\TranslatorPlayerWizard.py MY_WIDGET_NAME
 
  $ python-build\PCBuild\python.exe CTK-build\QtTesting\Utilities\Scripts\TranslatorPlayerWizard.py MY_WIDGET_NAME
 
  $ cd ../MY_WIDGET_NAME
 
  $ cd ../MY_WIDGET_NAME
Line 151: Line 179:
 
Finally move the 4 files to the same directory as your widget
 
Finally move the 4 files to the same directory as your widget
  
  $ cd path/to/Slicer4-Superbuild/CTK-build/QtTesting/Utilities/MY_WIDGET_NAME
+
  $ cd path/to/Slicer-Superbuild/CTK-build/QtTesting/Utilities/MY_WIDGET_NAME
 
  $ mv MY_WIDGET_NAMETranslator.cpp MY_WIDGET_NAMETranslator.h MY_WIDGET_NAMEPlayer.cpp MY_WIDGET_NAMEPlayer.h Path/To/My/Widget/Directory
 
  $ mv MY_WIDGET_NAMETranslator.cpp MY_WIDGET_NAMETranslator.h MY_WIDGET_NAMEPlayer.cpp MY_WIDGET_NAMEPlayer.h Path/To/My/Widget/Directory
  

Latest revision as of 19:34, 26 April 2022

Home < Documentation < Nightly < Developers < QtTesting


For the latest Slicer documentation, visit the read-the-docs.


3DSlicerQtTesting.png

Overview

Recording of macro using QtTesting should be considered experimental.
We suggest to implement workflow automation through Python scripts operating at a lower level by changing properties of MRML nodes and calling module logic functions. See https://discourse.slicer.org/t/macro-xml-to-python/15803

QtTesting provides us a testing framework to test our application. It complements unit tests, by testing the application in its globality.

Following is a step by step guide to create slicer unit test, or module unit test, using this framework, which can be run as part of a nightly build automatic testing.

Rules for Developers

QtTesting records high level events, and try to not record mouse events as mouse press, release or move.
Furthermore, QtTesting is still under heavy development and some low level events won't be properly recorded.

Please be aware of those following warning :

  • Use "activated" signal instead of "clicked", when connection Qt views

How QtTesting and CTK work

CtkQtTestingSchemaPlay.png

CtkQtTestingSchemaRecord.png

How it works :

When pqTestUtility starts the playback, it calls pqEvenDispatcher to play a specific script. Then pqEventDispatcher finds an eventSource able to read the script. Then pqEventDispatcher forwards each event to the pqEventPlayer. Finally, this last class tries to find the best player to playback the action into the application. If the action has been done, we start over and over the same process until the end. If there is one error, or an action couldn’t be handled, the player fails with an error.

How it works :

When pqTestUtility starts to record events, it finds the observer which will write the file, creates the file, and starts the pqEventRecorder. Then every time there is a user action in the application, a specific translator will catch it and transform it into an high level event. This high level event is sent to the pqEventTranslator, and then to the pqEventObserver which writes the action recorded into a specific language, XML for example.

Write a 3DSlicer test using QtTesting

QtTesting framework can also be use to create module test.
Use the description below, but instead of Slicer/Applications/SlicerApp/Testing all the path start by Slicer/Path/To/The/Modules/Testing

Following is the different step to create a test using QtTesting framework.

Create your XML script

See QtTesting user Wiki to know how to create this xml script.
Save the xml script to Slicer/Applications/SlicerApp/Data/Input/MYTUTORIALNAME.xml

Create a Python script

This Python test will start 3DSlicer and run the previously recorded xml script

Create a python file with the following lines

   import os
   import slicer
   import ApplicationsSlicerAppData as data
   filepath = data.input + '/MYTUTORIALNAME.xml')     ex : NeurosurgicalPlanningTutorial.xml
   testUtility = slicer.app.testingUtility()
   success = testUtility.playTests(filepath)
   if not success :
     raise Exception('Failed to finished properly the play back !')

Save to the directory Slicer/Applications/SlicerApp/Testing/Python/MYTUTORIALNAME.py

Add the test in CMake

Adding the test in CMake will allow the test to be run with CTest and will also be run by the dashboard machines every nights.

Edit the file Slicer/Applications/SlicerApp/Testing/Python/CMakeLists.txt
Add your test at the end of the Slicer_USE_QTTESTING condition and add the name in the set_tests_properties function :

   if(Slicer_USE_QTTESTING)
     {   
     ...
     slicer_add_python_test(
       SCRIPT MYTUTORIALNAME.py
       SLICER_ARGS --launcher-no-splash --qt-testing)
     set_tests_properties(
       py_NeurosurgicalPlanningTutorial
       py_DiffusionTensorImagingTutorial
       ....
       py_MYTUTORIALNAME
       ...
       PROPERTIES RUN_SERIAL ON
       )
     }


You just wrote a Slicer Unit Test using QtTesting framework !

Run your test

You can either use directly CTest or the pyhton script created.

Using CTest

Linux Mac Windows

You need to open a terminal.
Then type in the terminal:

$ cd path/to/Slicer-Superbuild/Slicer-build
$ ctest -R MYTUTORIALNAME -VV

You need to open a terminal.
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt
Then type in the terminal:

$ cd path\to\Slicer-Superbuild\Slicer-build
$ path\to\cmake\ctest.exe -R MYTUTORIALNAME -VV

Using the python script

Linux Mac Windows

You need to open a terminal.
Then type in the terminal:

$ cd path/to/Slicer/Application/SlicerApp/Testing/Python
$ python MYTUTORIALNAME.py

You need to open a terminal.
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt
Then type in the terminal:

$ cd path\to\Slicer\Applicaiton\SlicerApp\Testing\Python
$ path\to\Slicer-Superbuild\python-build\PCBuild\python.exe MYTUTORIALNAME.py

Create custom translator/player

After created a custom widget, it might be possible that all its actions are not properly recorded, or played.
QtTesting provides us basic translator and player template, to create our custom translator and player according to the widget.

You can use the following command to create those basic files.

Linux Mac Windows

You need to open a terminal.
Then type in the terminal:

$ cd path/to/Slicer-Superbuild/CTK-build/QtTesting/Utilities/Scripts
$ python TranslatorPlayerWizard.py MY_WIDGET_NAME
$ cd ../MY_WIDGET_NAME

You need to open a terminal.
Start -> Microsoft Visual Studio 2008 -> Visual Studio Tools -> Visual Studio 2008 Command Prompt
Then type in the terminal:

$ cd path\to\Slicer-Superbuild
$ python-build\PCBuild\python.exe CTK-build\QtTesting\Utilities\Scripts\TranslatorPlayerWizard.py MY_WIDGET_NAME
$ cd ../MY_WIDGET_NAME

Finally move the 4 files to the same directory as your widget

$ cd path/to/Slicer-Superbuild/CTK-build/QtTesting/Utilities/MY_WIDGET_NAME
$ mv MY_WIDGET_NAMETranslator.cpp MY_WIDGET_NAMETranslator.h MY_WIDGET_NAMEPlayer.cpp MY_WIDGET_NAMEPlayer.h Path/To/My/Widget/Directory

or just a simple copy to the same directory as your widget

$ cp MY_WIDGET_NAMETranslator.cpp MY_WIDGET_NAMETranslator.h MY_WIDGET_NAMEPlayer.cpp MY_WIDGET_NAMEPlayer.h Path/To/My/Widget/Directory

Open bugs

Work in progress, following the two bug trackers :

Slicer issues

QtTesting issues