Documentation/Labs/BuildSystem ImproveCMakeConfigurationTime

From Slicer Wiki
Jump to: navigation, search
Home < Documentation < Labs < BuildSystem ImproveCMakeConfigurationTime

CMake performance Analysis

During the 31st 3D Slicer Project Week, we found a loss of performance in CMake for the generation of Slicer targets in the operating system Windows. The performance loss means the time for the generation of the target is larger than the operating system Linux. This page analysises the performance loss in Windows in several computers: two desktops and a laptop. The computer configurations are shown in the table below.

Desktop1 Desktop2 Surface
Processor Intel i7-6700 CPU 3.40GHz Intel i7-6700 CPU 3.40GHz Intel i7-8650U CPU 2.11GHz
RAM 16GB 16GB 16GB
Disk storage Hard disk Drive (HDD) Solid State Drive (SSD) Solid State Drive (SSD)
Operating system Windows 10 x64 Windows 10 x64 Windows 10 x64


Enviroment

The test enviroment uses the CMake version 3.14.4 and the master branch of Slicer repository of the fork mt4sd link.

The 21 tests were generated in different configuration variables. All tests were executed in the folder Slicer-SuperBuild/Slicer-build after the Slicer had been built.

The modified variables are: Slicer_BUILD_CL, Slicer_BUILD_I18N_SUPPORT, Slicer_USE_PYTHONQT, Slicer_BUILD_QTLOADABLEMODULES, Slicer_BUILD_QTSCRIPTEDMODULES, Slicer_EP_LABEL_REMOTE_MODULE, CMAKE_CONFIGURATION_TYPES. The internationalization(I18n) is not activated any tests.

Results

The table below shows the command line and the times for each test.

Test name Command Line Desktop1 Desktop2 Surface
test1 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 411.44 233.75 234.09
test2 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel" . 277.93 174.74 206.47
test3 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Debug;Release" . 206.36 151.39 170.40
test4 cmake Slicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Debug" . 140.92 103.25 122.25
test5 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Release" . 142.92 88.69 117.91
test6 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 328.13 206.30 244.59
test7 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 326.36 228.41 251.93
test8 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="MultiVolumeExplorer_SOURCE_DIR;MultiVolumeImporter_SOURCE_DIR;SimpleFilters_SOURCE_DIR;BRAINSTools_SOURCE_DIR;DataStore_SOURCE_DIR;CompareVolumes_SOURCE_DIR;LandmarkRegistration_SOURCE_DIR" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 315.46 233.08 241.44
test9 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 281.66 176.60 208.52
test10 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 106.35 58.53 80.26
test11 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel;RelWithDebInfo" . 64.51 42.02 39.71
test12 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug;Release;MinSizeRel" . 50.51 32.92 33.8
test13 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug;Release" . 39.71 24.23 28.11
test14 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug" . 27.96 15.76 22.38
test15 Slicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Release" . 28.04 15.76 22.28
test16 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="RelWithDebInfo" . 28.33 16.20 22.56
test17 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug" . 42.19 15.86 35.32
test18 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug" . 32.31 19.69 26.32
test19 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=0 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug" . 36.63 24.95 31.65
test20 cmake -DSlicer_BUILD_CLI=0 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=0 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=0 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug" . 98.78 65.97 75.91
test21 cmake -DSlicer_BUILD_CLI=1 -DSlicer_BUILD_I18N_SUPPORT=0 -DSlicer_USE_PYTHONQT=1 -DSlicer_BUILD_QTLOADABLEMODULES=1 -DSlicer_BUILD_QTSCRIPTEDMODULES=1 -DSlicer_EP_LABEL_REMOTE_MODULE="" -DCMAKE_CONFIGURATION_TYPES="Debug" . 131.82 86.52 109.27

The figure below shows the previous information in a graphic mode.

CMakePerformance.png

We observe several behaviours:

  • When the number of the targets is decreased, the time saving is meaningful. In the desktop 1, the time reduction is around 65% and is around 64% and 49% in the desktop2 and the surface respectively.
  • The hard disk technologies affect the time reduction. If the hard disk technology is SSD, the time reduction average is 29 %. Moreover, the time saving is larger test 1 (43%) than tests 4 y 5, because the number of files generated in test 1 is larger than tests 4 y 5 in the configuration four targets.
  • The variable Slicer_BUILD_QTLOADABLEMODULES has a relevant impact on generate time. The time reduction is between from 65% to 75% when the variable Slicer_BUILD_QTLOADABLEMODULES is not active and four targets. All remaining variables affect lesser in the reduction time.

Conclusions

The number of the targets, the hard disk technologies and the variable Slicer_BUILD_QTLOADABLEMODULES have an important impact on the the CMake configuration time. The time saving average is 59% when the number of targets is one. The time reduction average is 29 % if the SSD technology is used.