Grundlett: Text replacement - "(.)@(.)<\/email>" to "$1{{@}}$2"

2019-11-27T05:26:06Z

Text replacement - "<email>(.*)@(.*)<\/email>" to "$1{{@}}$2"

UpdateBot: Nightly -> 4.10

2018-10-19T00:43:34Z

Nightly -> 4.10

New page

== Release planning ==

* Decide what will be the version number for the next stable release
* Update issue tracker: https://issues.slicer.org/roadmap_page.php
* Define a release date
* Announce the release: https://discourse.slicer.org/t/upcoming-slicer-4-8-release/1120

== Day 1: Release ==

Since there all development occurs on <code>master</code>, each time version is updated, two commits will be required.

=== CMakeLists.txt: Update the Slicer version information for the release ===

<ol start="1" style="list-style-type: decimal;">

<li>Update the Slicer version information for the release:
<ol style="list-style-type: lower-roman;">
<li>In <code>CMakeLists.txt</code>, update at least one these variables: <code>Slicer_VERSION_MAJOR</code>, <code>Slicer_VERSION_MINOR</code>, <code>Slicer_VERSION_PATCH</code>
</li>
<li>
Re-run CMake with <code>-DSlicer_RELEASE_TYPE:STRING=Stable</code>. This is required to update <code>Utilities/Scripts/SlicerWizard/__version__.py</code>.
{{note}}If not doing a clean build, make sure to explicitly specify the version re-configuring with <code>-DSlicer_VERSION_MAJOR:STRING=X -DSlicer_VERSION_MINOR:STRING=Y -DSlicer_VERSION_PATCH:STRING=Z .</code>
</li>
<li>Commit the above changes using this message like:
ENH: Slicer X.Y.Z
</li>
<li>Keep track of the <tt>SVN_REVISION</tt> and <tt>SVN_BRANCH</tt> for the next steps</li>
</ol>
</li>

</ol>

=== CMakeLists.txt: Update the Slicer version information for the development ===

<ol start="2" style="list-style-type: decimal;">
<li>Update the Slicer version information for the development:
<ol style="list-style-type: lower-roman;">
<li>In <code>CMakeLists.txt</code>,
* Update <code>Slicer_VERSION_MAJOR</code> and/or <code>Slicer_VERSION_MINOR</code>
</li>
<li>
Re-run CMake with <code>-DSlicer_RELEASE_TYPE:STRING=Experimental</code>. This is required to update <code>Utilities/Scripts/SlicerWizard/__version__.py</code>.
{{note}}If not doing a clean build, make sure to explicitly specify the version re-configuring with <code>-DSlicer_VERSION_MAJOR:STRING=X -DSlicer_VERSION_MINOR:STRING=Y -DSlicer_VERSION_PATCH:STRING=Z .</code>
</li>
<li>Commit the above changes using this message like:
cd ~/Projects/Slicer
git add -A
git commit -m "ENH: Begin X.Y.Z development"
</li>
</ol>
</li>

</ol>

=== Tag and publish SlicerBuildEnvironment docker image ===

<ol start="3" style="list-style-type: decimal;">
<li>
Tag the docker build environment image. See https://github.com/Slicer/SlicerBuildEnvironment/blob/master/README.rst#maintainers
{{note}}If the environment did not change, you may skip this step for patch release.
Keep track of the selected <tt>TAG</tt> for updating the release scripts
</li>
</ol>

=== Update release scripts ===

<ol start="4" style="list-style-type: decimal;">
<li>Update release scripts. See https://github.com/Slicer/DashboardScripts#maintenance-guides</li>
</ol>

=== Disable regular nightly builds ===

<ol start="5" style="list-style-type: decimal;">
<li>On each factory machines, disable regular nightly build in crontab and task scheduler.</li>
</ol>

=== Update ExtensionsIndex ===

This applies to https://github.com/Slicer/ExtensionsIndex

<ol start="6" style="list-style-type: decimal;">
<li>Create branch X.Y based of master:
DEST_VERSION=X.Y

cd /tmp
git clone git@github.com:Slicer/ExtensionsIndex
cd ExtensionsIndex
git checkout master
git push origin master:${DEST_VERSION}
</li>
</ol>

=== Update CDash ===

This applies to the CDash instance associated with http://slicer.cdash.org/index.php?project=Slicer4

<ol start="7" style="list-style-type: decimal;">
<li>Create new CDash groups for extension submissions associated with <code>X.Y</code> release:
<pre>
Extensions-X.Y-Nightly
Extensions-X.Y-Continuous
</pre></li>

</ol>

=== Generate application and extension packages ===

<ol start="8" style="list-style-type: decimal;">
<li>Generate packages running Slicer package scripts on each factory machines. (These are the scripts updated in the previous step)</li>
</ol>

=== Tag the repository ===

<ol start="9" style="list-style-type: decimal;">
<li>Tag the repository:
Set version variables:
X=<major-version>
Y=<minor-version>
Z=<patch-version>
echo "X.Y.Z is ${X}.${Y}.${Z}"

Set SVN revision variable:
SVN_REVISION=<svn-revision-for-release>

SVN:
cd ~/Projects
svn checkout http://svn.slicer.org/Slicer4/trunk Slicer-${X}.${Y}-SVN
cd Slicer-${X}.${Y}-SVN
svn copy -r${SVN_REVISION} http://svn.slicer.org/Slicer4/trunk \
http://svn.slicer.org/Slicer4/tags/Slicer-${X}-${Y} \
-m "ENH: Tag of ${X}.${Y}.${Z} release based on r${SVN_REVISION}."

GIT:
cd ~/Projects
git clone git@github.com:Slicer/Slicer Slicer-${X}.${Y}
cd Slicer-${X}.${Y}

#
# Rollback by one commit to ignore the "ENH: Begin X.Y.Z development" commit
#
git reset --hard HEAD^

#
# >>> Make sure the current commit corresponds to ${SVN_REVISION}. <<<
#
git log -1

git tag -s -m "ENH: Slicer ${X}.${Y}.${Z}" v${X}.${Y}.${Z} master
git push origin v${X}.${Y}.${Z}
</li>
</ol>

=== Create release branch ===

<ol start="10" style="list-style-type: decimal;">
<li>Create release branch:
Set version variables (same value as in the previous step):
X=<major-version>
Y=<minor-version>
Z=<patch-version>
echo "X.Y.Z is ${X}.${Y}.${Z}"

Set SVN revision variable (same value as in the previous step):
SVN_REVISION=<svn-revision-for-release>

SVN (Keep track of the branch revision. It will be referred below as <code>BRANCHED_REVISION</code>.):
cd ~/Projects/Slicer-${X}.${Y}-SVN
svn copy -r ${SVN_REVISION} http://svn.slicer.org/Slicer4/trunk http://svn.slicer.org/Slicer4/branches/Slicer-${X}-${Y} \
-m "ENH: Branching from trunk to Slicer-${X}-${Y} at r${SVN_REVISION}"

svn switch http://svn.slicer.org/Slicer4/branches/Slicer-${X}-${Y}

BRANCHED_REVISION=$(svn info | grep "Revision" | awk '{print $2}')
echo "BRANCHED_REVISION is ${BRANCHED_REVISION}"

GIT:
cd ~/Projects/Slicer-${X}.${Y}

# Source: http://ivanz.com/2009/01/15/selective-import-of-svn-branches-into-a-gitgit-svn-repository/
# and https://stackoverflow.com/questions/19712735/git-svn-cannot-setup-tracking-information-starting-point-is-not-a-branch

git config --add svn-remote.svn-${X}${Y}.url http://svn.slicer.org/Slicer4/branches/Slicer-${X}-${Y}
git config --add svn-remote.svn-${X}${Y}.fetch :refs/remotes/git-svn-${X}${Y}

git svn fetch svn-${X}${Y} -r ${BRANCHED_REVISION}

git checkout -b master-${X}${Y} remotes/git-svn-${X}${Y}

git svn rebase

git push origin master-${X}${Y}
</li>
</ol>

=== Update Release Details on the wiki ===

<ol start="11" style="list-style-type: decimal;">
<li>Update [[Release Details]] with release information</li>
</ol>

== Day 2: Post release ==

=== Re-enable regular nightly builds ===

On each factory machines, re-enable build in crontab and task scheduler.

=== Update Mantis ===

This applies to the Mantis instance associated with https://issues.slicer.org

* "Release" current target
* Create new target
* Check the "fixed in" fields

=== Update Slicer wiki ===

This applies to the MediaWiki instance associated with https://wiki.slicer.org

There are two steps involved:
* copy of the pages associated with the <code>Nightly</code> namespace into the <code>X.Y</code> namespace.
* update of permanent pages referencing the current Slicer version

As of November 2016, the process is fully automated and can be done using [https://github.com/Slicer/Slicer/tree/master/Base/Python/slicer/release/wiki.py Base/Python/slicer/release/wiki.py] script.

1) Install prerequisites:

* Check with <email>jchris.fillionr@kitware.com</email> to get the credential associated with <code>UpdateBot</code> user.
* Install [https://pypi.python.org/pypi/mwdoc mwdoc]:

<pre>
$ pip install mwdoc
</pre>

2) Query current version information:

<pre>
$ python Base/Python/slicer/release/wiki.py query
</pre>

3) Copy and update pages

<pre>
$ DEST_VERSION=X.Y

$ python Base/Python/slicer/release/wiki.py update ${DEST_VERSION}
$ python Base/Python/slicer/release/wiki.py copy ${DEST_VERSION}
</pre>

The following pages have been updated:

* [[Template:Documentation/prevversion]]
* [[Template:Documentation/nextversion]]
* [[Template:Documentation/currentversion]]
* [[Template:Documentation/versionlist]]
* [[Template:Documentation/acknowledgments-versionlist]]
* [[Documentation]]
* [[Documentation/Release/Acknowledgments]]

The following pages with #REDIRECT have been updated:

* [[FAQ]]
* [[Documentation/Release]]
* [[Documentation/Release/Announcements]]
* [[Documentation/Release/Report a problem]]
* [[Documentation/UserTraining]]
* [[Documentation/UserFeedback]]
* [[Documentation/Release/SlicerApplication/HardwareConfiguration]]

=== Clean-up older nightly packages ===

This applies to the Midas instance associated with http://slicer.kitware.com

See https://gist.github.com/jcfr/ea9ef199bd5a3e071b8f

=== Manually sign packages ===

<ol start="1" style="list-style-type: decimal;">

<li>If needed, create a X.Y.Z folder in [http://slicer.kitware.com/midas3/folder/274 Slicer/Packages/Application/Release]</li>

<li>Download package from http://slicer.cdash.org/index.php?project=Slicer4</li>

<li>Sign the package following these instructions:
<ul>

<li>Windows: [[Documentation/Nightly/Developers/Windows_Code_Signing]]</li>
</ul></li>

<li>Upload the signed package using [[Documentation/Nightly/Developers/ReleaseProcess#Script_to_upload_Slicer_packages|upload-Slicer-to-slicer-kitware-com.cmake]].</li>

<li>Manually update the name of the item appending <code> [signed]</code>. Make sure to not update the bitstream name. For example: see <tt>Slicer-4.8.0-win-amd64.exe [signed]</tt> available [http://slicer.kitware.com/midas3/item/316852 here]</li>

<li>For future reference, copy the unsigned packages into the folder created above (Keep the default Create a reference to the existing item)</li>

</ol>

=== Tag release packages ===

This applies to the Midas instance associated with http://slicer.kitware.com

Since the Windows package was uploaded during the manual signing, it is already associated with the release tag. On the other hand, due to [[#Limitations_of_download.slicer.org|limitations]]
of http://download.slicer.org, the Linux and macOS packages must be downloaded and re-uploaded.

1) Download packages

2) Compute md5 sums and check they correspond at the sum associated with download link posted on [http://slicer.cdash.org/index.php?project=Slicer4 http://slicer.cdash.org].

2) Re-upload each packages using [[#Script_to_upload_Slicer_packages|upload-Slicer-to-slicer-kitware-com.cmake]] script.

=== Confirm that packages are tagged ===

To confirm that all three packages are tagged. The following URL should return three packages:

http://slicer.kitware.com/midas3/api/json?method=midas.slicerpackages.package.list&release=4.8.0

=== Version NA-MIC data tree ===

This applies to the Midas instance associated with http://slicer.kitware.com

<ol start="1" style="list-style-type: decimal;">

<li>If you do not have an account and/or if you do not belong to the <code>DataManager</code> group:

<ul>

<li>[[Documentation/{{documentation/version}}/Developers/Tutorials/ObtainExtensionServerApiKey|Create an account on the extension server and obtain an API Key]]. You will then use your midas login and api key to substitute <code><YOUR-MIDAS-LOGIN></code> and <code><YOUR-MIDAS-APIKEY></code> in the examples.</li>

<li>If not already done, go to [http://slicer.kitware.com/midas3/community/23 NA-MIC community] and click on <code>Join community</code></li>

<li>Send an email on the developer list asking to be added to the <code>DataManager</code> group on NA-MIC community. That will grant you read/write permissions to the <code>Data</code> folder and sub-folders.</li>

</ul></li>

<li>Install prerequisites
<pre>
$ mkvirtualenv -p python2.7 version_namic_datatree
$ pip install pydas
</pre></li>

<li>Confirm that the <code>id</code> of the <code>Data</code> folder is still [http://slicer.kitware.com/midas3/folder/301 301].</li>

<li>Simulate creation of <code>X.Y</code> data folders based <code>Nightly</code> ones
<pre>
MIDAS_EMAIL=<YOUR-MIDAS-LOGIN>
MIDAS_API_KEY=<YOUR-MIDAS-APIKEY>
DEST_VERSION=X.Y
DATA_FOLDER_ID=301
</pre>

<pre>
$ cd /path/to/Slicer/Base/Python/slicer/release
$ python midasdata.py --url=http://slicer.kitware.com/midas3 --data_id=$DATA_FOLDER_ID \
--email=$MIDAS_EMAIL --apikey=$MIDAS_API_KEY --source_version=Nightly --dest_version=$DEST_VERSION --dry-run
</pre>

Example of output:
<pre>
Application ( folder_id: 302 )
'-Nightly ( folder_id: 831 )
'-'-Testing ( folder_id: 832 )
'-'-'-Baseline ( folder_id: 889 )
'-'-'-'-DiffusionTensorImagingTutorial.png ( item_id: 12067 )
'-'-'-'-NeurosurgicalPlanningTutorial.png ( item_id: 12066 )
'-'-'-'-SlicerTestingTest.png ( item_id: 17760 )
'-'-'-Input ( folder_id: 833 )
'-'-'-'-AtlasTests ( folder_id: 834 )
'-'-'-'-'-2012-10-26-BrainAtlas.mrb ( item_id: 10276 )
[...]
Module: VotingBinaryHoleFillingImageFilter ( folder_id: 1491 )
'-Nightly ( folder_id: 1491 )
'-'-Testing ( folder_id: 1492 )
'-'-'-Baseline ( folder_id: 1493 )
'-'-'-'-VotingBinaryHoleFillingImageFilterTest.nhdr ( item_id: 103418 )
'-'-'-'-VotingBinaryHoleFillingImageFilterTest.raw.gz ( item_id: 103419 )

</pre>
</li>

<li>Create <code>X.Y</code> data tree based <code>Nightly</code> ones
<pre>
$ python midasdata.py --url=http://slicer.kitware.com/midas3 --data_id=301 \
--email=$MIDAS_EMAIL --apikey=$MIDAS_API_KEY --source_version=Nightly --dest_version=$DEST_VERSION

Versioning of the NA-MIC Data tree for release X.Y...
Versioning Modules...
[...]
Versioning Modules...[DONE]
Versioning Application...
Creating folder X.Y under Application directory
Duplicating subfolders from Nightly to X.Y...
Duplicating subfolders from Nightly to X.Y...[DONE]
Versioning Application...[DONE]
Versioning of the NA-MIC Data tree for release X.Y...[DONE]

</pre></li>

</ol>

=== Update external website ===

* http://en.wikipedia.org/wiki/3DSlicer
* http://www.nitrc.org/projects/slicer/

=== Update ExtensionStats extension ===

Update https://github.com/ihnorton/SlicerDeveloperToolsForExtensions/blob/5ceceabbb3ae35a669309389aa34e596699ca058/ExtensionStats/ExtensionStats.py#L322-L340



== Appendix ==

=== Sign packages ===

==== MacOSX ====
As discussed in issue [http://na-mic.org/Bug/view.php?id=3965 #3965], the code signing command is of the form:

<pre>
codesign --deep --force --sign "Developer ID Application: <Name-of-certificate-in-keychain>" ./Slicer.app
</pre>

where the argument to <code>--sign</code> is the name of the developer's certificate in Mac OS X Keychain.app .

'''Note: The signing of MacOSX package is not yet implemented in our nightly and release build workflows.''' See [[Documentation/Nightly/Developers/Mac_OS_X_Code_Signing]] for more information.

==== Windows ====

TBD - See [http://na-mic.org/Bug/view.php?id=2697 #2697]

Also see [[Documentation/Nightly/Developers/Windows_Code_Signing]] for more information.

=== Backport commit into release branch ===

The following steps will lead to the creation of new git-svn clone having two branches:
* <code>git-svn</code>
* <code>git-svn-XY</code>

Reference: http://www.dmo.ca/blog/20070608113513/

==== Step 1: Checkout sources ====

git clone git://github.com/Slicer/Slicer.git Slicer-X.Y
cd Slicer-X.Y
git svn init http://svn.slicer.org/Slicer4/trunk
git update-ref refs/remotes/git-svn refs/remotes/origin/master
git checkout master
git svn rebase

==== Step 2: Add release branch remote ====

Edit <code>.git/config</code>, and in addition to the existing 'git-svn' remote, add the following one:

[svn-remote "svn-XY"]
url = http://svn.slicer.org/Slicer4/branches/Slicer-X-Y
fetch = :refs/remotes/git-svn-XY

==== Step 3: Pull associated SVN branch ====

git svn fetch svn-XY
git checkout -b master-XY git-svn-XY

==== Step 4: Backport ====

We can now cherry pick commit associated with master (trunk) into <code>master-XY</code> (Slicer-X-Y)



=== Script to upload Slicer packages ===

<pre>
File: upload-Slicer-to-slicer-kitware-com.cmake
</pre>

<pre>
cmake_minimum_required(VERSION 3.9.0)

#
# To run this script:
#
# cmake -P upload-Slicer-to-slicer-kitware-com.cmake
#

#
# Variables to update:
#
# CMAKE_MODULE_PATH
#
# release
# source_revision
# source_checkoutdate
# MIDAS_PACKAGE_EMAIL
# MIDAS_PACKAGE_API_KEY
#

list(APPEND CMAKE_MODULE_PATH
D:/D/N/Slicer-1/CMake
)
include(MIDASAPIUploadPackage)

set(operating_system "win") # linux, macosx, win

set(release "4.8.0")
set(source_revision "26489")
set(source_checkoutdate "2017-10-18 05:11:33")
set(MIDAS_PACKAGE_EMAIL "<YOUR-MIDAS-LOGIN>")
set(MIDAS_PACKAGE_API_KEY "<YOUR-MIDAS-APIKEY>")

set(ext_linux "tar.gz")
set(ext_macosx "dmg")
set(ext_win "exe")

set(package_filepath "C:/Users/dashboard/Downloads/Slicer-${release}-${operating_system}-amd64.${ext_${operating_system}}")

message(STATUS "Upload ${package_filepath}")

set(MIDAS_PACKAGE_URL http://slicer.kitware.com/midas3)
set(submission_type "experimental")
set(architecture "amd64")
set(package_type "installer")

midas_api_upload_package(
SERVER_URL ${MIDAS_PACKAGE_URL}
SERVER_EMAIL ${MIDAS_PACKAGE_EMAIL}
SERVER_APIKEY ${MIDAS_PACKAGE_API_KEY}
SUBMISSION_TYPE ${submission_type}
SOURCE_REVISION ${source_revision}
SOURCE_CHECKOUTDATE ${source_checkoutdate}
OPERATING_SYSTEM ${operating_system}
ARCHITECTURE ${architecture}
PACKAGE_FILEPATH ${package_filepath}
PACKAGE_TYPE ${package_type}
RELEASE ${release}
RESULT_VARNAME output
)
set(expected_output "ok")
if(NOT "${output}" STREQUAL "${expected_output}")
message(FATAL_ERROR "Problem with midas_api_upload_package()\n"
"output:${output}\n"
"expected_output:${expected_output}")
endif()
</pre>

=== Limitations of download.slicer.org ===

Considering that

(1) few minutes after a package is uploaded on http://slicer.kitware.com, it is indexed in the database associated with http://download.slicer.org, and 
(2) the system indexing the package does NOT rewrite its database entries,

using the [[#Explicit_tagging_of_release_packages|steps below]] will NOT result in an update of the packages reported on http://download.slicer.org.

Source code available at https://github.com/mhalle/slicer4-download.

=== Explicit tagging of release packages ===

Steps:

* If needed, create a X.Y.Z folder in [http://slicer.kitware.com/midas3/folder/274 Slicer/Packages/Application/Release]
** Copy uploaded packages into the folder created above (Keep the default Create a reference to the existing item)
** Identify the item_id associated with uploaded packages. For example: 316530, 316529
** From Kitware internal network, SSH connect to <code>jcfr@slicer.kitware.com</code> (or <code>jcfr@192.168.120.210</code>)
** Connect to mysql
*** <code>password=$(cat /var/www/midas3/core/configs/database.local.ini | grep database.params.password | cut -d"=" -f2 | head -n1 | sed 's/"//g')</code>
*** <code>mysql -u midas -p$password -D midas</code>
*** See file <code>/var/www/midas3/core/configs/database.local.ini</code> for password
*** Selected database is midas (<code>use midas</code>)
** List packages associated with identified items and check they are the appropriate ones:
*** <code>select * from slicerpackages_package as p , item as i where i.item_id = p.item_id and p.item_id in (316530, 316529);</code>
** Set release field
*** <code>update slicerpackages_package set `release`="4.8.0" where item_id in (316530, 316529);</code>

=== User Survey (not used anymore) ===

See https://docs.google.com/forms/d/1rggxoTV5KL_vt3gX9BNNAcKH4pnL5lkMqPnklOThINg

With each release, the title of the survey had to be updated.

Now, the Slicer welcome module has been updated to ask user to report feedback on the Slicer forum

@@ Line 225: / Line 225: @@
 ) Install prerequisites:
-* Check with <email>jchris.fillionr@kitware.com</email> to get the credential associated with <code>UpdateBot</code> user.
+* Check with jchris.fillionr{{@}}kitware.com to get the credential associated with <code>UpdateBot</code> user.
 * Install [https://pypi.python.org/pypi/mwdoc mwdoc]:

Documentation/4.10/Developers/ReleaseProcess - Revision history

Grundlett: Text replacement - "(.*)@(.*)<\/email>" to "$1{{@}}$2"

UpdateBot: Nightly -> 4.10

Grundlett: Text replacement - "(.)@(.)<\/email>" to "$1{{@}}$2"