Difference between revisions of "Slicer-3.6-QA"
Line 50: | Line 50: | ||
!Score!!Name!!Documentation!!Help <ref>including link to website documentation</ref>!!Acknowledgment <ref>including name of author and of supervisor</ref> !!Test coverage <ref>in % for non-interactive part</ref>!!valgrind errors!!Current Owner!!Comments <ref>Test coverage percentage determined by: [# lines code covered / total # lines of code (covered and non-covered) *100] of relevant .cxx and .h files</ref>!!3.6 Documentation Name <ref>http://wiki.slicer.org/slicerWiki/index.php/Documentation-3.6</ref> | !Score!!Name!!Documentation!!Help <ref>including link to website documentation</ref>!!Acknowledgment <ref>including name of author and of supervisor</ref> !!Test coverage <ref>in % for non-interactive part</ref>!!valgrind errors!!Current Owner!!Comments <ref>Test coverage percentage determined by: [# lines code covered / total # lines of code (covered and non-covered) *100] of relevant .cxx and .h files</ref>!!3.6 Documentation Name <ref>http://wiki.slicer.org/slicerWiki/index.php/Documentation-3.6</ref> | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"|my module||complete||yes and yes||yes, yes, yes||80%||0|| |||| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| DiffusionWeightedTest ||no||no||yes and yes|| 69.8% || 0||CF Westin||||n/a|| | |style="background:hotpink"| Hazard||style="width:33%"| DiffusionWeightedTest ||no||no||yes and yes|| 69.8% || 0||CF Westin||||n/a|| | ||
|- | |- | ||
− | |style="background:silver"| | + | |style="background:silver"| 2Silver||style="width:33%"| DiffusionTensorEstimation || complete ||yes and no||yes and yes|| 79.7% || 9 ||CF Westin||||same|| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| DiffusionTensorMathematics ||no||yes and no||yes and yes|| 56.2% || 5||CF Westin||||Diffusion Tensor Scalar Measurements|| | |style="background:hotpink"| Hazard||style="width:33%"| DiffusionTensorMathematics ||no||yes and no||yes and yes|| 56.2% || 5||CF Westin||||Diffusion Tensor Scalar Measurements|| | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| OrientImage ||complete||yes and yes||yes and yes|| 85.7% || 0 ||Bill Lorensen||||OrientImages|| |
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| LinearRegistration ||complete||yes and no||yes and yes|| 66.7% || 4 ||Daniel Blezek||||same|| |
|- | |- | ||
|style="color:white; background:black"|Coal||style="width:33%"| RigidRegistration ||complete||yes and yes||yes and yes||style="background:red"| 1.0% || 92 ||Daniel Blezek||||same|| | |style="color:white; background:black"|Coal||style="width:33%"| RigidRegistration ||complete||yes and yes||yes and yes||style="background:red"| 1.0% || 92 ||Daniel Blezek||||same|| | ||
Line 68: | Line 68: | ||
|style="color:white; background:black"|Coal||style="width:33%"| Fast Affine Registration ||complete||yes and no||yes and yes||style="background:red" |1.4% || 82 || Daniel Blezek||||same|| | |style="color:white; background:black"|Coal||style="width:33%"| Fast Affine Registration ||complete||yes and no||yes and yes||style="background:red" |1.4% || 82 || Daniel Blezek||||same|| | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| BSplineDeformableRegistration ||complete||yes and yes||yes and yes|| 82.3% || 6 || Bill Lorensen||||Fast Nonrigid BSpline Registration|| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| TestGridTransformRegistration ||no||no||yes and yes || 90.7% || 0 ||Yinglin Lee||||n/a|| | |style="background:hotpink"| Hazard||style="width:33%"| TestGridTransformRegistration ||no||no||yes and yes || 90.7% || 0 ||Yinglin Lee||||n/a|| | ||
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| CheckerBoard ||complete||yes and yes||yes and yes|| 67.4% || 0 ||Bill Lorensen||||CheckerBoard Filter|| |
|- | |- | ||
|style="background:darkorange"|Clay||style="width:33%"| ResampleVolume ||complete||yes and yes||yes and yes|| 57.1% || 0 ||Bill Lorensen||||same|| | |style="background:darkorange"|Clay||style="width:33%"| ResampleVolume ||complete||yes and yes||yes and yes|| 57.1% || 0 ||Bill Lorensen||||same|| | ||
Line 80: | Line 80: | ||
|style="background:darkorange"|Clay||style="width:33%"| GaussianBlurImageFilter ||complete||yes and no||yes and yes|| 55.9% || 0 ||Julien Jomier, Stephen Aylward||||Gaussian Blur|| | |style="background:darkorange"|Clay||style="width:33%"| GaussianBlurImageFilter ||complete||yes and no||yes and yes|| 55.9% || 0 ||Julien Jomier, Stephen Aylward||||Gaussian Blur|| | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| ConfidenceConnected||complete||yes and no||yes and yes|| 87.5% || 0 ||Jim Miller||||Simple region growing|| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| ExecutionModelTour ||no||yes and yes||yes and yes|| 87.2% || 0 ||Daniel Blezek, Bill Lorensen||||n/a|| | |style="background:hotpink"| Hazard||style="width:33%"| ExecutionModelTour ||no||yes and yes||yes and yes|| 87.2% || 0 ||Daniel Blezek, Bill Lorensen||||n/a|| | ||
Line 86: | Line 86: | ||
|style="background:hotpink"| Hazard||style="width:33%"| ImageReadDicomWrite ||no||yes and yes||yes and yes|| 86.1% || 0 ||Bill Lorensen||||Create a Dicom Series|| | |style="background:hotpink"| Hazard||style="width:33%"| ImageReadDicomWrite ||no||yes and yes||yes and yes|| 86.1% || 0 ||Bill Lorensen||||Create a Dicom Series|| | ||
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| CurvatureAnisotropicDiffusion ||complete||yes and yes||yes and yes|| 64.3% || 0 ||Bill Lorensen||||same|| |
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| GradientAnisotropicDiffusion ||complete||yes and yes||yes and yes|| 63.4% || 0 ||Bill Lorensen||||GradientAnisotropicFilter|| |
|- | |- | ||
|style="background:darkorange"|Clay||style="width:33%"| MedianImageFilter ||complete||yes and yes||yes and yes|| 51.1% || 0 ||Bill Lorensen||||same|| | |style="background:darkorange"|Clay||style="width:33%"| MedianImageFilter ||complete||yes and yes||yes and yes|| 51.1% || 0 ||Bill Lorensen||||same|| | ||
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| HistogramMatching ||complete||yes and yes||yes and yes|| 65.0% || 0 ||Bill Lorensen||||same|| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| OtsuThresholdImageFilter ||no||yes and yes||yes and yes|| 100% || 0 ||Bill Lorensen||||OtsuThreshold|| | |style="background:hotpink"| Hazard||style="width:33%"| OtsuThresholdImageFilter ||no||yes and yes||yes and yes|| 100% || 0 ||Bill Lorensen||||OtsuThreshold|| | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| OtsuThresholdSegmentation ||complete||yes and no||yes and yes|| 90.9% || 0 ||Bill Lorensen||||same|| |
|- | |- | ||
− | |style="background:silver"| | + | |style="background:silver"|2Silver||style="width:33%"| Subtract ||complete||yes and yes||yes and yes|| 71.7% || 0 ||Bill Lorensen||||SubtractImages|| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| Multiply ||no||yes and no||yes and yes|| 77.6% || 0 ||Bill Lorensen||||n/a|| | |style="background:hotpink"| Hazard||style="width:33%"| Multiply ||no||yes and no||yes and yes|| 77.6% || 0 ||Bill Lorensen||||n/a|| | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| Add ||complete||yes and yes||yes and yes|| 84.9% || 0 ||Bill Lorensen||||AddImages|| |
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| Threshold ||complete||yes and no||yes and yes|| 55.8% || 0 ||Nicole Aucoin||||ThresholdImage|| |
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| Mask ||complete||yes and no||yes and yes|| 80.0% || 0 ||Nicole Aucoin||||Mask Image|| |
|- | |- | ||
|style="color:white; background:black"|Coal||style="width:33%"| Cast ||complete||yes and no||yes and yes||style="background:red"| 14.6% || 0 ||Nicole Aucoin||||Cast Image|| | |style="color:white; background:black"|Coal||style="width:33%"| Cast ||complete||yes and no||yes and yes||style="background:red"| 14.6% || 0 ||Nicole Aucoin||||Cast Image|| | ||
|- | |- | ||
− | |style="background:gold"| | + | |style="background:gold"|1Gold||style="width:33%"| VotingBinaryHoleFillingImageFilter ||complete||yes and yes||yes and yes|| 100% || 0 ||Bill Lorensen||||Voting Binary Hole Filling|| |
|- | |- | ||
|style="background:darkorange"|Clay||style="width:33%"| ModelMaker ||complete||yes and yes||yes and yes|| 57.4% || 1 ||Nicole Aucoin||||same|| | |style="background:darkorange"|Clay||style="width:33%"| ModelMaker ||complete||yes and yes||yes and yes|| 57.4% || 1 ||Nicole Aucoin||||same|| | ||
Line 122: | Line 122: | ||
|style="background:darkorange"|Clay||style="width:33%"| GrayscaleFillHoleImageFilter ||complete||yes and yes||yes and yes|| 57.1% || 0 ||Bill Lorensen||||Grayscale Fill Hole|| | |style="background:darkorange"|Clay||style="width:33%"| GrayscaleFillHoleImageFilter ||complete||yes and yes||yes and yes|| 57.1% || 0 ||Bill Lorensen||||Grayscale Fill Hole|| | ||
|- | |- | ||
− | |style="background:darkgoldenrod"| | + | |style="background:darkgoldenrod"|3Bronze||style="width:33%"| GrayscaleGrindPeakImageFilter ||complete||yes and yes||yes and yes|| 64.5% || 0 ||Bill Lorensen||||Grayscale Grind Peak|| |
|- | |- | ||
|style="background:hotpink"| Hazard||style="width:33%"| LabelMapSmoothing ||no||yes and no||yes and yes|| 87.9% || 0 ||Dirk Padfield, Josh Cates, Ross Whitaker||||n/a|| | |style="background:hotpink"| Hazard||style="width:33%"| LabelMapSmoothing ||no||yes and no||yes and yes|| 87.9% || 0 ||Dirk Padfield, Josh Cates, Ross Whitaker||||n/a|| |
Revision as of 02:32, 29 April 2010
Home < Slicer-3.6-QAReturn to Slicer 3.6 documentation
- This page contains our assessment of the Slicer 3.6 modules
- See also the module culling event at the end of April 2010
Contents
Testing Status
Core Modules
Score | Name | Documentation | Help [1] | Acknowledgment [2] | Current Owner |
---|---|---|---|---|---|
?? | Camera | complete | yes, no link | no | Sebastien Barre |
?? | Volumes | complete | yes and yes | yes and yes | Steve Pieper |
?? | Diffusion Editor | complete | yes and yes | no | CF Westin |
?? | Models | complete | yes and yes | yes and yes | Alex Yarmarkovich |
?? | Measurements | complete | yes and yes | yes and yes | Nicole Aucoin |
?? | Fiducials | complete | yes and yes | yes and yes | Nicole Aucoin |
?? | Data | complete | yes and yes | yes and yes | Steve Pieper |
?? | Slices | complete | yes and yes | yes and yes | Jim Miller |
?? | Colors | complete | yes and yes | yes and yes | Nicole Aucoin |
?? | Interactive Editor | complete | no and yes | yes and no | Steve Pieper |
?? | ROI Module | complete | no and yes | no and yes | Alex Yarmarkovich |
?? | Volume Rendering | complete | yes and yes | no and yes | Yanling Liu |
?? | PET/CT Fusion | complete | yes and yes | yes and yes | Wendy Plesniak |
Application CLI Modules
Score | Name | Documentation | Help [1] | Acknowledgment [2] | Test coverage [3] | valgrind errors | Current Owner | Comments [4] | 3.6 Documentation Name [5] | |
---|---|---|---|---|---|---|---|---|---|---|
1Gold | my module | complete | yes and yes | yes, yes, yes | 80% | 0 | ||||
Hazard | DiffusionWeightedTest | no | no | yes and yes | 69.8% | 0 | CF Westin | n/a | ||
2Silver | DiffusionTensorEstimation | complete | yes and no | yes and yes | 79.7% | 9 | CF Westin | same | ||
Hazard | DiffusionTensorMathematics | no | yes and no | yes and yes | 56.2% | 5 | CF Westin | Diffusion Tensor Scalar Measurements | ||
1Gold | OrientImage | complete | yes and yes | yes and yes | 85.7% | 0 | Bill Lorensen | OrientImages | ||
3Bronze | LinearRegistration | complete | yes and no | yes and yes | 66.7% | 4 | Daniel Blezek | same | ||
Coal | RigidRegistration | complete | yes and yes | yes and yes | 1.0% | 92 | Daniel Blezek | same | ||
?? | RegisterImages (RegisterImagesMultiRes) | complete | yes and yes | yes and yes | Casey Goodlett, Stephen Aylward | same | ||||
Coal | Fast Affine Registration | complete | yes and no | yes and yes | 1.4% | 82 | Daniel Blezek | same | ||
1Gold | BSplineDeformableRegistration | complete | yes and yes | yes and yes | 82.3% | 6 | Bill Lorensen | Fast Nonrigid BSpline Registration | ||
Hazard | TestGridTransformRegistration | no | no | yes and yes | 90.7% | 0 | Yinglin Lee | n/a | ||
3Bronze | CheckerBoard | complete | yes and yes | yes and yes | 67.4% | 0 | Bill Lorensen | CheckerBoard Filter | ||
Clay | ResampleVolume | complete | yes and yes | yes and yes | 57.1% | 0 | Bill Lorensen | same | ||
Hazard | PolyDataToLabelmap | no | yes and yes | yes and yes | 87.9% | 0 | Nicole Aucoin, Xiaodong Tao | n/a | ||
Clay | GaussianBlurImageFilter | complete | yes and no | yes and yes | 55.9% | 0 | Julien Jomier, Stephen Aylward | Gaussian Blur | ||
1Gold | ConfidenceConnected | complete | yes and no | yes and yes | 87.5% | 0 | Jim Miller | Simple region growing | ||
Hazard | ExecutionModelTour | no | yes and yes | yes and yes | 87.2% | 0 | Daniel Blezek, Bill Lorensen | n/a | ||
Hazard | ImageReadDicomWrite | no | yes and yes | yes and yes | 86.1% | 0 | Bill Lorensen | Create a Dicom Series | ||
3Bronze | CurvatureAnisotropicDiffusion | complete | yes and yes | yes and yes | 64.3% | 0 | Bill Lorensen | same | ||
3Bronze | GradientAnisotropicDiffusion | complete | yes and yes | yes and yes | 63.4% | 0 | Bill Lorensen | GradientAnisotropicFilter | ||
Clay | MedianImageFilter | complete | yes and yes | yes and yes | 51.1% | 0 | Bill Lorensen | same | ||
3Bronze | HistogramMatching | complete | yes and yes | yes and yes | 65.0% | 0 | Bill Lorensen | same | ||
Hazard | OtsuThresholdImageFilter | no | yes and yes | yes and yes | 100% | 0 | Bill Lorensen | OtsuThreshold | ||
1Gold | OtsuThresholdSegmentation | complete | yes and no | yes and yes | 90.9% | 0 | Bill Lorensen | same | ||
2Silver | Subtract | complete | yes and yes | yes and yes | 71.7% | 0 | Bill Lorensen | SubtractImages | ||
Hazard | Multiply | no | yes and no | yes and yes | 77.6% | 0 | Bill Lorensen | n/a | ||
1Gold | Add | complete | yes and yes | yes and yes | 84.9% | 0 | Bill Lorensen | AddImages | ||
3Bronze | Threshold | complete | yes and no | yes and yes | 55.8% | 0 | Nicole Aucoin | ThresholdImage | ||
1Gold | Mask | complete | yes and no | yes and yes | 80.0% | 0 | Nicole Aucoin | Mask Image | ||
Coal | Cast | complete | yes and no | yes and yes | 14.6% | 0 | Nicole Aucoin | Cast Image | ||
1Gold | VotingBinaryHoleFillingImageFilter | complete | yes and yes | yes and yes | 100% | 0 | Bill Lorensen | Voting Binary Hole Filling | ||
Clay | ModelMaker | complete | yes and yes | yes and yes | 57.4% | 1 | Nicole Aucoin | same | ||
Hazard | MultipleModelsExample | no | yes and no | yes and yes | 1.9% | 50 | Nicole Aucoin | n/a | ||
Hazard | GrayscaleModelMaker | no | yes and no | yes and yes | 82.1% | 1 | Bill Lorensen | same | ||
Hazard | MergeModels | no | yes and yes | yes and yes | 62.7% | 0 | Nicole Aucoin, Daniel Haehn | same | ||
Clay | GrayscaleFillHoleImageFilter | complete | yes and yes | yes and yes | 57.1% | 0 | Bill Lorensen | Grayscale Fill Hole | ||
3Bronze | GrayscaleGrindPeakImageFilter | complete | yes and yes | yes and yes | 64.5% | 0 | Bill Lorensen | Grayscale Grind Peak | ||
Hazard | LabelMapSmoothing | no | yes and no | yes and yes | 87.9% | 0 | Dirk Padfield, Josh Cates, Ross Whitaker | n/a | ||
Coal | ImageLabelCombine | complete | yes and no | yes and no | 7.5% | 50 | Alex Yarmarkovich | same | ||
Hazard | ResampleVolume2 | no | yes and yes | yes and yes | 46.8% | 0 | Francois Budin | Resample Scalar/Vector/DWI Volume | ||
Hazard | ZeroCrossingBasedEdgeDetectionImageFilter | no | yes and no | yes and yes | 62.5% | 0 | Elliot Uvero | n/a | ||
Hazard | FreesurferSurfaceSectionExtraction | no | yes and no | yes and yes | 60.0% | 62 | Sylvain Bouix | n/a/ | ||
?? | BRAINSDemonWarp | complete | yes and no | yes and no | 0 | Hans Johnson, Greg Harris | same | |||
?? | BRAINSFit | complete | yes and yes | yes and yes | 0 | Eun Young Kim | same | |||
?? | BRAINSMush | complete | no | no | 0 | Hans Johnson | same | |||
Hazard | BRAINSResample | no | yes and yes | yes and no | 0 | Hans Johnson | n/a | |||
Hazard | BRAINSROIAuto | no | yes and no | yes and no | 0 | Hans Johnson | n/a | |||
Hazard | DiffusionTensorEstimation BatchMake | no | yes and no | yes and no | 0 | Lauren O'Donnell | n/a | |||
?? | EMSegment BatchMake | complete | yes and yes | yes and yes | 0 | Julien Jomier, Stephen Aylward, Brad Davis | EM Segmenter batch | |||
?? | Gaussian Blur BatchMake | complete | yes and yes | yes and yes | 0 | Julien Jomier, Stephen Aylward, Julien Finet | Gaussian Blur batch | |||
?? | RegisterImages BatchMake | complete | yes and yes | yes and yes | 0 | Stephen Aylward, Julien Finet | Register Images batch | |||
?? | ResampleScalarVolume BatchMake | complete | yes and yes | yes and yes | 0 | Julien Finet | Resample Volume batch | |||
Hazard | Diffeomorphic Demons Algorithm | no | no | yes and yes | 0 | Tom Vercauteren, Ender Konukoglu, Kilian Pohl | n/a | |||
?? | ResampleDTIVolume | complete | yes and yes | yes and yes | 0 | Francois Budin | same | |||
?? | Rician LMMSE Image Filter | complete | yes and no | yes and yes | 0 | Antonio Tristan Vega, Santiago Aja Fernandez, Marc Niethammer | same | |||
?? | Unbiased Non Local Means filter for DWI | complete | yes and no | yes and yes | 0 | Antonio Tristán Vega, Santiago Aja-Fernandez | n/a | |||
?? | Joint Rician LMMSE Image Filter | complete | yes and no | yes and yes | 0 | Antonio Tristán Vega, Santiago Aja Fernández | same | |||
Hazard | Extract Skeleton | no | yes and no | no and yes | 0 | Pierre Seroul, Martin Styner, Guido Gerig, Stephen Aylward | n/a | |||
?? | MRI Bias Field Correction | complete | yes and yes | yes and yes | 0 | Sylvain Jaume | same | |||
Hazard | N4ITK MRI Bias Correction | no | yes and no | yes and yes | 0 | Nick Tustison, Andriy Fedorov | same | |||
?? | ACPC Transform | complete | yes and yes | yes and yes | 0 | Nicole Aucoin | same | |||
?? | Mesh Contour Segmentation | complete | yes and yes | yes and yes | 0 | Peter Karasev, Karol Chudy, Allen Tannenbaum | same | |||
Hazard | Utah BSpline Deformable registration | no | yes and no | yes and yes | 0 | Sam Gerber, Jim Miller, Steve Pieper, Ross Whitaker | n/a | |||
Hazard | Calculate Volume Statistics | no | no | no | 0 | Tri Ngo | n/a | |||
Hazard | Generate Connectivity Map | no | no | no | 0 | Tri Ngo | n/a | |||
Hazard | ROI Tract Filter | no | no | no | 0 | Tri Ngo | n/a | |||
Hazard | Stochastic Tractography Filter | no | no | no | 0 | Tri Ngo | n/a | |||
Hazard | MIDAS Data and Slicer Tutorials | no | yes and no | yes and no | 0 | Patrick Reynolds, Stephen Aylward | n/a |
- ↑ including link to website documentation
- ↑ including name of author and of supervisor
- ↑ in % for non-interactive part
- ↑ Test coverage percentage determined by: [# lines code covered / total # lines of code (covered and non-covered) *100] of relevant .cxx and .h files
- ↑ http://wiki.slicer.org/slicerWiki/index.php/Documentation-3.6
Testing Partition
Most Slicer modules have a GUI component and a Data Processing component.
Testing GUI components is still a challenge, so we will focus here on testing the Data Processing components. This can be done in most cases by partitioning the module into a GUI section and a Data Processing section, where the second one usually takes the form of a C++ class (although that is not a requirement).
The data processing section can be tested by using standard CTest/CMake mechanisms. Basically by adding ADD_TEST() entries to the CMakeLists.txt file of the module.
Luis Ibanez' scoring system
The following scoring will be applied to the data processing sections of all modules:
Score | Code Coverage | Valgrind Errors | Documentation | Tutorial |
---|---|---|---|---|
1Gold | > 80% | 0 | yes | yes |
2Silver | > 70% | < 10 | yes | yes |
3Bronze | > 60% | < 50 | yes | yes |
4Clay | > 50% | < 100 | yes | yes |
5Coal | < 50% | > 100 | yes | yes |
6Hazard | unknown | unknown | no | no |
The code coverage and Valgrind error must be the ones reported on the Nightly Slicer Dashboard. Anecdotal data is not acceptable.