Difference between revisions of "Documentation/Nightly/Developers/Tutorials/Troubleshooting"

From Slicer Wiki
Jump to: navigation, search
Line 85: Line 85:
 
  > continue
 
  > continue
 
  > end
 
  > end
  (gdb) commands 1
+
  (gdb) commands 2
 
  > continue
 
  > continue
 
  > end
 
  > end

Revision as of 16:06, 14 August 2014

Home < Documentation < Nightly < Developers < Tutorials < Troubleshooting


For the latest Slicer documentation, visit the read-the-docs.


Here are some tips to narrow down various issues such as crash, or memory leaks

Disable features

Try running Slicer with as few features as possible:

  • Disable Slicer options via the command line
    $ ./Slicer --no-splash --ignore-slicerrc --disable-cli-module --disable-loadable-module --disable-scriptedmodule
    • Look at all the possible options
      $ ./Slicer --help
  • Disable ITK plugins
    CLI modules silently load the ITK plugins in lib/Slicer-4.10/ITKFactories. These plugins are used to share the volumes between Slicer and the ITK filter without having to copy them on disk.
    • rename lib/Slicer-4.10/ITKFactories into lib/Slicer-4.10/ITKFactories-disabled
  • Disable Qt plugins
    • rename lib/Slicer-4.10/iconengine into lib/Slicer-4.10/iconengine-disabled

Track memory leaks

  1. Turn ON the VTK_DEBUG_LEAKS CMake variable and build Slicer
  2. Create a test that reproduces the memory leak systematically
  3. After execution, the memory leaks are printed automatically by VTK on the standard output: 224: vtkDebugLeaks has detected LEAKS! 224: Class "vtkProperty2D" has 60 instances still around. ... Alternatively, you can simply run Slicer instead of a custom test.
  4. Find what specific instance of a class (here vtkProperty2D) leaks.
  5. If the class is instantiated a large amount of time, it is hard to know what instance is leaking. By making verbose the constructor and destructor of the faulty class, you can infer which instance is leaking. Below are 2 techniques to print whenever the con/destructors are called:
    1. By recompiling VTK
    2. You can edit the constructor and destructor of the class (here vtkProperty2D::vtkProperty2D() and vtkProperty2D::~vtkProperty2D()) vtkProperty2D::vtkProperty2D() { ... static int count = 0; std::cout << "CREATE vtkProperty2D instance #" << count++ << " " << this << std::endl; } vtkProperty2D::~vtkProperty2D() { ... static int count = 0; std::cout << "DELETE vtkProperty2D instance #" << count++ << " " << this << std::endl; } Don't forget to rebuild VTK if the class is from VTK (no need to build Slicer inner build) After running the test, you should see outputs similar to ... CREATE vtkProperty2D instance #0 0x0123456 ... CREATE vtkProperty2D instance #1 0x01234E5 ... DELETE vtkProperty2D instance #0 0x0123456 ... DELETE vtkProperty2D instance #1 0x01234E5 ... CREATE vtkProperty2D instance #2 0x0123A23 ... CREATE vtkProperty2D instance #3 0x0124312 ... Copy/paste the listing in a text editor and ensure that for each CREATE of a pointer (e.g. 0x0123456) it exists a DELETE. If there isn't, you found what instance is leaking. Note the instance # (here instance #2 and #3)
    3. Without recompiling any library but using the debugger
    4. Any debugger with advanced breakpoint controls should work.
      • With Visual Studio
      1. Set breakpoints in the class constructor and destructor methods.
      2. Right click on the breakpoint, select "When Hit..." and "Print a message" with "Function: $FUNCTION {this}". Make sure the "Continue execution" checkbox is selected.
      3. Execute the test or Slicer
      4. Open the "Output" tab and copy paste the contents into an advanced file editor (not Visual Studio)
      • With GDB
      1. Start gdb
      2. Using the launcher $ ./Slicer --gdb Or sometimes the following works as well $ gdb ./bin/SlicerApp-real
      3. Place breakpoints in the functions
      4. (gdb) break vtkProperty2D::vtkProperty2D() (gdb) break vtkProperty2D::~vtkProperty2D() gdb will stop in those methods each time the program steps into. It will then print a line such as: Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x123456789) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22 22 vtkProperty2D::vtkProperty2D()
      5. Automatically continue execution after each break
      6. (gdb) commands 1 > continue > end (gdb) commands 2 > continue > end
      7. Start the execution and copy paste the logs printed by gdb into an advanced file editor (e.g. emacs)
      8. (gdb) run

      After running the test(by recompiling or with debugger), you should see outputs similar to

      ...
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      22 vtkProperty2D::vtkProperty2D()
      ...
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      22 vtkProperty2D::vtkProperty2D()
      ...
      Breakpoint 2, vtkProperty2D::~vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:37
      37 vtkProperty2D::vtkProperty2D()
      ...
      Breakpoint 2, vtkProperty2D::~vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:37
      37 vtkProperty2D::vtkProperty2D()
      ...
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      22 vtkProperty2D::vtkProperty2D()
      ...
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0124312) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      22 vtkProperty2D::vtkProperty2D()
      ...
      Breakpoint 2, vtkProperty2D::~vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:37
      37 vtkProperty2D::vtkProperty2D()
      ...
      

      In an text editor, cleanup the logs by keeping only the "Breakpoint*" lines:

      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 2, vtkProperty2D::~vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:37
      Breakpoint 2, vtkProperty2D::~vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:37
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0124312) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 2, vtkProperty2D::~vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:37
      

      Save a copy of this file, and make the destructor lines similar to the constructor ones (using Replace tools):

      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0124312) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      

      Sort the file to make the constructor and destructor lines next to each other(emacs: M-x sort-lines, Notepad++: TextFX/TextFX Tools/Sort lines case sensitive):

      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0124312) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      

      Find the line that is not duplicated:

      • Using Notepad++ (version >=6):
        • Open the search dialog
        • Find what: "^(.*?)$\s+?^(?=.*^\1$)", Replace with: "dup ", toggle "Regular expression"
        • Replace All
        • The line without the "dup" prefix is the line we are looking for
      • Using emacs:
        • C-M-%
        • Replace regexp: \(.*\)<type C-q C-j>\1
        • with: dup<type C-q C-j>

      Extract the instance address from the line: e.g. 0x0124312

      From the original logs, keep only the Constructor lines:

      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123456) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x01234E5) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0123A23) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      Breakpoint 1, vtkProperty2D::vtkProperty2() (this=0x0124312) at /home/user/Slicer-superbuild/VTK/.../vtkProperty2D.cxx:22
      
      

      Search into the original saved file what line the address is: e.g. 4

  6. Find the leaking instance at run-time
  7. Run the test with the debugger, set a breakpoint in the class constructor and ignore the break as many time as the instance number (say it's 2):
    • GDB
    (gdb) break vtkProperty2D::vtkProperty2D() Breakpoint 1 at 0x5b22d0e55d04296: file vtkProperty2D.cxx, line 22 (gdb) ignore 1 2 (gdb) run When the debugger stops, check the call stack: (gdb) backtrace
    • Visual Studio
      • Set a breakpoint in vtkProperty2D::vtkProperty2D()
      • Right click on the breakpoint and select HitCount, select "break when the hit count is equal to" and type '2'
      • Start the application
    By analyzing the trace, you should be able to find the faulty instance.
  8. Analyze the code to see where the leak could be
  9. Here is a listing of the most common mistakes
    • this->MyXXX = vtkXXX::New();
      • Is there a matching this->MyXXX->Delete() ?
      • Are you sure this->MyXXX is 0 before being assigned a new pointer ? If not, then you need to add
    if (this->MyXXX != 0) { this->MyXXX->Delete(); this->MyXXX = 0; } this->MyXXX = vtkXXX::New();

Other resources

Track a crash while accessing already deleted object pointer

If the application crashes by accessing an invalid pointer. The goal here is to find when (and why) the pointer is deleted before it is accessed .

  1. Before the crash, print the value of the pointer:
  2. Add before the crash (i.e. this->MyObject->update()) std::cout << ">>>>>Object pointer: " << this->MyObject << std::endl;
  3. Add a counter in the destructor:
  4. Add in the object destructor (it can be in the base class (vtkObject or vtkMRMLNode) if you don't know the type): static int count = 0; std::cout << "#######Object Destructed: " << this << " " << count++ << std::endl;
  5. Run the application and make it crash.
  6. In the logs you shall see something like that:
  7. #######Object Destructed: 0x12345678 0 #######Object Destructed: 0x12345679 1 #######Object Destructed: 0x12345680 2 >>>>>Object Pointer: 0x12345660 >>>>>Object Pointer: 0x12345661 >>>>>Object Pointer: 0x12345662 #######Object Destructed: 0x12345660 3 #######Object Destructed: 0x12345661 4 #######Object Destructed: 0x12345662 5 #######Object Destructed: 0x12345663 6 >>>>>Object Pointer: 0x12345670 >>>>>Object Pointer: 0x12345671 >>>>>Object Pointer: 0x12345672 #######Object Destructed: 0x12345660 7 #######Object Destructed: 0x12345661 8 #######Object Destructed: 0x12345662 9 >>>>>Object Pointer: 0x12345663 Segfault
  8. Search in the logs when the pointer before crash has been deleted. Set a conditional breakpoint in the debugger:
  9. (gdb) break MyObj.cxx:103 if count == 6 or (gdb) break MyObj.cxx:103 (gdb) ignore 1 5 or if you don't want to use a debugger, you can make it crash the 6th time: assert(count != 6);
  10. Analyze the backtrace to understand why the pointer has been deleted without letting know the object that tries to access it.

Console output on Windows

On Windows, the application is built with no console output. A workaround for this issue is described in the following bug reports:

To add console output, you need to compile Slicer application with Slicer_BUILD_WIN32_CONSOLE set to ON at the configure time (uninitialized/OFF by default).