Complex data sets can cause stackoverflow in arcGrowth subroutine in FTM

[ghost]

Describe the bug
FTM in TTK crashes on a more complicated data set with a stack overflow in the arcGrowth.

To Reproduce
Steps to reproduce the behavior:

1. Get data here http://sci.utah.edu/~klacansky/data/ftm/magnetic_reconnection_512x512x512_float32.vti
2. Run ttkFTMTreeCmd -i magnetic_reconnection_512x512x512_float32.vti -T 1
3. The code will crash (stack overflow, can be seen with gdb or asan)

Expected behavior
A correct computation of a superlevel set merge tree.

System:

• OS: CentOS Linux release 7.5.1804 (Core)
• Compiler: GCC 7.3.1
• Boost 1.69, VTK 8.2.0

Additional context
Increasing the stack size for OMP worker threads helps (e.g., OMP_STACKSIZE = 128M) and using a separate stack on a heap would probably be more robust.

[julien-tierny]

@CharlesGueunet: can you look into this once you're done with EGPGV? cheers -- Dr Julien Tierny CNRS Researcher Sorbonne Universite http://lip6.fr/Julien.Tierny

…

On Saturday, March 2, 2019 11:34:29 PM CET klacansky wrote: **Describe the bug** FTM in TTK crashes on a more complicated data set with a stack overflow in the arcGrowth. **To Reproduce** Steps to reproduce the behavior: 1. Get data here http://sci.utah.edu/~klacansky/data/ftm/magnetic_reconnection_512x512x512_float32.vti 2. Run `ttkFTMTreeCmd -i magnetic_reconnection_512x512x512_float32.vti -T 1` 3. The code will crash (stack overflow, can be seen with gdb or asan) **Expected behavior** A correct computation of a superlevel set merge tree. **System:** - OS: CentOS Linux release 7.5.1804 (Core) - Compiler: GCC 7.3.1 - Boost 1.69, VTK 8.2.0 **Additional context** Increasing the stack size for OMP worker threads helps (e.g., OMP_STACKSIZE = 128M) and using a separate stack on a heap would probably be more robust.

[ghost]

I found an overflow. https://github.com/klacansky/ttk-testing/commit/dab76f0bb3647ee088ee915d4cc4bad07029f8cb

[ghost]

The sort crashes, probably some other overflow as the value of b is -9.

Thread 1 "ttkFTMTreeCmd" received signal SIGSEGV, Segmentation fault.
0x00007ffff1a18fd1 in ttk::ftm::FTMTree_MT::isLower<unsigned short, long long> (this=0x66b5a0, a=1, b=-9)
    at /home/sci/klacansky/scratch/ttk/core/base/ftmTree/FTMTree_MT.h:686
686              return ((scalarType *)scalars_->values)[a] < ((scalarType *)scalars_->values)[b] ||
(gdb) bt
#0  0x00007ffff1a18fd1 in ttk::ftm::FTMTree_MT::isLower<unsigned short, long long> (this=0x66b5a0, a=1, b=-9)
    at /home/sci/klacansky/scratch/ttk/core/base/ftmTree/FTMTree_MT.h:686
#1  0x00007ffff1a1415f in void ttk::ftm::FTMTree_MT::sortInput<unsigned short, long long>()::{lambda(unsigned long const&, unsigned long const&)#1}::operator()(unsigned long const&, unsigned long const&) const (__closure=0x7fffffffd380,
    a=@0x7fffffffd358: 1, b=@0x7fffffffd350: 18446744073709551607)

[ghost]

I fixed overflows (ones I found) and tested on a 2k^3 zeros field. The OpenMP stack overflow still needs work.

[julien-tierny]

note that you'll need to build ttk with TTK_ENABLE_64BIT_IDS=ON (advanced cmake option) to run properly on a 2,048^3 grid (identifiers are likely to overflow the range of the default integers). thanks for letting us know if that helps. cheers -- Dr Julien Tierny CNRS Researcher Sorbonne Universite http://lip6.fr/Julien.Tierny

…

On Monday, 8 April 2019 21:56:05 CEST klacansky wrote: I fixed overflows (ones I found) and tested on a 2k^3 zeros field. The OpenMP stack overflow still needs work.

[CharlesGueunet]

For the OpenMP stack overflow, I was able to solve the issue locally using the $OMP_STACKSIZE environnement variable (documented here)

[ghost]

I was able to run a zero field success fully (64 bits enabled). @CharlesGueunet I would document the need to increase the stack size (or somehow do a check and return error if stack fails to grow, much easier with explicit stack).