Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

carta_backend built with ASAN flags occasionally crashes running Moments ICD tests #1070

Closed
ajm-ska opened this issue Mar 22, 2022 · 3 comments · Fixed by #1137
Closed

carta_backend built with ASAN flags occasionally crashes running Moments ICD tests #1070

ajm-ska opened this issue Mar 22, 2022 · 3 comments · Fixed by #1137
Assignees
Labels
bug Something isn't working
Milestone

Comments

@ajm-ska
Copy link
Collaborator

ajm-ska commented Mar 22, 2022

Describe the bug
The carta-backend sometimes crashes when running the MOMENTS_GENERATOR_FITS.test.ts or MOMENTS_GENERATOR_HDF5.test.ts ICD tests. It can happen on Ubuntu 18.04, macOS12, and RHEL8.
It is rare on Ubuntu 18.04, but occurs more frequently on macOS12 with the HDF5 test.

To Reproduce
Build carta-backend with the Address Sanitizer flags.
Start the carta-backend.
Repeatedly run the MOMENTS_GENERATOR_HDF5.test.ts ICD test.
The carta-backend may crash on the 2nd or 3rd try.

Expected behavior
carta-backend should not crash.

Screenshots or videos
Here is the full crash log when running the MOMENTS_GENERATOR_FITS.test.ts test in the Ubuntu 18.04 Docker container moment-generator-crash-ubuntu-1804.log
Here is the full crash log when running the MOMENTS_GENERATOR_HDF5.test.ts on the M1 macOS12 moment-generator-crash-macos12.log

Platform info (please complete the following information):

  • OS: Ubuntu 18.04 Docker container (ajmasiaa/bionic-jenkins-mar2022) and macOS12

Additional context
I tried to trigger the issue through the frontend by manually following the steps in the ICD test, but can not create the crash event manually yet.

This issue is probably related to other reported issues.

@acdo2002
Copy link
Contributor

acdo2002 commented Mar 22, 2022

I confirmed this issue occurs in Ubuntu 20.04 Laptop and macOS12 Desktop.
It only occurs with carta_backend with Address Sanitizer flag.
The carta_backend Ubuntu 20.04 log can be download from MOMENT_HDF5_AS-Ubuntu.log

The carta_backend macOS12 log can be download from
MOMENT_HDF5_AS-Mac.log

If I use the carta_backend without Address Sanitizer flag, this issue will not occur.

@veggiesaurus
Copy link
Collaborator

The ubuntu error seems very odd to me. It looks like it's complaining about a double-free on std::vector<CARTA::StatsType> _spectral_stats, which is just handled by the default destructor of RegionHandler.

@veggiesaurus veggiesaurus added the bug Something isn't working label Mar 30, 2022
@veggiesaurus veggiesaurus added this to the v3.0b-3 milestone Mar 30, 2022
@veggiesaurus veggiesaurus removed this from the v3.0b-3 milestone May 5, 2022
@veggiesaurus veggiesaurus added this to the v3.0-stable milestone Jun 8, 2022
@Kechil Kechil assigned markccchiang and jolopezl and unassigned jolopezl Jun 22, 2022
@Kechil
Copy link

Kechil commented Jun 22, 2022

Hi @markccchiang, @jolopezl
Sorry for confusion about re-assigning. This is Jorge's task but we thought it would be good for Mark to help out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants