Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closes #2268 - Fixes Sporadic failures of Parquet segarray_write test with gasnet #2534

Conversation

Ethan-DeBandi99
Copy link
Contributor

Closes #2268

This PR updates the processing for SegArray reads and writes to prevent memory corruption errors from occurring. From my testing, these were the errors I was seeing. They were different than those reported. Either way these errors needed to be resolved as they occur mainly when a locale only contains empty segments.

  • Adds handling for writing to locales with segment values where all segments are empty
  • Adds handling for reading from files with only empty segments
  • Adds better memory management and cleanup of memory used during processing
  • Updates testing file names
  • Adds column with strings longer than 1 character for testing

If these updates do not also address the specific reported issue, we will reopen the issue and continue troubleshooting.

@Ethan-DeBandi99 Ethan-DeBandi99 changed the title Closes #2268 - Fixes segarray parquet gasnet bug Closes #2268 - Fixes Sporadic failures of Parquet segarray_write test with gasnet Jun 29, 2023
Copy link
Member

@stress-tess stress-tess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the logic looks good to me! It looks like github is complaining about a merge conflict. Once that gets resolved and the leftover prints get removed, this should be good to go

tests/parquet_test.py Outdated Show resolved Hide resolved
tests/parquet_test.py Outdated Show resolved Hide resolved
src/ParquetMsg.chpl Show resolved Hide resolved
@Ethan-DeBandi99 Ethan-DeBandi99 force-pushed the 2268_segarray_parquet_gasnet_bug branch from ffa74fa to b722e18 Compare June 30, 2023 17:31
Copy link
Contributor

@jaketrookman jaketrookman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@stress-tess stress-tess added this pull request to the merge queue Jul 6, 2023
Merged via the queue into Bears-R-Us:master with commit 4c1570f Jul 6, 2023
9 checks passed
@Ethan-DeBandi99 Ethan-DeBandi99 deleted the 2268_segarray_parquet_gasnet_bug branch July 7, 2023 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sporadic failures of Parquet segarray_write test with gasnet
3 participants