Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip fragments missing new attribute after evolution #3528

Merged
merged 8 commits into from Sep 22, 2022

Conversation

shaunrd0
Copy link
Contributor

TYPE: BUG
DESC: Fix segfault after schema evolution when reading using TILEDB_UNORDERED

@shortcut-integration
Copy link

This pull request has been linked to Shortcut Story #21652: Segfault after schema evolution.

+ Will revert this commit before merge
@shaunrd0
Copy link
Contributor Author

The segmentation fault is no longer occurring, but I'm now noticing fill values are not applied to the final read in the example I pushed in the latest commit.

To summarize, I evolve the schema here

// Evolve array
ArraySchemaEvolution se(ctx);
double val = -1.0;
auto attr = Attribute::create<double>(ctx, "a2")
    .set_fill_value(&val, sizeof(val));
se.add_attribute(attr);
se.array_evolve(array_name);

And later read from the array after writing 1.0 to the new attribute at coordinate (1, 1) in the 2D array. The output produced is

a1: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
a2: 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

But I would expect the output to be

a1: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
a2: 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1

Any advice would be appreciated

@KiterLuc KiterLuc marked this pull request as ready for review September 22, 2022 11:45
@ihnorton ihnorton merged commit a7f385e into dev Sep 22, 2022
@ihnorton ihnorton deleted the smr/sc-21652/debug-evolution branch September 22, 2022 15:41
@github-actions
Copy link
Contributor

The backport to release-2.11 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-release-2.11 release-2.11
# Navigate to the new working tree
cd .worktrees/backport-release-2.11
# Create a new branch
git switch --create backport-3528-to-release-2.11
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick --mainline 1 a7f385e6d27cb236afd9b63b3d3290e2f66cf2dd
# Push it to GitHub
git push --set-upstream origin backport-3528-to-release-2.11
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-release-2.11

Then, create a pull request where the base branch is release-2.11 and the compare/head branch is backport-3528-to-release-2.11.

ihnorton pushed a commit that referenced this pull request Sep 22, 2022
* Skip fragments missing new attribute after evolution
* Copy fill values to result buffers
* Fixing sparse readers for schema evolution.
* Unit tests for evolved sparse read layouts

(cherry picked from commit a7f385e)

Co-authored-by: Luc Rancourt <lucrancourt@gmail.com>
Co-authored-by: Shaun Reed <shaun.reed@tiledb.com>
ihnorton added a commit that referenced this pull request Sep 23, 2022
* Skip fragments missing new attribute after evolution (#3528)

* Skip fragments missing new attribute after evolution
* Copy fill values to result buffers
* Fixing sparse readers for schema evolution.
* Unit tests for evolved sparse read layouts

(cherry picked from commit a7f385e)

Co-authored-by: Luc Rancourt <lucrancourt@gmail.com>
Co-authored-by: Shaun Reed <shaun.reed@tiledb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants