Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Thread Panic from grenad - must be greater than #3043

Closed
1 of 3 tasks
ryanrussell opened this issue Nov 14, 2022 · 2 comments · Fixed by #3084
Closed
1 of 3 tasks

Thread Panic from grenad - must be greater than #3043

ryanrussell opened this issue Nov 14, 2022 · 2 comments · Fixed by #3084
Assignees
@loiclec
Labels
bug Something isn't working as expected milli Related to the milli workspace v0.30.0 PRs/issues solved in v0.30.0 released on 2022-11-28
Milestone
v0.30.0

Comments

@ryanrussell
Copy link
Contributor

ryanrussell commented Nov 14, 2022 •
edited by curquiza
Loading

Describe the bug
During ingest, and trying to re-create the panic from #3042, I also saw this:

thread '<unnamed>' panicked at '[3, 115, 49, 55, 48, 49, 0, 50] must be greater than [3, 115, 49, 55, 48, 49, 0, 51]', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/grenad-0.4.3/src/block_writer.rs:109:17

thread '<unnamed>' panicked at '[3, 115, 49, 55, 48, 49, 0, 50] must be greater than [3, 115, 49, 55, 48, 49, 0, 53]', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/grenad-0.4.3/src/block_writer.rs:109:17

To Reproduce
Have not found minimal, reproducible example.

Believe it is related to changing the ETL to run in multiple workers who may re-write the same indices/ID's.

I have moved ETL scripts into Airflow and parallelized them, so I am GUESSING there are instances where inserts from different workers are over-writing the same ID in close succession.

Expected behavior
Handle parallel ETL inserts dealing with the same ID without crash

Screenshots

Meilisearch version: v0.30.0rc1

Additional context
This may be related to #3042 -- It was seen during the same ETL trials.

TODO

  • Implement changes in Milli
  • Release a Milli version containing these changes
  • Bump this new Milli version in Meilisearch and merge it into release-v0.30.0
@curquiza curquiza added this to the v0.30.0 milestone Nov 14, 2022
@curquiza curquiza added the bug Something isn't working as expected label Nov 14, 2022
loiclec added a commit to meilisearch/milli that referenced this issue Nov 14, 2022
@loiclec
Fix bug in prefix DB indexing
c71c9e5 
Where the batch's information was not properly updated in cases
where only the proximity changed between two consecutive word pair
proximities.

Closes meilisearch/meilisearch#3043
@loiclec loiclec mentioned this issue Nov 14, 2022
Merged
@curquiza curquiza added the milli Related to the milli workspace label Nov 14, 2022
loiclec added a commit to meilisearch/milli that referenced this issue Nov 16, 2022
@loiclec
Fix bug in prefix DB indexing
444c790 
Where the batch's information was not properly updated in cases
where only the proximity changed between two consecutive word pair
proximities.

Closes meilisearch/meilisearch#3043
bors bot added a commit to meilisearch/milli that referenced this issue Nov 17, 2022
@bors @loiclec
Merge #697
57c9f03 
697: Fix bug in prefix DB indexing r=loiclec a=loiclec

Where the batch's information was not properly updated in cases where only the proximity changed between two consecutive word pair proximities.

Closes partially meilisearch/meilisearch#3043



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
@curquiza
Copy link
Member

Re opening, not fixed until we merged the new milli's version into release-v0.30.0

@curquiza curquiza reopened this Nov 17, 2022
@curquiza curquiza linked a pull request Nov 17, 2022 that will close this issue
Bump the milli and grenad dependencies #3084
Merged
@Kerollmops Kerollmops removed a link to a pull request Nov 17, 2022
Bump the milli and grenad dependencies #3084
Merged
@Kerollmops Kerollmops linked a pull request Nov 17, 2022 that will close this issue
Bump the milli and grenad dependencies #3084
Merged
@curquiza
Copy link
Member

Closed by #3084

@meili-bot meili-bot added the v0.30.0 PRs/issues solved in v0.30.0 released on 2022-11-28 label Nov 28, 2022
amab8901 pushed a commit to amab8901/milli that referenced this issue Dec 2, 2022
@loiclec @amab8901
Fix bug in prefix DB indexing
ff67cdf 
Where the batch's information was not properly updated in cases
where only the proximity changed between two consecutive word pair
proximities.

Closes meilisearch/meilisearch#3043
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@loiclec loiclec

Labels
bug Something isn't working as expected milli Related to the milli workspace v0.30.0 PRs/issues solved in v0.30.0 released on 2022-11-28
Projects
None yet
Milestone
v0.30.0
Development

Successfully merging a pull request may close this issue.

Bump the milli and grenad dependencies meilisearch/meilisearch
4 participants
@ryanrussell @loiclec @curquiza @meili-bot