Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply upstream fixes #1

Merged
merged 11 commits into from
Dec 7, 2022
Merged

Apply upstream fixes #1

merged 11 commits into from
Dec 7, 2022

Conversation

Kerollmops
Copy link
Member

@Kerollmops Kerollmops commented Dec 6, 2022

This PR applies the commits from upstream and moves our patches (the last two) on top of it. We wanted ITS #9806 to be fixed, and it is now!

This PR is related to meilisearch/meilisearch#3062.

hyc and others added 11 commits June 7, 2021 15:56
That should hopefully shut coverity up
It also contains the cacheflush function declaration.
32 was chosen for page size of 4KB. Not large nough for 16KB pages.
fakepage pointers are only guaranteed to be 2-byte aligned. Use a
2-byte aligned struct definition when referencing 2-byte page members
if a page pointer possibly points to a fakepage.
It was an issue to keep the pages in a chained cache list,
it was growing a lot and consumming a lot of memory.

The only way to free this chained list was to close the environment,
this solution was a little bit too much, so we now entirely skip the cache
and let malloc to that for us, by itself.
Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok for me

Copy link

@dureuill dureuill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK from here, let's try this!

@Kerollmops Kerollmops merged commit 6a01c0a into main Dec 7, 2022
@Kerollmops Kerollmops deleted the apply-upstream-fixes branch December 7, 2022 09:33
bors bot added a commit to meilisearch/lmdb-rs that referenced this pull request Dec 7, 2022
12: Bump the LMDB submodule r=Kerollmops a=Kerollmops

This PR bumps the LMDB submodule to bring the fixes of [ITS #9806](https://bugs.openldap.org/show_bug.cgi?id=9806) plus our patches. Related to meilisearch/lmdb#1.

Co-authored-by: Kerollmops <clement@meilisearch.com>
bors bot added a commit to meilisearch/meilisearch that referenced this pull request Dec 7, 2022
3210: Fix `MDB_PAGE_FULL` by bumping LMDB r=Kerollmops a=Kerollmops

This PR fixes #3062 by upgrading LMDB to the latest version.

The changes were made in meilisearch/lmdb#1 and meilisearch/lmdb-rs#12. As heed directly depends on the latest main commit of https://github.com/meilisearch/lmdb-rs, we can bump the `lmdb-rkv-sys` dependency in the Meilisearch _Cargo.lock_ by doing a:

```
cargo update -p lmdb-rkv-sys
```

Co-authored-by: Kerollmops <clement@meilisearch.com>
@leiless
Copy link

leiless commented Oct 10, 2023

Hi, I'm not a collaborator of meilisearch/lmdb, so I'm not allowed to crate pull requests for this repo.

But I think we should apply upstream LMDB fixes.

Pull Request

Related issue

Fixes ITS#10095, ITS#10095, ITS#9961

What does this PR do?

Keep up-to-date with upstream LMDB source code.

I recently encountered the bug: ITS#8447 fix cursor_put(MDB_CURRENT) on DUPSORT DB with different-sized data.

The new data item content(value) is interleaved with its previous data, which causes the new value corrupted.
(New value len is shorter than old value len)

See full thread:
https://www.openldap.org/lists/openldap-bugs/201606/msg00049.html
https://www.openldap.org/lists/openldap-bugs/201606/msg00050.html
https://www.openldap.org/lists/openldap-bugs/201606/msg00053.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants