Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

appending to variables sized atom bats other than str bats with force flag may result in corrupted heap #6465

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2017-11-13 15:27:36 +0100
From: @sjoerdmullender
To: GDK devs <>
Version: 11.27.9 (Jul2017-SP2)

Last updated: 2017-12-14 14:46:00 +0100

Comment 25873

Date: 2017-11-13 15:27:36 +0100
From: @sjoerdmullender

The SQL front end uses the force flag in calls to BATappend and other BAT changing functions to overrule the read-only setting of the BAT because SQL maintains its own recovery mechanism. The understanding SQL has is that by appending, the already existing part of the BAT is not changed, and so recovery in the case of a crash (or just exit) before the appended data was properly committed consists of just replaying the appends (which are stored in the write-ahead log). This works fine for the most part. The string heap of string BATs don't just append, but this is already handled by the recovery code.
The problem here is that other atom heaps of variable sized atoms (e.g. BLOB) also modify more in the heap when data gets appended, and these modifications are not handled when the heap is recovered at startup.

Comment 25876

Date: 2017-11-14 14:44:42 +0100
From: MonetDB Mercurial Repository <>

Changeset 7253f70231be made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=7253f70231be

Changeset description:

Recover varheaps on first loading after startup.
This fixes bug #6465.

Comment 25902

Date: 2017-11-23 14:56:33 +0100
From: @sjoerdmullender

I believe this is now fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant