Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

race condition in SQL UDF with update #6641

monetdb-team opened this issue Nov 30, 2020 · 0 comments

race condition in SQL UDF with update #6641

monetdb-team opened this issue Nov 30, 2020 · 0 comments


Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2018-09-08 08:52:24 +0200
From: @sjoerdmullender
To: SQL devs <>
Version: 11.31.7 (Aug2018)

Last updated: 2018-10-17 10:07:10 +0200

Comment 26610

Date: 2018-09-08 08:52:24 +0200
From: @sjoerdmullender

Created attachment 607
Plam from which the excerpt was taken (produced from within gdb)

There is a pretty serious race condition that sometimes gets triggered by the storagemodel test.

The problem is in the plan for the storagemdelinit function in 75_storagemdel.sql. Here is a relevant excerpt from the plan:

 X_628=<tmp_1247>[278]:bat[:lng] := sql.bind(X_586=0:int, "sys":str, "storagemodelinput":str, "distinct":str, 1:int);
 X_631=<tmp_1247>[278]:bat[:lng] :=<sql_empty_lng_bat>[0]:bat[:lng], X_629=<sql_empty_oid_bat>[0]:bat[:oid], X_630=<sql_empty_lng_bat>[0]:bat[:lng], X_628=<tmp_1247>[278]:bat[:lng]);
 X_683=<tmp_556>[98]:bat[:lng] := algebra.projectionpath(C_676=<tmp_1454>[98]:bat[:oid], C_590=<tmp_1456>[278]:bat[:oid], X_631=<tmp_1247> 278]:bat[:lng]);


 algebra.projection(C_590=<tmp_1456>[278]:bat[:oid], X_631=<tmp_1247>[278]:bat[:lng]);


 sql.update(X_586=0:int, "sys":str, "storagemodelinput":str, "distinct":str, C_690=<tmp_557>[98]:bat[:oid], X_682=<tmp_1445>[98]:bat[:lng]);



The problem is as follows. The first blurb creates a new BAT, here tmp_1247, which holds the values for the "distinct" column of the sys.storagemodelinput table after it was filled in by the call to in the storagemodelinit() function. All values are equal at this point (i.e. sorted and revsorted).

Some time later, this BAT is used in a call to algebra.projection(). (Actually, when running it, I normally see a call to algebra.projectionpath in which it is the last (third) argument. This is relevant because of the sorted flag.)

And here is the race: just a few lines further, there is a call to sql.update of that same column. Since the only other instructions between the projection(path) and the update are other projections and an unrelated aggr.count() call, the projection may happen simultaneous with the update. When this happens (and it does happen sometimes!) the projection produces a BAT that it thinks is sorted (because the inputs were sorted when the projection started) but the second input is changed while running and so the output isn't in fact sorted.

I guess the language.pass call should be moved to in front of the call to sql.update, but where does that happen? (I haven't yet looked.)

Attached file: m (text/plain, 70602 bytes)
Description: Plam from which the excerpt was taken (produced from within gdb)

Comment 26611

Date: 2018-09-08 09:28:40 +0200
From: MonetDB Mercurial Repository <>

Changeset 2b9a71c4600e made by Sjoerd Mullender in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=2b9a71c4600e

Changeset description:

Force a dataflow breakpoint before sql update instructions.
This fixes bug #6641.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant