Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault in mserver5 on Python2 UDF with TIMESTAMP column input that has NULL values (conversion.c:438, PyNullMask_FromBAT) #6508

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2018-01-08 00:11:21 +0100
From: Niklas B <<niklas.bivald>>
To: SQL devs <>
Version: 11.27.11 (Jul2017-SP3)
CC: @njnes, niklas.bivald

Last updated: 2018-02-12 16:12:21 +0100

Comment 26058

Date: 2018-01-08 00:11:21 +0100
From: Niklas B <<niklas.bivald>>

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Build Identifier:

I haven't found this bug in the archive, so I'm reporting it. Hopefully it's not a dupe :)

Using a Python UDF which takes a TIMESTAMP column as input crashes mserver5 on Debian Stretch if the table column has NULL values in it.

Running (linux~debian stretch) on commit a7f8b1f865b7 (https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a7f8b1f865b7)

Thread 6 "mserver5" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe97c1700 (LWP 22566)]
0x00007ffff1c53d5c in PyNullMask_FromBAT (b=0x7fffd817d200, t_start=0, t_end=2) at conversion.c:438
438 CreateNullMask(lng);
(gdb) l
433 break;
434 case TYPE_int:
435 CreateNullMask(int);
436 break;
437 case TYPE_lng:
438 CreateNullMask(lng);
439 break;
440 case TYPE_flt:
441 CreateNullMask(flt);
442 break;

Only tried on Debian Stretch build from source at commit a7f8b1f865b7, with and without debugging enabled. GDB produced the output above.

Reproducible: Always

Steps to Reproduce:

drop all function test_timestamp_with_nulls;
CREATE FUNCTION test_timestamp_with_nulls(actual_takeoff_datetime timestamp) RETURNS STRING LANGUAGE PYTHON {
return "it works";
};

drop table if exists example;

CREATE TABLE example (
"column1" timestamp
);

insert into example ("column1") values ('2017-01-01 00:00:01');
insert into example ("column1") values (NULL);

select test_timestamp_with_nulls("column1") from example;

Actual Results:

Segmentation fault

Expected Results:

Query should return data

If I cast the column to a STRING it works:

select test_timestamp_with_nulls(cast("column1" as string)) from example;

Comment 26059

Date: 2018-01-08 00:17:28 +0100
From: Niklas B <<niklas.bivald>>

I realise I perhaps wasn't as clear in my communication as I hoped, I apologise, it's after midnight. To summarise:

If I have a Python UDF that takes timestamp input, like so:

CREATE FUNCTION test_timestamp_with_nulls(actual_takeoff_datetime timestamp) RETURNS STRING LANGUAGE PYTHON {
return "it works";
};

And I pass a column to it, like so:

select test_timestamp_with_nulls("column1") from example;

Then if "column1" in table "example" is a timestamp and any value in the column is NULL, mserver5 crashes. If there are no NULl values, it works.

I've only tested this on Linux Debian Stretch, but I can reproduce it every time. I haven't made any changes to the source.

Comment 26061

Date: 2018-01-08 10:16:12 +0100
From: MonetDB Mercurial Repository <>

Changeset 7d2a93f65310 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=7d2a93f65310

Changeset description:

Use correct bat.
This fixes bug #6508.

Comment 26062

Date: 2018-01-08 10:16:17 +0100
From: MonetDB Mercurial Repository <>

Changeset cc06e0c02cb1 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=cc06e0c02cb1

Changeset description:

Added test for bug #6508.

Comment 26063

Date: 2018-01-08 11:40:58 +0100
From: Niklas B <<niklas.bivald>>

Works like a charm, thank you so much :)

Comment 26064

Date: 2018-01-08 22:19:06 +0100
From: Niklas B <<niklas.bivald>>

Sorry for the re-open, but it would appears as if even though it doesn't crash, the NULL:s are actually converted into string "nil":

drop all function test_timestamp_with_nulls;

CREATE FUNCTION test_timestamp_with_nulls(actual_takeoff_datetime timestamp) RETURNS STRING LANGUAGE PYTHON {
import json
return json.dumps(actual_takeoff_datetime.tolist());
};

drop table if exists example;
CREATE TABLE example (
"column1" timestamp
);

insert into example ("column1") values ('2017-01-01 00:00:01');
insert into example ("column1") values (NULL);

select test_timestamp_with_nulls("column1") from example;

Gives:

+------------------------------------+
| L2 |
+====================================+
| ["2017-01-01 00:00:01.000", "nil"] |
+------------------------------------+

But should give:

+------------------------------------+
| L2 |
+====================================+
| ["2017-01-01 00:00:01.000", null] |
+------------------------------------+

I.e the null value shouldn't be converted to string "nil", (but probably a numpy or python NaN)

Running commit cc06e0c02cb1

Or is NIL actually expected behaviour? If so feel free to close.

Comment 26067

Date: 2018-01-10 16:43:46 +0100
From: MonetDB Mercurial Repository <>

Changeset 0dd61564584d made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=0dd61564584d

Changeset description:

Don't replace bat; updated test.
This fixes the reopened bug #6508.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant