Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL lexer fails to detect string end if it the last character is U+FEFF ZERO WIDTH NO-BREAK SPACE #3641

monetdb-team opened this issue Nov 30, 2020 · 0 comments


Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2014-12-17 00:35:11 +0100
From: sorear
To: SQL devs <>
Version: 11.19.7 (Oct2014-SP1)

Last updated: 2015-01-29 14:07:41 +0100

Comment 20533

Date: 2014-12-17 00:35:11 +0100
From: sorear

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Build Identifier:

If the last character of a string literal is a U+FEFF, then the lexer will spuriously generate a "waiting for more input" response.

Reproducible: Always

Steps to Reproduce:

  1. start mclient
  2. enter "select ' ';", run it, see it return a value
  3. paste a zero-width no-break space (aka a "byte order mark") immediately before the closing quote, and try to run it again

Actual Results:

MonetDB considers the version with just spaces to be a complete SQL statement, but responds to the version with the ZWNBS with a secondary prompt (my actual code uses MAPI, and receives a \1\2\n).

Expected Results:

Both versions are complete SQL statements.

MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved
Visit for further information
Found 1.8GiB available memory, 2 available cpu cores
libpcre: 7.8 2008-09-05 (compiled with 7.8)
openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11 Feb 2013)
libxml2: 2.7.6 (compiled with 2.7.6)
Compiled by: (x86_64-unknown-linux-gnu)
Compilation: gcc -O3 -fomit-frame-pointer -pipe -march=native -D_FORTIFY_SOURCE=2
Linking : /usr/bin/ld -m elf_x86_64

Comment 20545

Date: 2014-12-23 09:51:22 +0100
From: MonetDB Mercurial Repository <>

Changeset 72699051b01e made by Sjoerd Mullender in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=72699051b01e

Changeset description:

The server should not treat U+FEFF specially, that's an interface issue.
This fixes bug #3641.
Also added test.

Comment 20607

Date: 2015-01-29 14:07:41 +0100
From: @sjoerdmullender

Oct2014-SP2 has been released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant