embedded null byte when connecting to sqlite database using a bytes object

[ferferga]

BPO	40154
Nosy	@ferferga
Files	bytes_io.py: Proof of concept

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-08.17:43:28.121>
created_at = <Date 2020-04-02.12:23:03.625>
labels = ['extension-modules', 'invalid', 'type-bug', '3.8']
title = 'embedded null byte when connecting to sqlite database using a bytes object'
updated_at = <Date 2020-04-17.14:47:39.546>
user = 'https://github.com/ferferga'

bugs.python.org fields:

activity = <Date 2020-04-17.14:47:39.546>
actor = 'ferferga'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-08.17:43:28.121>
closer = 'SilentGhost'
components = ['Extension Modules']
creation = <Date 2020-04-02.12:23:03.625>
creator = 'ferferga'
dependencies = []
files = ['49025']
hgrepos = []
issue_num = 40154
keywords = []
message_count = 4.0
messages = ['365582', '365985', '365994', '366656']
nosy_count = 2.0
nosy_names = ['SilentGhost', 'ferferga']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40154'
versions = ['Python 3.8']

[ferferga]

Hello. I think that I found a bug in how sqlite3 module handle bytes.

The connect function of sqlite3 accepts strings, FilePath objects and bytes. However, it's impossible for me to connect to bytes objects that are read from BufferedReaders. I always get:

"ValueError: embedded null byte"

This is my current code (byteDec is the BytesIO object):

==============================================

byteDec.seek(0)
conn = sqlite3.connect(byteDec.read())

==============================================

That returns the "embedded null byte" error. However, if I do:

==============================================

byteDec.seek(0)
with open("db.db", "wb" as f:
    f.write(byteDec.read())
conn = sqlite3.connect("db.db")

==============================================

Everything works flawlessly, so the BufferedReader that I have in-memory is not corrupted in any way, as it's readable from a file. I want to avoid writing to disk at all, so this is not a solution for me.

I attach to this issue a very basic proof of concept to understand the issue.

I'm running Pyhton 3.8.2 amd64 on Windows 10 1909

[ferferga]

bump?

[SilentGhost]

Hi Fernando,

the first parameter of the connect function is described in documentation as follows:

database is a path-like object giving the pathname (absolute or relative to the current working directory) of the database file to be opened. You can use ":memory:" to open a database connection to a database that resides in RAM instead of on disk.

So, while it can be a bytes object, it's still would be a bytes object representing a file-path. It's not bytes object representing a file content of the database.

Hope that helps.

[ferferga]

Hello SilentGhost,

Okay, now I understand the difference and had my code working! Thank you very much for your answer and to all of you who help in making Python better.

(Wish I had more knowledge of it to help)

Have a nice day!