dbm.open(..., flag="n") does not work and does not give a warning

[sonyachiko]

BPO	18039
Nosy	@bitdancer, @PCManticore, @berkerpeksag, @serhiy-storchaka
Files	dbmopen.py test_dbm.py dbm_dumb_open.patch issue18039.patch: Use with in test and minor doc fixes issue18039_1.patch issue18039_2.patch issue18039_3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-06-10.19:22:15.224>
created_at = <Date 2013-05-23.00:39:58.025>
labels = ['type-feature', 'library']
title = 'dbm.open(..., flag="n") does not work and does not give a warning'
updated_at = <Date 2014-06-10.19:22:15.223>
user = 'https://bugs.python.org/sonyachiko'

bugs.python.org fields:

activity = <Date 2014-06-10.19:22:15.223>
actor = 'r.david.murray'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-06-10.19:22:15.224>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2013-05-23.00:39:58.025>
creator = 'sonyachiko'
dependencies = []
files = ['30435', '32454', '32455', '34359', '35544', '35549', '35555']
hgrepos = []
issue_num = 18039
keywords = ['patch']
message_count = 15.0
messages = ['189841', '190401', '190403', '201944', '201945', '217249', '219553', '220111', '220113', '220153', '220184', '220187', '220188', '220190', '220192']
nosy_count = 6.0
nosy_names = ['r.david.murray', 'Claudiu.Popa', 'python-dev', 'berker.peksag', 'serhiy.storchaka', 'sonyachiko']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue18039'
versions = ['Python 3.5']

[sonyachiko]

I use Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)] on win32.

dbm.open(..., flag="n") is supposed to "Always create a new, empty database, open for reading and writing".
In Windows case it does not. So I cannot empty a dbm database when I need it.

http://docs.python.org/3/library/dbm.html#module-dbm.dumb
Here it is written:
"The optional flag argument is currently ignored; the database is always opened for update, and will be created if it does not exist."

If it is ignored, there should be at least a warning. How am I supposed to now that this is a known bug?

[berkerpeksag]

I can't reproduce it on Windows 7 with Python 3.3.2.

Attaching my test script.

Here's the output:

C:\Python33>python.exe -V
Python 3.3.2

C:\Python33>python.exe dbmopen.py
<dbm.dumb._Database object at 0x027C15B0>
bzdew.dat exists? True
bzdew.dir exists? True
b'hello' b'there'

Could you run it and paste here the output?

[sonyachiko]

Here is the output.

*** Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)] on win32. ***
*** Remote Python engine  is active ***
>>> 
*** Remote Interpreter Reinitialized  ***
>>> 
<dbm.dumb._Database object at 0x02B95210>
yoqaA.dat exists? True
yoqaA.dir exists? True
b'hello' b'there'
>>> 

I would like to focus your attention, that flag "n" creates a database, but does not overwrite it once it is created. So in Windows case there is no difference between "c" and "n" flag.
You can have a look at this link where it was originally discussed.
http://stackoverflow.com/questions/16647131/how-to-empty-dbm-file-in-python-efficiently

[PCManticore]

Here's a patch which adds support for the n value of the flag. It makes the dbm consistent across implementations and it's thus more reliable.
Also, attached a script which replicates the problem for Windows (where dbm.dumb is used).

My output (after a couple of runs):

D:\Projects\snippets\DBM>py -3 a.py u7
test
[b'u3', b'u2', b'u5', b'u4', b'u7', b'u6']
test.dat exists? True
test.dir exists? True
b'u3' b'there'
b'u2' b'there'
b'u5' b'there'
b'u4' b'there'
b'u7' b'there'
b'u6' b'there'

[PCManticore]

And the patch.

[PCManticore]

Can anyone review this patch?

[PCManticore]

Serhiy, could you please have a look at this patch? Given the fact that you committed my last dbm patch, I hope you have a couple of minutes to have a look at this one as well.

[serhiy-storchaka]

With this patch 2 of 4 modes ('c' and 'n') work as for other dbm modules. So it would be better to document that 'r' and 'w' work nonstandard, than document that 'n' is not ignored.

And while you are here, may be add a warning in 'r' and 'w' mode when the file does not exist?

[PCManticore]

Thanks, Serhiy. Here's the new version of the patch. Hope that the warning message is clear enough.

[PCManticore]

Thanks for the reviews, Serhiy. Here's the new version of the patch.

[python-dev]

New changeset 3f944f44ee41 by Serhiy Storchaka in branch 'default':
Issue bpo-18039: dbm.dump.open() now always creates a new database when the
http://hg.python.org/cpython/rev/3f944f44ee41

[serhiy-storchaka]

Committed original Claudiu's patch (bpo-18039.patch) with minor changes. For warnings I opened separate issue (bpo-21708) because patch becomes too large.

Thank you Claudiu for your patch.

[bitdancer]

Isn't this change going to cause unexpected data loss for (possibly mythical) people depending on the existing behavior? At an absolute minimum it needs an entry in the What's New porting section, but I'm wondering if a deprecation period is more appropriate.

[serhiy-storchaka]

I think the chance of this is pretty small. I can imagine that some people use default, 'r' or 'w' mode unintentionally, and all works for them even when database is missed (for example our tests do this). But it is very unlikely that someone use the 'n' mode and expect that database will be not cleaned.

[bitdancer]

Yeah, hopefully you are right. (I didn't mean to reopen the issue, by the way).