Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RE: The DB_File locking method is flawed #601

p5pRT opened this issue Sep 20, 1999 · 2 comments

RE: The DB_File locking method is flawed #601

p5pRT opened this issue Sep 20, 1999 · 2 comments


Copy link

p5pRT commented Sep 20, 1999

Migrated from (status was 'resolved')

Searchable as RT1460$

Copy link

p5pRT commented Sep 20, 1999

From David

Hi David,

thanks for creating an *excellent* test harness to illustrate a nasty flaw
in the recommended method of locking DB_File databases. I had no problem
reproducing it here.

I agree with your analysis of why things aren't working as expected with the
current DB_File locking technique. The Berkeley DB cache is certainly the
root of the problem. Looks like the locking section in needs

WRT your DB_Wrap module, I need to have a play with it to see what its
limitations are.

I've cc'ed this to perl5-porters to widen the discussion on the merits of
DB_Wrap. For the benefit of p5p, DB_Wrap creates an external lock file for
the database (rather than locking the database itself) to control access.



forwarding this on to you...
Paul because you are the DB_File guy and
Stas because you asked me to CC you

This was just posted in comp.lang.perl.misc a few min ago.

- David Harris
Principal Engineer, DRH Internet Services

David Harris <dharris@​> wrote in message

I've been doing some work with the DB_File module and mod_perl and I've
found a bug in the "commonly accepted" procedure for locking a database
file. This procedure is outlined in the POD documentation for the DB_File
module and in the Programming Perl book where it covers the DB_File module
and other examples and guides. The bug can lead to database corruption.

The example locking method goes like this​:

$db = tie\(%db\, 'DB\_File'\, '/tmp/foo\.db'\, O\_CREAT|O\_RDWR\, 0644\)
    || die "dbcreat /tmp/foo\.db $\!";
$fd = $db\->fd;
open\(DB\_FH\, "\+\<&=$fd"\) || die "dup $\!";
flock \(DB\_FH\, LOCK\_SH\) || die "flock&#8203;: $\!";

The problem is that the database file is opened and then later locked --
when the database is opened the first 4k (in my dbm library) are read and
then cached in memory. Therefore, a process can open the database file and
cache the first 4k, then block in the flock while another process modifies
the first 4k of the file. When the original process gets the lock is now
a inconsistent view of the database, and if it writes using this
view, it may corrupt the database on disk.

This does not cause corruption every time a process has to block in the
flock call, because one can do quite a bit of writing to the database file
without actually changing the first 4k of the file.

I first saw this problem when I investigated using strace to see a listing
of all the system calls. I have verified that the tie line not only
an open system call, but also a read system call on the database file.

To be sure of the problem, I've created an example program shows database
corruption on disk when using this locking method. It simply forks off two
processes. The first gets the lock and writes a bunch of records to the
database, while the second blocks in the flock call. Then the first
finishes and the second writes a bunch of records. When the second process
is done, most of the records written by the first process are no longer in
the database. Stas Beckman (author of the mod_perl guide) has looked at
program and verified that he is also seeing corruption.

My example program, along with some documentation and the system call
are available in this archive​:


I believe that the correct fix for this problem is to simply gain a lock
before letting DB_File touch the database file. For my own work, I've
created a module to this effect called DRH​::DB_Wrap which wraps the
module and adds one argument to the tie command, which specifies what
locking should be done, "read" or "write". IHMO, this is a nice solution
the locking problem, and I'm wondering if it would be useful to others.
be glad to formally package it up (of course removing my internal "DRH​::"
prefix) if people think this would be useful.

You can get a copy of this DRH​::DB_Wrap module at​:


I'm looking forward to seeing this get fixed.

- David Harris
Principal Engineer, DRH Internet Services

Copy link

p5pRT commented Apr 22, 2003

@iabyn - Status changed from 'stalled' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet

No branches or pull requests

1 participant