Skip to content

Conversation

@DerDakon
Copy link
Member

@DerDakon DerDakon commented Mar 6, 2020

The code checked if a file with the same name already exists and then tries to create it. It properly checks if the latter fails and retries later, but this is totally unnecessary: it could just try to create it, and handle it at this point if the file already exists. This also saves one stat() syscall per local delivery.

Depends on #113.

@DerDakon DerDakon added the bug Something isn't working label Mar 6, 2020
@DerDakon DerDakon force-pushed the Dakon-local-race branch 3 times, most recently from 0373a78 to 956b31e Compare May 14, 2020 16:01
@DerDakon DerDakon requested review from josuah and leahneukirchen May 24, 2020 17:57
Copy link

@josuah josuah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The string handling for path creation and file descriptor are interleaved, and with the changes the order has moved, but reading multiple times, I cannot see any change of behavior.

@DerDakon DerDakon force-pushed the Dakon-local-race branch from 956b31e to 098cc72 Compare May 28, 2020 06:22
@DerDakon
Copy link
Member Author

Would be nice to hear the opinion of @xenotrope and @schmonz given their work on #90.

@xenotrope
Copy link

Would be nice to hear the opinion of @xenotrope and @schmonz given their work on #90.

I read this code about six or seven times to see what it's doing. Classic qmail uses good, not great, uniqueness in picking Maildir filenames; The "stat(fntmptph,&st)" call loops 3 times (loops 0, 1, and 2) trying to pick a time-and-pid-combination filename that does not exist, then called open_excl on it, which is wrapper around "open(fn,O_WRONLY | O_EXCL | O_CREAT,0644)". O_EXCL will "Error if O_CREAT is set and file exists."

In short, the open_excl() will fail and call "_exit(1)" if fntmptph exists, same as in the stat() loop. Removing the stat() doesn't save a syscall so much as it forces another fork() if the file already exists...

...but, this PR keeps the sleep(2) and moves the open_exl() into the loop, so I really can't see how this behavior is dramatically different. I haven't perf tested existing file failure reporting speeds between stat() and open(). That's the only real difference here with respect to catching and failing when fntmptph is already there.

(Depending on the qmail server itself, you could possibly have multiple maildir_child() children spawning in the same directory within one second; PID recycling is not something that many OSes guarantee won't happen. Increasing Maildir uniqueness with other sources of randomness is a good way prevent this logic from ever needing to actually be useful.)

@DerDakon
Copy link
Member Author

DerDakon commented Jun 9, 2020

The "saves one stat()" is the good case. The good case previously is: check if the file exists with stat(), then open(). The stat() is now avoided. The bad case is previously: stat() fails, succeeds in next loop, then open() fails because of the race and things cost an extra fork() and everything on retry. Now we do open(), which fails in the first loop, then succeeds and now we own that file and there is no race.

@schmonz
Copy link
Member

schmonz commented Jun 11, 2020

I’m not smart enough to read this and be sure about it. Automated tests of some sort would be awesome here, if you’re up for it.

@DerDakon DerDakon force-pushed the Dakon-local-race branch 2 times, most recently from f2a77d6 to 8049530 Compare June 18, 2020 17:09
@DerDakon DerDakon force-pushed the Dakon-local-race branch 2 times, most recently from 45555a9 to af02dc0 Compare July 8, 2020 17:42
The code checked if a file with the same name already exists and then tries to
create it. It properly checks if the latter fails and retries later, but this is
totally unnecessary: it could just try to create it, and handle it at this point
if the file already exists. This also saves one stat() syscall per local
delivery.
@schmonz schmonz removed the request for review from alanpost November 3, 2020 15:02
@schmonz
Copy link
Member

schmonz commented Nov 3, 2020

If you're still confident in this, let's merge it.

@DerDakon DerDakon merged commit 129357b into master Nov 3, 2020
@DerDakon DerDakon deleted the Dakon-local-race branch November 3, 2020 17:23
@schmonz schmonz modified the milestones: 1.90, 1.09 Sep 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants