race condition when sqlite database is created #626
Comments
|
Hello,
Unfortunately I can't do anything about that to be honest.
Sqlite is not really designed for this kind of use. Your case study make me
feel that you need to use a more concurrent backend such as Files or
memory-based backend such as Redis, Ssdb, etc.
Cheers,
Georges
…On Tue, Jul 3, 2018, 12:18 AM Michael Biebl ***@***.***> wrote:
Version: 7.0.1
I'm using phpfastcache with php 7.2 under IIS/Windows Server 2016 using
php in CGI mode.
The backend is sqlite.
If the sqlite database does not exist yet, and I have multiple clients
access the server at the same time, there seems to be a race condition when
the database is created, as I get the following in the php error log:
[02-Jul-2018 17:58:28 UTC] PHP Fatal error: Uncaught ErrorException: mkdir(): File exists in C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastcache\Core\Pool\IO\IOHelperTrait.php:107
Stack trace:
#0 [internal function]: exception_error_handler(2, 'mkdir(): File e...', 'C:\\src\\Gal2Go-s...', 107, Array)
#1 C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastcache\Core\Pool\IO\IOHelperTrait.php(107): mkdir('C:\\Windows\\TEMP...', 511, true)
#2 C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastcache\Drivers\Sqlite\Driver.php(74): Phpfastcache\Drivers\Sqlite\Driver->getPath()
#3 C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastcache\Drivers\Sqlite\Driver.php(82): Phpfastcache\Drivers\Sqlite\Driver->getSqliteDir()
#4 C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastcache\Core\Pool\DriverBaseTrait.php(70): Phpfastcache\Drivers\Sqlite\Driver->driverCheck()
#5 C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastc in C:\src\Gal2Go-server\localhost\vendor\phpfastcache\phpfastcache\lib\Phpfastcache\Core\Pool\IO\IOHelperTrait.php on line 107
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#626>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABRTZ1p22hN7dNKygpZkiF6WhXoPJ7Mqks5uCpwugaJpZM4VABQg>
.
|
|
I certainly feel like this is a bug.It's not a problem of sqlite itself, but how the sqlite db is created in the first place. |
|
as for ssdb, its upstream authors do not recommend to run it on windows. |
|
The Files backend is well-tested against concurrency in multiple website on
my side (including a million visit/month one) and I never encountered this
issue before. This is a known issue with Sqlite which badly supports
concurrency due to its design itself.
Plus, Files drivers has both options (disabled by default) against cache
slam protection and secured file manipulation.
…On Tue, Jul 3, 2018, 1:28 AM Michael Biebl ***@***.***> wrote:
I certainly feel like this is a bug.It's not a problem of sqlite itself,
but how the sqlite db is created in the first place.
Fwiw, I've also tried the files backend, and it was even worse in that
regard.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#626 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABRTZzyqWivX7uNKLl5sH3h3JspKweqAks5uCqyagaJpZM4VABQg>
.
|
|
again, it's not about sqlite itself behaving badly with concurrent access. |
|
Regarding the files backend, I've just reran the tests: So it seems to suffer from a similar issue. Turning on |
|
I'm not gently sliding to trolling, but Windows & PHP environment are not know to be very friendly since many extension are not even supported or simply not updated since a while (Memcache, Xcache, APC, etc). It looks like you're option-limited and since you looks like developing locally, I'm advising you to use some Vagrant/Docker VM to run your app under Unix env which would save you further nightmare not only about Phpfastcache. The problem with Sqlite is that the creation of the database cannot be concurrent since we're on a unique binary file unlike every else backend drivers :/ |
|
Have you seen the response regarding the Files backend? It seems to have the same problem. |
|
What can I say: You are the first reporting me this issue after 3 major release, some users I know are using it on highly sensitive productions, I'm myself using the Files driver on a million visitor per month website (with an average of 10~15Mo/s rw I/O), and a couple of API apps and this kind of error never appeared once on my side. If you can identity some issue I would be glad and thankful but actually, at this point, I'm completely unable to reproduce your case. |
|
I just ran 10 tests in the exact same time in my company using 10000 items each and no errors appeared using Files driver (On a Win10x64 computer). Sqlite succeeded too despite its horrible time-spend. Honestly I'm not sure how to reproduce your issue ATM, at least give me a code sample or open a pull request if you identified the potential cause. |
|
I think the key to reproduce the problem is to remove the cache files, then start multiple clients at once (in my case I use a jmeter test which simulates 500 concurrent users with no ramp up time). |
|
Obviously if you run 500 concurrent requests in the exact same time (which is very unlikely, even with 1M visitor/month this never happened) you are clearly looking for troubles ;) In your case indeed there's no way on my level to fix that bug even by putting some kind of .tmp file to queue the requests because even the .tmp file would be overridden. Your case clearly suggest me to encourage your to move to a stronger cache backend that handle concurrency far better than Sqlite. Redis/SSdb are doing perfectly the job as much are Cassandra, Couchdb and Couchbase are doing it. |
|
The whole list of supported backends ATM is available below: |
|
Well, a fix for that would be to create a lock which is released once the db is created |
|
Yes but all your concurrent requests would create the lock in the same time. Making the fix useless :/ |
|
You try to acquire a lock and you are blocked if it's already taken. Only one can take it, not all of them. |
|
Well, a lock is not reliable at all since some system put the lock at the process-level and not at the thread-level. Php being frequently configured in a fcgi mode this would not be reliable... |
Yeah, the load test was intended to look for trouble or rather problems in our application. |
|
I'm usually always trying to improve Phpfastcache performance and efficiency by respecting at least three fundamental rules:
Thanks for your report, this is always appreciated even in the case I can't fix it immediately for any reason whatsoever. Cheers, |
Version: 7.0.1
I'm using phpfastcache with php 7.2 under IIS/Windows Server 2016 using php in CGI mode.
The backend is sqlite.
If the sqlite database does not exist yet, and I have multiple clients access the server at the same time, there seems to be a race condition when the database is created, as I get the following in the php error log:
The text was updated successfully, but these errors were encountered: