-
-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PDOException: SQLSTATE[HY000]: General error: 5 database is locked #61
Comments
Use |
I have an index file.I used this command.I get this error when I search later.Could it be because of the query at the same time? @team5ru [2017-01-16 13:10:18] local.ERROR: PDOException: SQLSTATE[HY000]: General error: 5 database is locked in /var/www/laravel/vendor/teamtnt/tntsearch/src/TNTSearch.php:404 |
Same issue here, I use a single queue to handle indexing so concurrency on insert/update would be weird. What could be the issue here?
|
Hello, Same issue here as well.
How to proceed in this scenario? |
@richardblondet removing the tntsearch db files and rebuilding the db solves the issue most of the time, seems likes it gets corrupted by concurrency now and then.. |
I am experiencing the same thing. For me it is using multiple queue workers through supervisor and using sqlite as the index db. Only one worker gets to work on the database at a time. I think setting PRAGMA journal_mode=wal; will solve the problem. But I have not tested. |
Same error here. |
For anyone still perplexed by this, it's because you're creating too many models too quickly and it's indexing just as fast. Simple solution is to bring the app down with |
I have queue enabled and the same thing happens to be. Is there was a way to disable indexing on the model and only have it work when running the index manually? |
Still having this problem, any news? |
Any news on this? I think this can be solved with a sleep wait time for updating the sqlite index. This issue makes this driver unusable |
Same issue here. |
Same here. |
same |
For anyone wondering i just find the issue in my case: Seems that both handle the queue so its causing that sometimes two proceses try to access to the same database |
As a good tip you can use the command |
Had the same problem: If the supervisor is used (https://laravel.com/docs/5.6/queues#supervisor-configuration) and config parameter numprocs is set to more than 1 (8 by laravel docs). When indexing a lot of models (not by artisan bulk import command but for example by ENTITY::select()->searchable();) scout chunks the models (see config/scout.php):
and creates multiple jobs, each 500 models. Now the 8 running queue workers start to pickup these job in parallel or at least they do not wait for the first job to finish and want to access the sqlite db at the same time. Solution: Define an extra queue (i am using my own searchable trait and job, but config scout.queue.queue should do the job as well) and configured a second "program" in the supervisor to run only this queue with numprocs=1 Now there should be 8 queue workers/listeners for the default queue and 1 for the search index queue. |
let me add an example for usage of //config/scout.php
'queue' => [
'queue' => 'search'
], //config/horizon.php
'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'simple',
'processes' => 10,
'tries' => 3,
],
'supervisor-2' => [
'connection' => 'redis',
'queue' => ['search'],
'balance' => 'simple',
'processes' => 1,
'tries' => 3,
],
], But there is one problem: you cannot disable queuing via env. laravel/scout#289 |
Experiencing the same issue here. Happens when I loop through a large amount of JSON data and try to do a big relationship sync with them. |
Ok... I may have resolved this issue on my side, that was only on prod server by the way. |
For those coming here for solutions... in my case I was custom building the |
My specific issue was that the |
For anyone still getting this error and using queued jobs with more than 1 worker, you can now prevent the search indexing jobs from overlapping so they don't all try to access the sqlite database at the same time (which is the root of the error). Add these 2 jobs to your app: <?php
declare(strict_types=1);
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\Middleware\WithoutOverlapping;
use Illuminate\Queue\SerializesModels;
use Laravel\Scout\Jobs\MakeSearchable as MakeSearchableBase;
class MakeSearchable extends MakeSearchableBase
{
use Dispatchable;
use InteractsWithQueue;
use Queueable;
use SerializesModels;
/**
* @return array<int, object>
*/
public function middleware(): array
{
return [
(new WithoutOverlapping('scout.index'))
->shared()
->releaseAfter(10)
->expireAfter(30),
];
}
} <?php
declare(strict_types=1);
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\Middleware\WithoutOverlapping;
use Illuminate\Queue\SerializesModels;
use Laravel\Scout\Jobs\RemoveFromSearch as RemoveFromSearchAliasBase;
class RemoveFromSearch extends RemoveFromSearchAliasBase
{
use Dispatchable;
use InteractsWithQueue;
use Queueable;
use SerializesModels;
/**
* @return array<int, object>
*/
public function middleware(): array
{
return [
(new WithoutOverlapping('scout.index'))
->shared()
->releaseAfter(10)
->expireAfter(30),
];
}
} Then in a service provider's \Laravel\Scout\Scout::makeSearchableUsing(MakeSearchable::class);
\Laravel\Scout\Scout::removeFromSearchUsing(RemoveFromSearch::class); Relevant issue: #151 |
Why am I experiencing such an error in the log file? Available in the index storage file
The text was updated successfully, but these errors were encountered: