Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datastore corruption when the disk is full #1810

Closed
siSSSou opened this issue Nov 4, 2021 · 5 comments · Fixed by #1913
Closed

Datastore corruption when the disk is full #1810

siSSSou opened this issue Nov 4, 2021 · 5 comments · Fixed by #1913
Labels
bug it's broken!
Milestone

Comments

@siSSSou
Copy link

siSSSou commented Nov 4, 2021

The system drive went full (100%) this morning, and Shaarli is displaying this error instead of normal data :

An unexpected error happened, and the error template could not be displayed.

unserialize() expects parameter 1 to be string, bool given
/var/www/Shaarli/application/bookmark/BookmarkIO.php90
#0 /var/www/Shaarli/application/bookmark/BookmarkIO.php(90): unserialize(false)
#1 /var/www/Shaarli/application/bookmark/BookmarkFileService.php(71): Shaarli\Bookmark\BookmarkIO->read()
#2 /var/www/Shaarli/application/container/ContainerBuilder.php(97): Shaarli\Bookmark\BookmarkFileService->__construct(Object(Shaarli\Config\ConfigManager), Object(Shaarli\History), Object(malkusch\lock\mutex\FlockMutex), true)
#3 /var/www/Shaarli/vendor/pimple/pimple/src/Pimple/Container.php(118): Shaarli\Container\ContainerBuilder->Shaarli\Container\{closure}(Object(Shaarli\Container\ShaarliContainer))
#4 /var/www/Shaarli/vendor/slim/slim/Slim/Container.php(109): Pimple\Container->offsetGet('bookmarkService')
#5 /var/www/Shaarli/vendor/slim/slim/Slim/Container.php(164): Slim\Container->get('bookmarkService')
#6 /var/www/Shaarli/application/container/ContainerBuilder.php(110): Slim\Container->__get('bookmarkService')
#7 /var/www/Shaarli/vendor/pimple/pimple/src/Pimple/Container.php(118): Shaarli\Container\ContainerBuilder->Shaarli\Container\{closure}(Object(Shaarli\Container\ShaarliContainer))
#8 /var/www/Shaarli/vendor/slim/slim/Slim/Container.php(109): Pimple\Container->offsetGet('pageBuilder')
#9 /var/www/Shaarli/vendor/slim/slim/Slim/Container.php(164): Slim\Container->get('pageBuilder')
#10 /var/www/Shaarli/application/front/controller/visitor/ErrorController.php(20): Slim\Container->__get('pageBuilder')
#11 [internal function]: Shaarli\Front\Controller\Visitor\ErrorController->__invoke(Object(Slim\Http\Request), Object(Slim\Http\Response), Object(TypeError))
#12 /var/www/Shaarli/vendor/slim/slim/Slim/App.php(728): call_user_func_array(Object(Shaarli\Front\Controller\Visitor\ErrorController), Array)
#13 /var/www/Shaarli/vendor/slim/slim/Slim/App.php(396): Slim\App->handlePhpError(Object(TypeError), Object(Slim\Http\Request), Object(Slim\Http\Response))
#14 /var/www/Shaarli/vendor/slim/slim/Slim/App.php(297): Slim\App->process(Object(Slim\Http\Request), Object(Slim\Http\Response))
#15 /var/www/Shaarli/index.php(176): Slim\App->run(true)
#16 {main}

Using an older datastore.php I get normal behavior of the app.

I suppose that current datastore.php has been corrupted when the storage incident happened !
Is it possible to recover these data in datastore.php ?

NB : sadly I make daily bachups of mysql databases but not files... I know it's dumb ;-(

@nodiscc nodiscc added the bug it's broken! label Nov 12, 2021
@nodiscc
Copy link
Member

nodiscc commented Nov 12, 2021

Hi,
known problem https://github.com/shaarli/Shaarli/issues?q=is%3Aissue+data+loss+is%3Aclosed, a full disk corrupts the datastore, but it's supposed to be fixed in #1570 if I remember correctly... which Shaarli version are you using and how did you install it?

sadly I make daily bachups of mysql databases but not files

Yep. https://shaarli.readthedocs.io/en/master/Backup-and-restore/

@siSSSou
Copy link
Author

siSSSou commented Nov 18, 2021

Hi, known problem https://github.com/shaarli/Shaarli/issues?q=is%3Aissue+data+loss+is%3Aclosed, a full disk corrupts the datastore, but it's supposed to be fixed in #1570 if I remember correctly... which Shaarli version are you using and how did you install it?

I installed 0.12.1 from release ZIP.

But I re-installed it and set up a cron backup.
I suppose my issue can be closed now ;-)

@nodiscc
Copy link
Member

nodiscc commented Dec 17, 2021

Leaving this open so I can recheck/try to reproduce datastore corruption when the disk is full.

@nodiscc nodiscc self-assigned this Dec 17, 2021
@nodiscc nodiscc added this to the 1.0.0 milestone Dec 17, 2021
@wolfskaempf
Copy link

I can confirm that attempting a write operation from the web UI (add new bookmark) while the disk is full corrupted my datastore.php which is now size 0. Luckily I have a recent backup though.

@nodiscc nodiscc pinned this issue Nov 16, 2022
@nodiscc nodiscc changed the title Following a disc space problem, an unexpected error happened Datastore corruption when the disk is full Nov 16, 2022
@nodiscc nodiscc removed their assignment Nov 16, 2022
ArthurHoaro added a commit that referenced this issue Nov 25, 2022
On every disk write operation, we check the available free disk space (+ slight marging of arbitrary 500kB),
and if there is not enough space we do not attempt to write anything.

This should prevent most case of corrupted datastore becoming an empty file due to full HDD.

Fixes #1810
@ArthurHoaro ArthurHoaro modified the milestones: 1.0.0, 0.12.2 Nov 25, 2022
@wolfskaempf
Copy link

@ArthurHoaro Thank you so much for fixing this, I really appreciate it!

@nodiscc nodiscc unpinned this issue Dec 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug it's broken!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants