Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[stable9] Filecache repair step #29491

Merged
merged 21 commits into from Nov 8, 2017

Conversation

@PVince81
Copy link
Member

PVince81 commented Nov 7, 2017

Re-re-send of #29489 for CI

@tomneedham

PVince81 and others added 19 commits Jun 28, 2017
Whenever a parent-child relationship describes a specific path but the
entry's actual path is different, this new repair step will adjust it.

In the event where another entry already exists under that name, the
latter is deleted and replaced by the above entry. This should help
preserve the metadata associated with the original entry.
When fixing failed cross-storage moves in the file cache and using the
storage id filter, we filter by target storage for phase 1. However, we
also need to fix the source storages in phase 2. To do so, a list of
affected source storages is now gathered in phase 1 to be run on phase
2.
This instead of iterating over all storages which is way less efficient
due to the 1-N nature of potential failed cross-storage moves that we
are repairing.

If singleuser mode is enabled and "--all --repair" is passed, all
storages will be repaired in bulk (no repair filter). If not, it will
fall back to iterating over each storage which is slower.
…or isolation
@PVince81

This comment has been minimized.

Copy link
Member Author

PVince81 commented Nov 7, 2017

All green. But would be good to address my comments from the original PR: #29045 (review)

@PVince81 PVince81 mentioned this pull request Nov 7, 2017
7 of 15 tasks complete
private function countResultsToProcessParentIdWrongPath($storageNumericId = null) {
$qb = $this->connection->getQueryBuilder();
$qb->select($qb->createFunction('COUNT(*)'));
$this->addQueryConditionsParentIdWrongPath($qb, $storageNumericId);

This comment has been minimized.

Copy link
@tomneedham

tomneedham Nov 8, 2017

Member

@PVince81 can you check this - PHPStorm shows that this second param is not used in the method 👀

This comment has been minimized.

Copy link
@PVince81

PVince81 Nov 8, 2017

Author Member

ohh... some old code it seems. In the past we passed around the storage numeric id, but for this one method it just uses the local attribute now.

This comment has been minimized.

Copy link
@PVince81

PVince81 Nov 8, 2017

Author Member

nothing to worry about: the $this->storageNumericId is also passed from the outside for some methods, while other methods use the attribute directly... a bit messy, yes

->where($qb->expr()->eq('fileid', $qb->createNamedParameter($fileId)));
$qb->execute();
$text = "Fixed file cache entry with fileid $fileId, set wrong path \"$wrongPath\" to \"$correctPath\"";

This comment has been minimized.

Copy link
@PVince81

PVince81 Nov 8, 2017

Author Member

unused code

$storageIds = [];
foreach ($rows as $row) {
$storageIds[] = $row['storage'];

This comment has been minimized.

Copy link
@PVince81

PVince81 Nov 8, 2017

Author Member

you killed too much, the upgrade is not reporting any more.

We need to emit an event on $this object as it's a BasicEmmiter, it will output the entry on the console

@PVince81

This comment has been minimized.

Copy link
Member Author

PVince81 commented Nov 8, 2017

  • BUG: missing output, need using basicemitter
  • BUG: run occ files:scan causes error:
Exception during scan: Undefined variable: out
#0 /srv/www/htdocs/owncloud/lib/private/repair/repairmismatchfilecachepath.php(507): OCA\Files\Command\Scan->exceptionErrorHandler(8, 'Undefined varia...', '/srv/www/htdocs...', 507, Array)
#1 /srv/www/htdocs/owncloud/apps/files/command/scan.php(157): OC\Repair\RepairMismatchFileCachePath->run()
#2 [internal function]: OCA\Files\Command\Scan->OCA\Files\Command\{closure}(Object(OCA\Files_Trashbin\Storage))
#3 /srv/www/htdocs/owncloud/lib/private/hooks/emittertrait.php(98): call_user_func_array(Object(Closure), Array)
#4 /srv/www/htdocs/owncloud/lib/private/hooks/publicemitter.php(32): OC\Hooks\BasicEmitter->emit('\\OC\\Files\\Utils...', 'beforeScanStora...', Array)
#5 /srv/www/htdocs/owncloud/lib/private/files/utils/scanner.php(170): OC\Hooks\PublicEmitter->emit('\\OC\\Files\\Utils...', 'beforeScanStora...', Array)
#6 /srv/www/htdocs/owncloud/apps/files/command/scan.php(207): OC\Files\Utils\Scanner->scan('/user2', true)
#7 /srv/www/htdocs/owncloud/apps/files/command/scan.php(290): OCA\Files\Command\Scan->scanFiles('user2', '/user2', false, Object(Symfony\Component\Console\Output\ConsoleOutput), true)
#8 /srv/www/htdocs/owncloud/3rdparty/symfony/console/Command/Command.php(259): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /srv/www/htdocs/owncloud/core/command/base.php(158): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /srv/www/htdocs/owncloud/3rdparty/symfony/console/Application.php(840): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /srv/www/htdocs/owncloud/3rdparty/symfony/console/Application.php(192): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /srv/www/htdocs/owncloud/3rdparty/symfony/console/Application.php(123): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /srv/www/htdocs/owncloud/lib/private/console/application.php(146): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /srv/www/htdocs/owncloud/console.php(96): OC\Console\Application->run()
#15 /srv/www/htdocs/owncloud/occ(11): require_once('/srv/www/htdocs...')
#16 {main}
@PVince81 PVince81 assigned PVince81 and unassigned tomneedham Nov 8, 2017
@PVince81

This comment has been minimized.

Copy link
Member Author

PVince81 commented Nov 8, 2017

Fixed the issues and worked.

@PVince81

This comment has been minimized.

Copy link
Member Author

PVince81 commented Nov 8, 2017

@tomneedham quick review ? 002417a

@tomneedham

This comment has been minimized.

Copy link
Member

tomneedham commented Nov 8, 2017

Nice - and we keep some output :) 👍

@PVince81 PVince81 merged commit 69176e8 into stable9 Nov 8, 2017
3 checks passed
3 checks passed
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla Contributor License Agreement is signed.
Details
@PVince81 PVince81 deleted the stable9-repair-filecache-resend-once-again branch Nov 8, 2017
@lock

This comment has been minimized.

Copy link

lock bot commented Aug 2, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
3 participants
You can’t perform that action at this time.