Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable background scan for home storage/cache #28987

Merged
merged 1 commit into from
Oct 19, 2017

Conversation

PVince81
Copy link
Contributor

Description

The home cache might have entries with size=-1 that are outside of
"files/" due to encryption or other metadata that generally do not
require the size to be set or accurate.

Instead of adjusting the HomeCache to ignore entries outside of
"files/", let's just disable it completely. The home storage is in the
data folder and must not receive external changes, so it should never
have any entries with size=-1 in "files/".

Related Issue

Fixes #28981

Motivation and Context

See description and #28981 (comment)

How Has This Been Tested?

Steps here #28981 (comment).
Before the fix: integrity check error.
After the fix: deletion worked.

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

The home cache might have entries with size=-1 that are outside of
"files/" due to encryption or other metadata that generally do not
require the size to be set or accurate.

Instead of adjusting the HomeCache to ignore entries outside of
"files/", let's just disable it completely. The home storage is in the
data folder and must not receive external changes, so it should never
have any entries with size=-1 in "files/".
@DeepDiver1975
Copy link
Member

Instead of adjusting the HomeCache to ignore entries outside of
"files/", let's just disable it completely. The home storage is in the
data folder and must not receive external changes, so it should never
have any entries with size=-1 in "files/".

no idea what the current usage out there is these days, but the intention was that changes to the data folder are detected automagically.

never the less all this automagic feature are the one which are killing us these days.

so 👍

we need to properly document that in case there are changes to the data folder files:scan has to be executed to catch these changes.

next step might even be to kill the background job and replace it with an explicit cron/occ command ...

@jvillafanez
Copy link
Member

Does the problem affect to external storages? We might need a more generic solution if this is the case.

@PVince81
Copy link
Contributor Author

@DeepDiver1975 no, for the "data" folder we decided that no one should touch its contents as it's exclusiviely accessed by OC. This is why we set "filesystem_check_changes" to 0 (disabled) for the home storage.

@jvillafanez the problem this solves is related to the "files_encryption", etc entries in the home storage, so it will not affect external storages.

@PVince81
Copy link
Contributor Author

@jvillafanez basically we don't want to background scan the "data/$userId/files_encryption" folders in the home storage

@PVince81
Copy link
Contributor Author

we need to properly document that in case there are changes to the data folder files:scan has to be executed to catch these changes.

@kdslkdsaldsal do you remember whether we had this documented somewhere ?

We've already got used to tell people to not change data directly inside the data folder but I'm not sure it's documented yet and can't seem to find it right now.

@PVince81
Copy link
Contributor Author

some past ref from when we changed the expectation about data folder: #16897

@PVince81
Copy link
Contributor Author

raised owncloud-archive/documentation#3388 to document this

@ghost
Copy link

ghost commented Sep 12, 2017

Concerning the modification of the data dir there is a big red "The data directory on the server is exclusive to ownCloud and must not be modified manually." at:

https://doc.owncloud.org/server/latest/admin_manual/issues/general_troubleshooting.html#debugging-sync-issues

https://doc.owncloud.org/desktop/latest/troubleshooting.html#isolating-other-issues

But as always: People are mostly not reading the documentation and won't notice this in there.

@PVince81
Copy link
Contributor Author

@DeepDiver1975 please see comments above, does that shed enough light ?

@PVince81
Copy link
Contributor Author

@DeepDiver1975 please rereview

@lock
Copy link

lock bot commented Aug 2, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrity violation during parallel DELETE
4 participants