Unable to recursively delete directories in OneDrive Business when Retention Policy is enabled #338
Comments
|
Also, the following was printed to STDERR (so not included in the log): onedrive.OneDriveException@src/onedrive.d(713): HTTP request returned status code 403 (Forbidden)
{
"error": {
"code": "accessDenied",
"innerError": {
"date": "2019-01-10T14:28:50",
"request-id": "b144d7a0-f0db-4027-a7c5-1b02dce79ff7"
},
"message": "Request was cancelled by event received. If attempting to delete a non-empty folder, it's possible that it's on hold"
}
}
----------------
src/onedrive.d:48 void onedrive.OneDriveApi.checkHttpCode(ref const(std.json.JSONValue)) [0x5c1f69]
src/onedrive.d:408 void onedrive.OneDriveApi.del(const(char)[]) [0x5c119f]
src/onedrive.d:222 void onedrive.OneDriveApi.deleteById(const(char)[], const(char)[], const(char)[]) [0x5c00d2]
src/sync.d:1700 void sync.SyncEngine.uploadDeleteItem(itemdb.Item, immutable(char)[]) [0x5d213b]
src/sync.d:1091 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf487]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1082 void sync.SyncEngine.uploadDirDifferences(itemdb.Item, immutable(char)[]) [0x5cf409]
src/sync.d:1059 void sync.SyncEngine.uploadDifferences(itemdb.Item) [0x5cf219]
src/sync.d:1016 void sync.SyncEngine.scanForDifferences(immutable(char)[]) [0x5cf001]
src/main.d:661 void main.performSync(sync.SyncEngine, immutable(char)[], bool, bool, bool) [0x5aa430]
src/main.d:482 _Dmain [0x5a9a49] |
|
@jacklovell |
|
@jacklovell Consider the following tree: Then I issue: Then I perform a new sync: Please can you provide more detailed reproduction steps that can trigger this issue 100% of the time. |
|
@jacklovell |
|
@jacklovell
Please can you run the above PR with the following command: Sanitise your logfile using: Depending on the 'volume' of files / sub folders, this could be an API issue where 'files / folders' with > than X number of entries take a while to delete, thus it would seem like it is attempting to delete non empty items for you. |
|
@jacklovell |
|
@abraunegg Thanks for looking into this. It seems to be an issue with the way OneDrive Business's retention policy is configured: see for example https://office365.uservoice.com/forums/289138-office-365-security-compliance/suggestions/19799911-onedrive-for-business-unable-to-perform-delete-fol. I also encounter this issue when I try to delete a non-empty folder in the OneDrive web interface (in my case, https://onedrive.com redirects to https://<employer name>-my.sharepoint.com/../onedrive.aspx). I get the following notification when the delete fails: It seems that I'm currently removing loads of files from my Onedrive, and it's taking a very long time to sync. Once I'm down to a more manageable amount of files, I'll checkout your PR above and provide some more detailed logs, if these are still useful. |
@jacklovell Do you know how to 'enable' this feature on OneDrive Business / SharePoint? I have looked around 'Site Settings' for a SharePoint Library and nothing stands out as being obvious. |
abraunegg
changed the title
|
I'm afraid I don't know much more about this feature. As I understand it, eDiscovery holds need to be in place, but this requires an administrator to change. I think this is the relevant documentation, but I don't have admin rights and so can't see the Security & Compiliance Centre options: https://docs.microsoft.com/en-gb/Office365/SecurityCompliance/ediscovery-cases?redirectSourcePath=%252fen-us%252farticle%252fmanage-ediscovery-cases-in-the-security--compliance-center-9a00b9ea-33fd-4772-8ea6-9d3c65e829e6. |
|
@jacklovell The current PR will start with providing what is the OneDrive response when a 'Retention Policy' is enabled. If you can run that PR against some test tree / test folder / test account that is affected that would be greatly appreciated. |
|
Attached is the log output of jlovell@jlovell-thinkpad ~ $ onedrive --version
onedrive v2.2.4-23-g9832702 |
|
@jacklovell |
|
@jacklovell The response from OneDrive should now be correctly captured & a warning message printed as to why the remote folder could not be deleted. Currently working through the right way on how to recursively delete files remotely when this situation occurs & currently have no way to test either. |
|
I've tested again this morning. Looks good: the error message and suggested workaround appear in both verbose ( |
* When OneDrive Business is configured with Retention Policies, attempts to delete a 'non empty' folder results in an an error response. Correctly handle the error response to print a warning message that OneDrive content must be manually deleted. **Note:** A future version of this client will handle the removal of 'non empty' folders when Retention Policies are enabled.
abraunegg
added
Fixed
Workaround Available
|
@jacklovell Workaround: A future version of onedrive will attempt to resolve this automatically. |
|
I belive I am getting this error too: My concern is that onedrive seems to exit upon this error. Being a background process (that I do not monitor), I would rather expect it to skip over it, even if the file is not synchronized & the client hits the error again & again. |
Which error is the client exiting as you have listed 2 different errors ? Whilst the first is logged as an "error" this is not a client error or issue - this is due OneDrive Business Retention Policy being applied to your OneDrive account and this causes this error. The second, again, whilst logged as an "error" this occurs because you have 2 folders with the same name according to the Microsoft naming conventions - the only way to solve this one is for you to rename your directories to be compliant. |
|
You are right: I am not absolutely sure that the error I mentioned is causing onedrive to exit at that point. Given that every few days I find onedrive stopped, I am trying to capture why it exited (or was killed). Note that I am getting these error on stderr rather than in the log file: you may want to fix it. I just reproduced the issue, but I do not find one drive exiting, so this might not be the actual issue I am facing. |
Most likely some sort of transient network access issue. It would be advisable to run the client in debug & https debug mode to determine further how and why the client is exiting as per https://github.com/abraunegg/onedrive/wiki/Generate-https-debug-log-for-support
Noted - however all of these are log.error - which goes to stderr .. so this is not a bug. Please raise a new issue for a change as this is not a bug, but would be treated as an enhancement. As your running as a service (given it is a background process) - if your running systemd, these errors as per stderr 'should' be logged to the systemd log file, thus logged. |
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
abraunegg
added
Fixed
and removed
Workaround Available
Bug Report Details
Describe the bug
After deleting folders locally,
onedrivefails to sync with the remote (O365) server. I get the following error message:This is due to
onedriveattempting to delete a top level folder before deleting all its contents. My current manual workaround is to delete all of the contents (find . -type f -delete), then synchronize, then loop deleting empty directories (find . -type d -empty -exec rmdir {} +) and synchronize, until there are no more empty directories. For a Onedrive account with many files, this is extremely time consuming.onedriveshould instead delete files and folders starting at the bottom of the hierarchy, which should avoid this problem.Application and Operating System Details:
dmd --versiononedrive --display-configcurl --versionTo Reproduce
Steps to reproduce the behavior if not causing an application crash:
onedrive --synchronizeComplete Verbose Log Output
A clear and full log of the problem when running the application in the following manner (ie, not in monitor mode):
Run the application in a separate terminal window or SSH session and provide the entire application output including the error & crash. When posing the logs, Please format log output to make it easier to read. See https://guides.github.com/features/mastering-markdown/ for more details.
Application Log Output:
Bug Report Checklist
The text was updated successfully, but these errors were encountered: