Conflict resolution, possible bug maybe?

[fernandoroa]

When I work some time without syncing or when resyncing, pseudo-conflicts (see below) arise and local files get renamed to ORIGINAL_FILENAME_MACHINE_NAME.EXTENSION. So, I tried the option --local-first, but it ignores the sync_list file, and uploads everything, which is also undesired.
Is there an option that acts as --local-first, i.e avoid renaming local files, and also considers the sync_list?
By the way, the "conflicts" that generate this file changes maybe bugs or pseudo-conflicts, because the files were not modified both locally and remotely, they just spend some time without being synced to the OneDrive Server.
So, if this is in fact a bug, another approach would be to identify better the pseudo-conflicts that arise for not syncing continuously some files, which would avoid the unwanted renaming of local files.

[abraunegg]

@fernandoroa
Firstly - what application version are you using?

[fernandoroa]

v2.4.25-1+np1

[abraunegg]

@fernandoroa
OK .. so what is your command you are using when these conflicts are occuring.

NOTE: You should never use --resync unless the application tells you too.

What sort of OneDrive account do you have - personal or business and/or are you syncing against a SharePoint device / folder?

In the v2.4.x code, a conflict file is created when there is a technical difference to the file online. SharePoint always modifies your file - whilst you may think no change has occured - there is. If you then use resync, you loose the state of the file, thus the backup occurs.

You can also bypass creating this backup - please read the documentation.

In the development of v2.5.x - there were several tweaks to enhance the conflict detection and backup creation. I highly suggest you give 'alpha-3' a run given its code maturity.

[fernandoroa]

Regarding --resync: I have used it following:

"After changing the sync_list, you must perform a full re-synchronization by adding --resync to your existing command line - for example: onedrive --synchronize --resync"

and also by application request.

Type of account: Business (university),
not a SharePoint

"Changes" or differences between file versions are apparent, i.e. an old version of the file (online) substitutes the new version (local) and the local gets renamed with the machine_name string. If I bypass the backup, which, sorry, I didn't find how to in the documentation, I would lose recent changes.

Thx for your suggestion 2.5.x,

what about implementing --local-first, to make it consider the sync_list?
am I right when I say --local-first ignores sync_list?

[abraunegg]

@fernandoroa
Nothing will be implemented in v2.4.x - as the focus is v2.5.x. This being said everything was re-written and sync_list evaluation (from memory) was tested with --local-first ... please read all the early threads - alpha-0,1,2 as to the test cases done.

[fernandoroa]

I had to resync after installing 2.5.
I got several timestamp errors, which were corrected when in dry run, but raised errors like this, when in normal run:

Processing FILENAME
The last modified timestamp has changed however the file content has not changed
The local item has the same hash value as the item online - correcting timestamp online

ERROR: Microsoft OneDrive API returned an error with the following message:
  Error Message:    HTTP request returned status code 412 (Precondition Failed)
  Error Reason:     The resource has changed since the caller last read it; usually an eTag mismatch
  Error Code:       resourceModified
  Error Timestamp:  2023-12-01T04:17:43
  API Request ID:   5f4aab61-e399-4d0f-9101-0f3fe99e4d46
  Calling Function: uploadLastModifiedTime()

When in dry run instead of the error, it just changes the timestamp of the online version

[abraunegg]

@fernandoroa
The reason is that in a dry run - nothing is technically changing, thus, no online push of dat / info is occuring - that is why there is a difference.

So what the v2.5.x output tells me is you are 100% uploading to a SharePoint backend ... your data is being modified online behind the scene .. however a 412 should not be generated like that.

Could you please generate a debug log using the v2.5.x code, as the the etag should not be used when pushing a timestamp modify - but it is .. hence probably the error ...

[fernandoroa]

Regarding the OP
The behavior was the same as before,
the local file was overwritten with the old online version, and the recent version got the name of the machine appended.

I think the resync was mandatory because of changing the app version.

onedrive --resync --sync --verbose

So, is there any command that keeps the local version as the source of truth when resync operations are mandatory?

[abraunegg]

@fernandoroa
Now that you are using v2.5.x - can you not use resync .. and retry your usecase?

You can also add --local-first to ensure your local data is being treated as the source of truth.

[abraunegg]

@fernandoroa

The last modified timestamp has changed however the file content has not changed
The local item has the same hash value as the item online - correcting timestamp online

...
The behavior was the same as before,
the local file was overwritten with the old online version, and the recent version got the name of the machine appended.

On this point: If you are using v2.5.x you can see that the only difference here is the timestamp. The code does not make a backup of the file if the timestamp is the only difference here - so please make sure you are not running any systemd service in the background running an older version .... as you are using a distribution package (v2.4.25-1+np1)

Secondly, the application is telling you there is no 'data difference' here ... if a backup file was being created, the logs and application output would 100% indicate this - so I am highly doubtful that any 'backup' file is being created by the v2.5.x codebase.

I think the resync was mandatory because of changing the app version.

No - the application upgrade would have 100% not required this activity - unless you also changed any of the items that trigger a resync to occur.

[fernandoroa]

The timestamp issue is not related to (really) conflicting updated files locally.

I had to do resync because after 2.5 installation drive_id in config file, cannot be empty, So a workaround was to delete config file, which implies resync afterwards.

I think I am now unable to do just sync because of that, which may turn the app unusable for me.

I checked in the OneDrive page if there is some sort of SharePoint (to add a drive_id), but I just see the normal folders.
But at it is a business/educational account, I guess it behaves different to common accounts.

The app seems to behave differently having a config file not modified (default) and having no config file at all, regarding the drive_id value error. But I am not sure

[abraunegg]

@fernandoroa

I had to do resync because after 2.5 installation drive_id in config file, cannot be empty, So a workaround was to delete config file, which implies resync afterwards.

That then means you are potentially using the OneDrive GUI - because that is what the v1.x version of that does - it writes out empty config options even if they are not being used. If you are using that, and you are sticking with v2.5.x of this code, you need to upgrade your GUI version to the latest release as per here: https://github.com/bpozdena/OneDriveGUI/releases

I think I am now unable to do just sync because of that, which may turn the app unusable for me

I do not think this is correct. Something is going on, and I am yet to see any evidence of what you are claiming (regarding the backup files being created) to help you work out what is occurring in regards to that.

[abraunegg]

@fernandoroa
Further to all of this conversation:

Regarding the OP
The behavior was the same as before,
the local file was overwritten with the old online version, and the recent version got the name of the machine appended.

Please provide a verbose debug log using v2.5.0-alpha-3 showing this.

Now - when you use --resync - you delete what the application knows is correct and valid and in-sync, thus you fall into the default category of the application not knowing what file should be the source of truth.

If the local file has any change + is newer (data has is different, timestamp is newer) than the online file, the local file will be backed up so you have a local backup copy when using --resync.

In a non-resync scenario, the application knows what the correct file version should be, thus, not create a backup - but will still create a backup if the local file has been modified in a way that the online version is different.

The key point here is - do not use --resync unless the application advises you to, or unless you 100% know why you want to use it.

Now .. you are saying that you are not consciously making any file changes to your local files. In prior investigations when this sort of activity was occurring, the user had file indexing going on. Given that you are using the OpenSuSE package - you are using a Debian based platform (hopefully not Ubuntu - but that is your choice). On Ubuntu, Mint and others, they run various file indexers.

All prior investigations when investigating 'not consciously making any file changes to your local files' have shown that these indexers modify your file locally ..... now you have said:

By the way, the "conflicts" that generate this file changes maybe bugs or pseudo-conflicts, because the files were not modified both locally and remotely, they just spend some time without being synced to the OneDrive Server.

If the application state is not being removed by using --resync (which you should not be using) - a local file that has not been modified by you is not going to cause a backup to be created. That is how the application works.

However .. a file indexer could be making these modifications behind the scene. These applications are 100% problematic because they are poorly coded in terms of how they track their 'indexing' status - as they modify your file:

• Dolphin File Indexer constantly accessing files and updating the modified timestamp. The process at fault was baloo_file_extractor. Every time this process ran, this created the issue.
• x_thumbnailer also appeared to touch and access every file
• kdeinit5 which is part of Okular appears to also touch and modify every file

If you are using any of these .. the issue you are facing if most likely being caused by these applications. Do not use them.

A further item to consider is WPS Office. WPS Office, when you save files, will delete the original file and save the current open application item as the same file name that was deleted. So .. lets say you open up a XLS document in WPS Office, make no change, click save, you have a 100% new file locally, but no change - newer timestamp, new file, potentially same file hash.

Another item here is - check your entire system for old application binaries - you are using the OpenSuSE package now - but what about in the past ... did you use the PPA, did you self compile? Please check your system for any other onedrive binary and/or rogue systemd process running that could be impacting your use here. There were older bugs, in older application versions where the file comparison aspect was not as complete as it should have been - but this was 99% correct in v2.4.25 and 100% re-written for v2.5.x

The last thought I have on this, now that you are using v2.5.0-alpha-3 ... please provide a verbose debug log (please read https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support) to illustrate the problem you are experiencing - so that this can be worked through and explain what is occurring so that if this is a bug or issue it can be fixed, or at least this can be explained as to why you are seeing this due to either how you are using the application or some other external influence is the cause.

[fernandoroa]

Thanks,
I suspect there are different scenarios in which resync is mandatory and almost unavoidable, such as having by mistake the the older GUI version and modifications to the config file. Or when adding/modifying the sync_list file.

So, I suppose a possible feature request would be to avoid --resync use after modifying sync_list, and regarding the other resyncs, I understand that they are needed after changes in config file.

So, the app is behaving as expected, creating a renamed copy when pseudoconflicts arise during resync. However, in the past I got one request from the app to resync, and I am not conscious of having modified sync_list or config. So, in a beginning I hoped the resyncing could somehow avoid creating copies derived from pseudoconflicts (feature request?), because I had the impression that the app would unpredictively request a resync.

[abraunegg]

@fernandoroa

I suspect there are different scenarios in which resync is mandatory and almost unavoidable, such as having by mistake the the older GUI version and modifications to the config file. Or when adding/modifying the sync_list file.

The requirements for v2.4.x when a --resync is required is detailed here: https://github.com/abraunegg/onedrive/blob/master/docs/USAGE.md#performing-a---resync

This is clarified in the v2.5.x documentation here: https://github.com/abraunegg/onedrive/blob/onedrive-v2.5.0-alpha-3/docs/usage.md#performing-a---resync

So, I suppose a possible feature request would be to avoid --resync use after modifying sync_list, and regarding the other resyncs, I understand that they are needed after changes in config file.

No - a feature request for this will be rejected.

Microsoft, wrote OneDrive and the API with the major assumption that is used with Windows - that everything stored online or locally is stored, verbatim in both places.

This client implements 'Client Side Filtering' so that a user can control what is synced with a fair degree of granularity.

If you make changes to these client side filtering rules - either to include or exclude 'stuff' .. the local cache database is not purged of those changes - thus it still thinks that needs to be included and/or done something with (keep|delete and so on) .. this is why, when you modify your 'client side rules' a --resync is required so that the local cache database uses your new rules as to what to include and keep.

So, the app is behaving as expected, creating a renamed copy when pseudoconflicts arise

When this is occurring with v2.5.x .. please provide verbose debug logs so that this can be worked through and explained as to what is occurring. It could be a bug, it could be a different use-case as to what has been tested, it could be something else entirely ... but the only way to understand what is occurring is to have these logs .. which so far have not been seen nor provided.

[fernandoroa]

I meant pseudoconflicts arise by resync, edited comment above.

[abraunegg]

@fernandoroa
Even if a conflict is detected, in a --resync scenario - the backup is only made if the file hash of the local item is different - that is logic that v2.5.x uses.

Again - if you are seeing this occur - with v2.5.x - please provide debug logs

[fernandoroa]

I understood this is the expected behavior, no need to debug.

If the local file has any change + is newer (data has is different, timestamp is newer) than the online file, the local file will be backed up so you have a local backup copy when using --resync.