Re\scan bug, some regular files treated as "local additions" (different), but they not. #8636
Comments
5x3
added
bug
|
Different doesn't just mean different in content, it also relates to timestamps and permissions. That may still be a bug, but we could use some more information. Can you grab debug output for an affected file and paste here? |
|
And please do include the Syncthing version. "Last" doesn't mean much, as just today a new release came out. |
timestamp ? MTIME? mtime the same since it was preserved on transfer. I re\check it twice with stat (man 1 stat). Selinux disabled on both ends. permission is exactly the same, uid\gid also, both PCs have same users with the same UID\GID settings exactly the same. No extended attributes defined, I re\check it also twice with lsattr (man 1 lsattr)
This is bug since out from 250 GiB files 54 GiB is stacked in "local additions", and "pc-sender" now have ethernal You can reproduce this by yourself, it reproduces easily.
this don't work as described, orig IDs in output obfuscated. please forward this to beta testers to confirm the bug for the future resolve. |
|
even send only + revert local changes is possible, gree. dear testers, use 250 GiB test dir with Linux OS data on ext4 as a test directory and many files with code and os data including many files 2M+, symlinks, hardlinks, device files, to test core re\sync functions, they broken, possibly by the design. |
|
I continue to dig into the problem this is same second, but looks like tar not supported this .723820588 appendix , what is this nanoseconds? no matter. I believe syncthing SHOUD round all MTIME (if MTIME involved to detecting change even if Ignore permission set) to who can check this in code and possibly fix ? P.S. much better behavior if MTIME different on a fractions of a second - compare checksum of file content, and if they just the same - do not treat as different file, not sure what is better by the code. will continue my finding since i am not shure that exactly this cause of the problem. |
|
also I found that it uses CTIME not the MTIME, which is totally wrong approach. |
This is probably related to the newly added support for syncing extended attributes. There are some issues with it currently (e.g. see my problem with conflicted copies at https://forum.syncthing.net/t/keep-getting-conflicts-generated-on-android-device-for-files-modified-only-on-a-desktop-pc/19060, and also https://forum.syncthing.net/t/conflicts-after-saving-a-file-from-a-single-machine/19265). The overall root of the problem here sounds similar to me, although in your case there is no Android involved. For now, I think the only solution/workaround will likely be to downgrade to v1.21.0 🙁. |
|
Yeah that discussion isn't really useful without the requested debug info, unfortunately. |
provided early here. |
|
Hi! I have the same issue: Android (1.22.1) and a Linux PC (1.22.1). Both nodes have "Ignore Permissions" set. This is the debug output for the file, executed from the PC: {
"availability": [
"7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4",
"XGZYEOG-GBQ4WSE-WXRGBW2-YK7EOOO-HEM42J5-XAQXUNG-NOLXNAD-L4CVFAY"
],
"global": {
"deleted": false,
"ignored": false,
"inodeChange": "2022-11-08T15:50:50.90502759+01:00",
"invalid": false,
"localFlags": 0,
"modified": "2022-11-08T15:50:50.895027722+01:00",
"modifiedBy": "KMSTLY5",
"mustRescan": false,
"name": "Daily/2022-11-08.md",
"noPermissions": true,
"numBlocks": 1,
"platform": {
"darwin": null,
"freebsd": null,
"linux": null,
"netbsd": null,
"unix": null,
"windows": null
},
"sequence": 1278,
"size": 147,
"type": "FILE_INFO_TYPE_FILE",
"version": [
"KMSTLY5:1667919060"
]
},
"globalVersions": "{{Version:{[{KMSTLY5 1667919060}]}, Deleted:false, Devices:{7777777, XGZYEOG}, Invalid:{}}}",
"local": {
"deleted": false,
"ignored": false,
"inodeChange": "2022-11-08T15:50:50.90502759+01:00",
"invalid": false,
"localFlags": 0,
"modified": "2022-11-08T15:50:50.895027722+01:00",
"modifiedBy": "KMSTLY5",
"mustRescan": false,
"name": "Daily/2022-11-08.md",
"noPermissions": true,
"numBlocks": 1,
"platform": {
"darwin": null,
"freebsd": null,
"linux": null,
"netbsd": null,
"unix": null,
"windows": null
},
"sequence": 1278,
"size": 147,
"type": "FILE_INFO_TYPE_FILE",
"version": [
"KMSTLY5:1667919060"
]
},
"mtime": {
"err": null,
"value": {
"real": "0001-01-01T00:00:00Z",
"virtual": "0001-01-01T00:00:00Z"
}
}
} |
|
@mattelacchiato thanks for the debug output, but that doesn't appear to be a "local addition". Is it taken from the device that actually shows the problem? |
|
@calmh Yes: I have the file |
|
Right. That's not a local addition, that's a conflict, caused by our current issue with Android, which there are a myriad threads about. |
|
Oh, okay... Sorry for bothering :-) |
|
other test shows some files are copyed to receiver-pc after rescan folder on sender-pc, and then copyed back to sender-pc after rescan folder on receiver-pc. ctime updating each time file writes on both pc - insanity, syncthing completely broken, at least on ext4, tested with ignore permission set and both with send&receive. you can test it easly, see "recent changes" on 1st pc, click rescan all folders, see "recent changes" again. another possilbe confirmation that if syncthing should fix bug with nanosecond part of timestamp on resync (explained early) and switch from CTIME to MTIME. if you have many files on Linux - you will get all this kind of mess - permission mess on folders while sync, rescan mess. |
|
You could help by providing the debug info requested earlier. We are still waiting on that from you. It does not help to claim "too much bugs", but please do help in debugging. There have been some recent fixes that might affect this. If you could collect the debug info for how it currently fails, please update to the latest release candidate version and see if it helps. |
|
I am closing this bug. Without the requested information there's nothing actionable here. Please open a topic on https://forum.syncthing.net for further assistance to debug this problem. Once a bug is identified, a targeted issue here on github can be opened. |
st-review
added
the
frozen-due-to-age
on Linux OS, last syncthing,
On
pc-sender, add any linux ext4 folder here with send only and share with pc-receiver,
copy shaded folder content with tar -p\rsync -a\cp -a (with maximum preserve) from pc-sender to pc-receiver
unpack data with preservation on pc-receiver, use md5sum\sha1sum\sha512sum or mtree to validate all files checksum on both pc-sender and pc-receiver - all should be equal.
On
pc-receiver, accept incoming folder request and add folder to copyed dir (already with data transferred), as receive only folder .
data have files that equal with sha512sum , both pc-sender and pc-receiver -- all files have same checksum on both PCs.
Look at web GUI - you will see "local additions" that will re-appears after rescan folder even after "Revert local changes".
I tested it with 250GiB folder and 2GiB folder, same result.
P.S. If you don't transfer data to pc-receiver and let syncthing to sync it from scratch - everyting would be fine - NO "local additions" will appear.
click "Revert local changes" - "local additions" disappears.
click Rescan on folder - "local additions" appears again, same file.
click rescan on pc-sender or turning on\off Ignore Permissions does not solve problem.
+
sha512 sum of every file on "local additions" show same checksum on both pc-sender and pc-receiver, what was expected.
this is Re\scan bug.
The text was updated successfully, but these errors were encountered: