Nextcloud-Client creating conflicts when it should not #2467
Comments
|
Same is happening here with Win10, Nextcloud Desktop 3.0.1 and Nextcloud Server 13.0.5. |
|
Same happening in Win10 , Nextcloud Desktop 3.0.1 and NC Server 16 17 18 19 |
|
Another example conflict. This conflict should not happen. The files changing on the client are due to activity. The only files changing on the server are the nextcloud-client uploads to the server.
As a separate issue, "Click for details" has no effect. A debug log is here |
|
After several conflicted copy incidents we have been observing randomly timestamp changes on some files on the storage backend. By appointing So as the problem of @Bockeman occurs also with files that are NFS mounted to gluster we should be looking more closely at the reliability of storage backends. |
|
I'm having this problem with Android Client and Docker server installation (20.0.4). Sorry, I don't understand the nature of the problem but is it not possible to obtain a MD5 sum in case of conflict to let the user know that the changes are related to date (access, creation, whatever) and not content? Even a option to ignore date changes would be fine. |
|
I am having this issue with the Windows 10 laptop of my wife quite often (at least a dozen times in within the last 12 months):
Conflict resolution is also really annoying as i can only resolve one file at a time:
Once i've finished manually resolving the conflicts everything seems to be fine for some time until my wife calls me again because the conflicts are back. No problems with our Linux and Android clients ... |
|
We could nail down the problem with an external storage that was changing the wrong timestamp. Instead of |
I advise everyone here to check this kind of things. Indeed with an external storage behaving that way there's nothing else the client can do but report conflicts. |
|
This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you! |
|
I'm now on Nextcloud Client version 3.1.3 and I am still getting conflicts. The general case appears to be when client files change during a Nextcloud sync, with one client and (obviusly) one server. In my opinion, if there is only one client, there should never be any conflicts under any circumstances, whether files change during the sync process, or there are network drop-outs. This should be all managed by the local and server database files with appropriate checksums to cover potential network issues. My experience continues to be that Nextcloud Client is creating conflicts when it should not. |
|
I'm even seeing this issue when using the built in markdown editor in Nextcloud. I did Then I save the file in the nextcloud gui and run stat again: A few seconds later, the GUI will tell me the file was edited "outside of the editor". I then When I looked down at my local clock, it read 14:40 as well. I don't know the exact number of milliseconds but this does not seem to be working as intended. These files are indeed on an "External SSH" share. |
|
I find the most conflicts appear when editing either git or a small
database file from a path being synced by the desktop app. Agreed that
there are fewer conflicts, but I'm still struggling in being unable to
merge changes.
Example
Conflict between desktop and server. The file on device will be the most
recent since I just edited it. "Choose a copy to keep" as File on server
is now out of sync, but if not resolved...
Conflict now affects editing the file on other devices. They must also
choose a copy to keep, local or server.
To resolve:
- I write down in a note what copy of the file is on what device
- backup each copy outside of nextcloud with a new naming scheme or tar
archive.
- manually merge changes
- delete as many copies as needed, possibly on both the devices and server
- upload and sync the proper copy
This is particularly difficult because these conflicted files are accessed
from computers in multiple geographic locations I do not have remote access
to, so I've never been able to merge and resolve all changes other than
eventually moving to a new name scheme that is not marked as a conflict in
order to move forward.
No doubt there is a better way for me to manage this.
…On Thu, Feb 25, 2021, 11:44 AM Ilya Kogan ***@***.***> wrote:
I'm even seeing this issue when using the built in markdown editor in
Nextcloud. I did stat on a file:
File: CC Transition Plan.md
Size: 6906 Blocks: 29 IO Block: 7168 regular file
Device: 4ah/74d Inode: 329327 Links: 1
Access: (0700/-rwx------) Uid: (1234/ kogan) Gid: (1234/ kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:39:43.303202466 -0500
Change: 2021-02-25 14:39:43.303202466 -0500
Birth:
Then I save the file in the nextcloud gui and run stat again:
File: CC Transition Plan.md
Size: 6905 Blocks: 29 IO Block: 7168 regular file
Device: 4ah/74d Inode: 329327 Links: 1
Access: (0700/-rwx------) Uid: (1234/ kogan) Gid: (1234/ kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:40:00.383787486 -0500
Change: 2021-02-25 14:40:00.383787486 -0500
Birth: -
A few seconds later, the GUI will tell me the file was edited "outside of
the editor". I then stat the file again:
File: CC Transition Plan.md
Size: 6905 Blocks: 29 IO Block: 7168 regular file
Device: 4ah/74d Inode: 329327 Links: 1
Access: (0700/-rwx------) Uid: (1234/ kogan) Gid: (1234/ kogan)
Access: 2021-02-25 13:53:08.275518704 -0500
Modify: 2021-02-25 14:40:00.383787486 -0500
Change: 2021-02-25 14:40:00.383787486 -0500
Birth: -
When I looked down at my local clock, it read 14:40 as well. I don't know
the *exact* number of milliseconds but this does not seem to be working
as intended.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#2467 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANUKZQLJ6MNUEXRBNOH3CTTA2SAJANCNFSM4RXOZKIQ>
.
|
|
I always had problems with Autoupload. It just doesn't work reliably. I really don't understand why is this happening. The only thing that should work, the basis and main focus of Nextcloud should be a secure and reliable sync, in my opinion. It isn't apparently. I don't care about all the fancy apps and bells and whistles. Or maybe, but first and foremost sync MUST work. Always. If I disable Autoupload and enable it again. It actually re-uploads the whole library. The files don't retain the original creation date and files_versions saves a copy of each file which is byte for byte the same as the re-uploaded one. I have the feeling the solution might be a md5sum (or blake2 or whatever) away because it is almost working but just not yet. Sorry for this heated feedback. I honestly greatly appreciate this project and someday it will be great but today a 644MB video didn't wanted to upload and I did again the disable/enable trick and while I was watching reupload my 6000+ photos and videos again I had enough. I pulled the plug of my Nextcloud instance and went back to seafile, which doesn't have all the fancy features but it does the sync right (creation date is also not preserved, though). |
|
This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you! |
|
With respect, I still think the synchronization algorithm needs further refinement. The mode of operation is one client which is actively and regularly updating files, and one server with no activity on the directories being synchronised. Other clients are being updated, but this is stricly read only. Thus NC is effectively working as a live backup from the client to the server with a distributed backup to other clients. Any file on the client may change or disappear during synchronization. In this scenario there should never be any conflicts. Conflicts that do arise are therefore because of a problem with the synchronization algorithm or implementation. As with any distributed system, other independent network or disk activity may mean that response times are sometimes longer than usual (with possible timeouts), but the synchronization algorithm should handle this gracefully with retries, or at worst some "unable to sync error". This scenario should never cause conflicts, yet NC continues to do so. Below are some recent examples. Notice the dates and frequency; more than one conflict per week. -rwxr-xr-x 1 bobw warren 50151 2021-03-26 11:02 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/prefs (conflicted copy 2021-03-26 110259).js -rwxr-xr-x 1 bobw warren 53 2021-03-26 11:03 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/sessionCheckpoints (conflicted copy 2021-03-26 110301).json -rwxr-xr-x 1 bobw warren 524288 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/cookies (conflicted copy 2021-03-26 110923).sqlite -rwxr-xr-x 1 bobw warren 8826 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/Mail/pop3.blueyonder.co.uk/Inbox (conflicted copy 2021-03-26 110923).msf -rwxr-xr-x 1 bobw warren 1173870 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/panacea (conflicted copy 2021-03-26 110923).dat -rwxr-xr-x 1 bobw warren 5242880 2021-03-26 11:09 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Thunderbird/Profiles/rdvpq1tu.default/places (conflicted copy 2021-03-26 110923).sqlite -rwxr-xr-x 1 bobw warren 1048576 2021-04-04 01:13 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/cookies (conflicted copy 2021-04-04 011354).sqlite -rwxr-xr-x 1 bobw warren 11206656 2021-04-06 00:19 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore (conflicted copy 2021-04-06 001936).sqlite -rwxr-xr-x 1 bobw warren 10485760 2021-04-08 20:40 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/places (conflicted copy 2021-04-08 204009).sqlite -rwxr-xr-x 1 bobw warren 11206656 2021-04-08 20:40 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/webappsstore (conflicted copy 2021-04-08 204010).sqlite -rwxr-xr-x 1 bobw warren 1048576 2021-04-10 01:23 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/cookies (conflicted copy 2021-04-10 012324).sqlite |
|
This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you! |
|
I upgraded to client 3.2.1. Can I provide more information or other assistance in order for this bug to at least get some attention? |
|
Just to pile on with everyone else. I have a nextcloud server running in a jail on a truenas ( TrueNAS-12.0-U8) server. My machines are Windows 10. I use nextcloud to sync between my desktop and notebook. I never have the same files open on both machines at the same time. The client and server are up to date. I use Excel files, alot. When I have one open I start getting the "Conflict" message many many times. This is new with the last major server update. I do not know if it is a coincidence or happenstance. But it is real frustrating. Thanks. |
|
I have been following up on the comments that @wonx has contributed, particularly that he has one configuration for which he has seen no false conflicts to date. It seems that there are several configuration settings that make a difference, and what seems to have the most effect is the opcache settings suggested by @vkahl above: Indeed when I replicate these settings on my test system (where data is stored on local memory of the server), then my rate of false conflicts drops singificantly. After 15 hours I got 8 false conflicts with a sleep 5 in my script If the sleep is less than 5, then the server does not have time to "catch up" and the rate of false conflicts drops. For systems where the access times to the server storage is slow, you may have to increase this sleep time. To quantify the rate of false conflicts, I use two measures:
Using the above configuration I get 240ppm MTBF=2hours. However, this configuration does not make sense to me. in other words, if a recommended configuration is not self-consistent, how can anyone rely upon it? So I adjusted my configuration as follows: I ran for 138 hours and got 16 false conflicts, that is 13ppm MTBF=8.6 hours. |
|
In an attempt to identify which of the opcache configuration settings has the most effect, I ran a control test: To my complete surprise, I have not, to date, seen any false conflicts. This test is still running. Even if I find no false conflicts after a week or so, that does not mean the problem has been solved (because, as demonstrated, changes in configuration affect the rate at which false conflicts occur, they do not address the fundamental problem). But this does suggest a viable temporary workaround for those that are suffering from false conflicts. Please could anyone who finds a false conflict with opcache disabled report here. Conversely, if this proves effective for you by dramatically reducing false conflicts, please upvote this comment (see the smiley on the strapline for this comment). |
|
Does disabling opcache have any noticeable effect on performance?
So the opcache affects performance where a significant amount of time is spent processing PHP scripts and where that same PHP script is used repeatedly. Experience on my production system suggests that page render times are dominated by the volume of data being fetched. Even on pages that I imagine might involve significant PHP processing (such as using the picoCMS App), I cannot discern any difference in page render times with opcache either enabled or disabled. The very first time on a given machine, after a reboot, or similar, page render times can be longer, and I put this down to fetching data that can be held in the local browser cache. The following are a sample of results that I repeated several times and after switching opcache on and off several times. Times are very approximate because of human deviations in recording button press time and observing page fully rendered. EDIT: My mistake, I had changed Subjectively, it feels like performance is slightly faster when opcache is disabled. That would make sense because no time is wasted in either searching for entries in the opcache nor in filling the opcache with newly compiled scripts/objects. Can anyone justify enabling opcache? |
|
opcache is a recommended part of setting up nextcloud. See the documentation here. |
|
@sunjam I'm gald you've joined the discussion.
Is that really true? Is opcache actually recommended? On the page you referenced, there is a link to the page to which I referred which goes into more detail about opcache for scripts/objects and how data is handled independently. It states:
Now it is not that I want to disagree with you, but rather I really want to understand why something might be recommended (and in this case, the documentation is, let's say "unclear", if not contradictory and confusing).
So from my perspective, I would say that the recommendation should be that opcache is disabled. Please could someone provide reasoned arguments as to why opcache might be recommended or not. |
|
@Bockeman In my case, I don't see any difference:
So, just like before. |
|
I may have to eat an awful lot of words! How do you ensure php.ini is honoured? How do you check if php has those settings? On my production system, I tried but, as you see, opcache still appears to be enabled. What am I doing wrong? ) |
|
Ok, in my installation, setting I checked that it was indeed enabled by creating a file in nextcloud's folder (e.g. info.php) with the following inside: and then browsing https://yourserver.com/info.php, and checking the Zend OPcache section you will see something like opcache.enable | On. But I guess it's equivalent as checking the php info from the command line as you did. |
|
@wonx thanks, the file I should have changed was /etc/php.d/10-opcache.ini |
|
I ran the test script, with After running for 36.7 hours 15 false conflicts were generated, 194ppm MTBF=2.4 hours This means that There is a bug in the client code (most likely in the exception handling) which erroneously causes conflicts to be generated. Please @FlexW @er-vin or @claell @mgallien @allexzander @camilasan can you help get some developer attention on this issue? |
|
I can confirm this issue is still present in Nextcloud 24.0.2, with the desktop client 3.5.1. In my case, I can only replicate it in sftp external folders, but not in SMB. |
|
My issue has gone away now that I've moved my data from external SMB storage to local (NVMe) disk. |
|
In my case, with client 3.6.0, and a SFTP external folder, the issue is still present. |
|
I have been running the same tests with Nextcloud 25.0.1, and the issue with sftp external folders seems to have gone away. |
|
Sounds almost too good to be true ;-). It will be a while until we can upgrade our production system to v25, though. |
|
Nah, I spoke too soon. Today I tried with the script that was posted a few coments above, and I started getting conflicts almost immediately. It has the same problems as before. |
|
#5188 and #5182 should help a lot with fake conflicts |
|
@mgallien thanks for all your work on nextcloud, it is much appreciated. Executive summaryYour 5188 PR does not solve the false conflicts problem. Executive questionIf 5188 PR addresses "update local file mtime on changes from server", but not "update server mtime on changes from client", then surely this will cause false conflicts? DetailsI upgraded all I could before using 5188 PR and confirmed that I still get false conflicts: I installed and ran your 5188 PR AppImage without the fuse random delays to check against regression. Surprisingly, I got a false conflict I repeated with fuse random delays and got many false conflicts. EnvironmentThe randomisation and delays within the fuse storage, plus the delays within the script loop are extremely sensitive. I am trying to catch random events and tuning the parameters to yeild the most false conflicts is tricky. The fuse program and loop script is not as posted above, but I can supply if requested. I suspect such scripts need to be tuned to the specific machine and environment to maximise the occurrence of false conflicts. ForumGiven no developer attention on this issue for two years, until now, might I ask, which would be the best forum and/or topic title to raise this issue to pique the interest of the relevant experts? |
|
@Bockeman Thank you for your work and analysis. Regarding your last question I'd suggest that someone with a payed support contract from Nextcloud should raise this issue with the official Nextcloud support. |
You should not worry, we are actively working on it. Most cases can be reproduced so we are busy with the bug fixes |
|
Hello @mgallien! Is there any update? I'm having the same issue, so I cannot use Nextcloud until this is fixed :( |
|
Same here with 24.0.12 Enterprise Version. Any Progress on this Issue? |
|
I can get a conflict every time with these repro steps:
I'm using Nextcloud server v27.0.2 (latest stable, run via Docker on an Ubuntu server), Desktop client v3.9.3 (latest stable, installed from nextcloud-devs PPA). Desktop client is running on a 64-bit Ubuntu 22.04 LTS desktop. |
|
Found an issue that might be related: |
|
Good News on this Issue: Enterprise Support fixed this Issue: nextcloud/server#40487 At least for us everything is fine now. Thanks! |
|
I would prefer to close this one and have people open a new one for current findings |
and others
How to use GitHub
Expected behaviour
Conflicts should only be created when more than one client uploads to the server the same file with different contents.
Actual behaviour
I have several Nextcoud-client 3.0.1 all running on Win10 machines.
One of the sync folders is /AppData/Roaming/Mozilla; this is an "active" folder tree with several files changing continuously. But I have only one Mozilla Firefox browser open.
I am getting several conflicts generated every hour or so.
Steps to reproduce
Client configuration
Client version: 3.0.1
Operating system: Win10
OS language: English
Server configuration
Nextcloud version: 19.0.3
Storage backend (external storage): No external storage. Files on server are NFS mounted (actually gluster).
Logs
The following are generated from scripts running on a linux server with the Win10 machine mounted under
/mnt/veriulan/d/Data/BobW
and show 4 separate occassions (with dates shown) when conflicts were generated:
Compare sizes and dates
-rwxr-xr-x 1 bobw warren 254622 2020-09-23 16:51 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 165141).jsonlz4
-rwxr-xr-x 1 bobw warren 254616 2020-09-23 16:51 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4
2020-09-23 17:01:43
2020-09-23 17:01:55
Compare sizes and dates
-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:20 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 172056).sqlite
-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:29 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite
2020-09-23 17:29:59
2020-09-23 17:30:04
Compare sizes and dates
-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:36 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 173605).sqlite
-rwxr-xr-x 1 bobw warren 114688 2020-09-23 17:56 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite
-rwxr-xr-x 1 bobw warren 2514944 2020-09-23 17:58 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 175858).sqlite
-rwxr-xr-x 1 bobw warren 114688 2020-09-23 17:56 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite
2020-09-23 19:39:10
2020-09-23 19:39:16
Compare sizes and dates
-rwxr-xr-x 1 bobw warren 261050 2020-09-23 20:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201018).baklz4
-rwxr-xr-x 1 bobw warren 264483 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.baklz4
-rwxr-xr-x 1 bobw warren 261112 2020-09-23 20:10 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201033).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4
-rwxr-xr-x 1 bobw warren 262513 2020-09-23 20:18 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 201853).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4
-rwxr-xr-x 1 bobw warren 262061 2020-09-23 20:22 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 202248).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4
-rwxr-xr-x 1 bobw warren 264463 2020-09-23 20:38 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 203812).baklz4
-rwxr-xr-x 1 bobw warren 264483 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.baklz4
-rwxr-xr-x 1 bobw warren 264409 2020-09-23 20:38 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery (conflicted copy 2020-09-23 203834).jsonlz4
-rwxr-xr-x 1 bobw warren 264488 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/sessionstore-backups/recovery.jsonlz4
-rwxr-xr-x 1 bobw warren 2523136 2020-09-23 20:08 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw (conflicted copy 2020-09-23 200812).sqlite
-rwxr-xr-x 1 bobw warren 2531328 2020-09-23 20:45 /mnt/veriulan/d/Data/BobW/AppData/Roaming/Mozilla/Firefox/Profiles/6qw6a8eb.default/storage/default/https+++web.whatsapp.com/idb/3166453069wcaw.sqlite
2020-09-23 20:46:07
2020-09-23 20:46:13
The text was updated successfully, but these errors were encountered: