-
-
Notifications
You must be signed in to change notification settings - Fork 4k
/
bisync.md
1854 lines (1561 loc) · 101 KB
/
bisync.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
---
title: "Bisync"
description: "Bidirectional cloud sync solution in rclone"
versionIntroduced: "v1.58"
status: Beta
---
## Bisync
`bisync` is **in beta** and is considered an **advanced command**, so use with care.
Make sure you have read and understood the entire [manual](https://rclone.org/bisync) (especially the [Limitations](#limitations) section) before using, or data loss can result. Questions can be asked in the [Rclone Forum](https://forum.rclone.org/).
## Getting started {#getting-started}
- [Install rclone](/install/) and setup your remotes.
- Bisync will create its working directory
at `~/.cache/rclone/bisync` on Linux, `/Users/yourusername/Library/Caches/rclone/bisync` on Mac,
or `C:\Users\MyLogin\AppData\Local\rclone\bisync` on Windows.
Make sure that this location is writable.
- Run bisync with the `--resync` flag, specifying the paths
to the local and remote sync directory roots.
- For successive sync runs, leave off the `--resync` flag. (**Important!**)
- Consider using a [filters file](#filtering) for excluding
unnecessary files and directories from the sync.
- Consider setting up the [--check-access](#check-access) feature
for safety.
- On Linux or Mac, consider setting up a [crontab entry](#cron). bisync can
safely run in concurrent cron jobs thanks to lock files it maintains.
For example, your first command might look like this:
```
rclone bisync remote1:path1 remote2:path2 --create-empty-src-dirs --compare size,modtime,checksum --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
```
If all looks good, run it again without `--dry-run`. After that, remove `--resync` as well.
Here is a typical run log (with timestamps removed for clarity):
```
rclone bisync /testdir/path1/ /testdir/path2/ --verbose
INFO : Synching Path1 "/testdir/path1/" with Path2 "/testdir/path2/"
INFO : Path1 checking for diffs
INFO : - Path1 File is new - file11.txt
INFO : - Path1 File is newer - file2.txt
INFO : - Path1 File is newer - file5.txt
INFO : - Path1 File is newer - file7.txt
INFO : - Path1 File was deleted - file4.txt
INFO : - Path1 File was deleted - file6.txt
INFO : - Path1 File was deleted - file8.txt
INFO : Path1: 7 changes: 1 new, 3 newer, 0 older, 3 deleted
INFO : Path2 checking for diffs
INFO : - Path2 File is new - file10.txt
INFO : - Path2 File is newer - file1.txt
INFO : - Path2 File is newer - file5.txt
INFO : - Path2 File is newer - file6.txt
INFO : - Path2 File was deleted - file3.txt
INFO : - Path2 File was deleted - file7.txt
INFO : - Path2 File was deleted - file8.txt
INFO : Path2: 7 changes: 1 new, 3 newer, 0 older, 3 deleted
INFO : Applying changes
INFO : - Path1 Queue copy to Path2 - /testdir/path2/file11.txt
INFO : - Path1 Queue copy to Path2 - /testdir/path2/file2.txt
INFO : - Path2 Queue delete - /testdir/path2/file4.txt
NOTICE: - WARNING New or changed in both paths - file5.txt
NOTICE: - Path1 Renaming Path1 copy - /testdir/path1/file5.txt..path1
NOTICE: - Path1 Queue copy to Path2 - /testdir/path2/file5.txt..path1
NOTICE: - Path2 Renaming Path2 copy - /testdir/path2/file5.txt..path2
NOTICE: - Path2 Queue copy to Path1 - /testdir/path1/file5.txt..path2
INFO : - Path2 Queue copy to Path1 - /testdir/path1/file6.txt
INFO : - Path1 Queue copy to Path2 - /testdir/path2/file7.txt
INFO : - Path2 Queue copy to Path1 - /testdir/path1/file1.txt
INFO : - Path2 Queue copy to Path1 - /testdir/path1/file10.txt
INFO : - Path1 Queue delete - /testdir/path1/file3.txt
INFO : - Path2 Do queued copies to - Path1
INFO : - Path1 Do queued copies to - Path2
INFO : - Do queued deletes on - Path1
INFO : - Do queued deletes on - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "/testdir/path1/" vs Path2 "/testdir/path2/"
INFO : Bisync successful
```
## Command line syntax
```
$ rclone bisync --help
Usage:
rclone bisync remote1:path1 remote2:path2 [flags]
Positional arguments:
Path1, Path2 Local path, or remote storage with ':' plus optional path.
Type 'rclone listremotes' for list of configured remotes.
Optional Flags:
--backup-dir1 string --backup-dir for Path1. Must be a non-overlapping path on the same remote.
--backup-dir2 string --backup-dir for Path2. Must be a non-overlapping path on the same remote.
--check-access Ensure expected RCLONE_TEST files are found on both Path1 and Path2 filesystems, else abort.
--check-filename string Filename for --check-access (default: RCLONE_TEST)
--check-sync string Controls comparison of final listings: true|false|only (default: true) (default "true")
--compare string Comma-separated list of bisync-specific compare options ex. 'size,modtime,checksum' (default: 'size,modtime')
--conflict-loser ConflictLoserAction Action to take on the loser of a sync conflict (when there is a winner) or on both files (when there is no winner): , num, pathname, delete (default: num)
--conflict-resolve string Automatically resolve conflicts by preferring the version that is: none, path1, path2, newer, older, larger, smaller (default: none) (default "none")
--conflict-suffix string Suffix to use when renaming a --conflict-loser. Can be either one string or two comma-separated strings to assign different suffixes to Path1/Path2. (default: 'conflict')
--create-empty-src-dirs Sync creation and deletion of empty directories. (Not compatible with --remove-empty-dirs)
--download-hash Compute hash by downloading when otherwise unavailable. (warning: may be slow and use lots of data!)
--filters-file string Read filtering patterns from a file
--force Bypass --max-delete safety check and run the sync. Consider using with --verbose
-h, --help help for bisync
--ignore-listing-checksum Do not use checksums for listings (add --ignore-checksum to additionally skip post-copy checksum checks)
--max-lock Duration Consider lock files older than this to be expired (default: 0 (never expire)) (minimum: 2m) (default 0s)
--no-cleanup Retain working files (useful for troubleshooting and testing).
--no-slow-hash Ignore listing checksums only on backends where they are slow
--recover Automatically recover from interruptions without requiring --resync.
--remove-empty-dirs Remove ALL empty directories at the final cleanup step.
--resilient Allow future runs to retry after certain less-serious errors, instead of requiring --resync. Use at your own risk!
-1, --resync Performs the resync run. Equivalent to --resync-mode path1. Consider using --verbose or --dry-run first.
--resync-mode string During resync, prefer the version that is: path1, path2, newer, older, larger, smaller (default: path1 if --resync, otherwise none for no resync.) (default "none")
--retries int Retry operations this many times if they fail (requires --resilient). (default 3)
--retries-sleep Duration Interval between retrying operations if they fail, e.g. 500ms, 60s, 5m (0 to disable) (default 0s)
--slow-hash-sync-only Ignore slow checksums for listings and deltas, but still consider them during sync calls.
--workdir string Use custom working dir - useful for testing. (default: {WORKDIR})
--max-delete PERCENT Safety check on maximum percentage of deleted files allowed. If exceeded, the bisync run will abort. (default: 50%)
-n, --dry-run Go through the motions - No files are copied/deleted.
-v, --verbose Increases logging verbosity. May be specified more than once for more details.
```
Arbitrary rclone flags may be specified on the
[bisync command line](/commands/rclone_bisync/), for example
`rclone bisync ./testdir/path1/ gdrive:testdir/path2/ --drive-skip-gdocs -v -v --timeout 10s`
Note that interactions of various rclone flags with bisync process flow
has not been fully tested yet.
### Paths
Path1 and Path2 arguments may be references to any mix of local directory
paths (absolute or relative), UNC paths (`//server/share/path`),
Windows drive paths (with a drive letter and `:`) or configured
[remotes](/docs/#syntax-of-remote-paths) with optional subdirectory paths.
Cloud references are distinguished by having a `:` in the argument
(see [Windows support](#windows) below).
Path1 and Path2 are treated equally, in that neither has priority for
file changes (except during [`--resync`](#resync)), and access efficiency does not change whether a remote
is on Path1 or Path2.
The listings in bisync working directory (default: `~/.cache/rclone/bisync`)
are named based on the Path1 and Path2 arguments so that separate syncs
to individual directories within the tree may be set up, e.g.:
`path_to_local_tree..dropbox_subdir.lst`.
Any empty directories after the sync on both the Path1 and Path2
filesystems are not deleted by default, unless `--create-empty-src-dirs` is specified.
If the `--remove-empty-dirs` flag is specified, then both paths will have ALL empty directories purged
as the last step in the process.
## Command-line flags
### --resync
This will effectively make both Path1 and Path2 filesystems contain a
matching superset of all files. By default, Path2 files that do not exist in Path1 will
be copied to Path1, and the process will then copy the Path1 tree to Path2.
The `--resync` sequence is roughly equivalent to the following (but see [`--resync-mode`](#resync-mode) for other options):
```
rclone copy Path2 Path1 --ignore-existing [--create-empty-src-dirs]
rclone copy Path1 Path2 [--create-empty-src-dirs]
```
The base directories on both Path1 and Path2 filesystems must exist
or bisync will fail. This is required for safety - that bisync can verify
that both paths are valid.
When using `--resync`, a newer version of a file on the Path2 filesystem
will (by default) be overwritten by the Path1 filesystem version.
(Note that this is [NOT entirely symmetrical](https://github.com/rclone/rclone/issues/5681#issuecomment-938761815), and more symmetrical options can be specified with the [`--resync-mode`](#resync-mode) flag.)
Carefully evaluate deltas using [--dry-run](/flags/#non-backend-flags).
For a resync run, one of the paths may be empty (no files in the path tree).
The resync run should result in files on both paths, else a normal non-resync
run will fail.
For a non-resync run, either path being empty (no files in the tree) fails with
`Empty current PathN listing. Cannot sync to an empty directory: X.pathN.lst`
This is a safety check that an unexpected empty path does not result in
deleting **everything** in the other path.
Note that `--resync` implies `--resync-mode path1` unless a different
[`--resync-mode`](#resync-mode) is explicitly specified.
It is not necessary to use both the `--resync` and `--resync-mode` flags --
either one is sufficient without the other.
**Note:** `--resync` (including `--resync-mode`) should only be used under three specific (rare) circumstances:
1. It is your _first_ bisync run (between these two paths)
2. You've just made changes to your bisync settings (such as editing the contents of your `--filters-file`)
3. There was an error on the prior run, and as a result, bisync now requires `--resync` to recover
The rest of the time, you should _omit_ `--resync`. The reason is because `--resync` will only _copy_ (not _sync_) each side to the other.
Therefore, if you included `--resync` for every bisync run, it would never be possible to delete a file --
the deleted file would always keep reappearing at the end of every run (because it's being copied from the other side where it still exists).
Similarly, renaming a file would always result in a duplicate copy (both old and new name) on both sides.
If you find that frequent interruptions from #3 are an issue, rather than
automatically running `--resync`, the recommended alternative is to use the
[`--resilient`](#resilient), [`--recover`](#recover), and
[`--conflict-resolve`](#conflict-resolve) flags, (along with [Graceful
Shutdown](#graceful-shutdown) mode, when needed) for a very robust
"set-it-and-forget-it" bisync setup that can automatically bounce back from
almost any interruption it might encounter. Consider adding something like the
following:
```
--resilient --recover --max-lock 2m --conflict-resolve newer
```
### --resync-mode CHOICE {#resync-mode}
In the event that a file differs on both sides during a `--resync`,
`--resync-mode` controls which version will overwrite the other. The supported
options are similar to [`--conflict-resolve`](#conflict-resolve). For all of
the following options, the version that is kept is referred to as the "winner",
and the version that is overwritten (deleted) is referred to as the "loser".
The options are named after the "winner":
- `path1` - (the default) - the version from Path1 is unconditionally
considered the winner (regardless of `modtime` and `size`, if any). This can be
useful if one side is more trusted or up-to-date than the other, at the time of
the `--resync`.
- `path2` - same as `path1`, except the path2 version is considered the winner.
- `newer` - the newer file (by `modtime`) is considered the winner, regardless
of which side it came from. This may result in having a mix of some winners
from Path1, and some winners from Path2. (The implementation is analogous to
running `rclone copy --update` in both directions.)
- `older` - same as `newer`, except the older file is considered the winner,
and the newer file is considered the loser.
- `larger` - the larger file (by `size`) is considered the winner (regardless
of `modtime`, if any). This can be a useful option for remotes without
`modtime` support, or with the kinds of files (such as logs) that tend to grow
but not shrink, over time.
- `smaller` - the smaller file (by `size`) is considered the winner (regardless
of `modtime`, if any).
For all of the above options, note the following:
- If either of the underlying remotes lacks support for the chosen method, it
will be ignored and will fall back to the default of `path1`. (For example, if
`--resync-mode newer` is set, but one of the paths uses a remote that doesn't
support `modtime`.)
- If a winner can't be determined because the chosen method's attribute is
missing or equal, it will be ignored, and bisync will instead try to determine
whether the files differ by looking at the other `--compare` methods in effect.
(For example, if `--resync-mode newer` is set, but the Path1 and Path2 modtimes
are identical, bisync will compare the sizes.) If bisync concludes that they
differ, preference is given to whichever is the "source" at that moment. (In
practice, this gives a slight advantage to Path2, as the 2to1 copy comes before
the 1to2 copy.) If the files _do not_ differ, nothing is copied (as both sides
are already correct).
- These options apply only to files that exist on both sides (with the same
name and relative path). Files that exist *only* on one side and not the other
are *always* copied to the other, during `--resync` (this is one of the main
differences between resync and non-resync runs.).
- `--conflict-resolve`, `--conflict-loser`, and `--conflict-suffix` do not
apply during `--resync`, and unlike these flags, nothing is renamed during
`--resync`. When a file differs on both sides during `--resync`, one version
always overwrites the other (much like in `rclone copy`.) (Consider using
[`--backup-dir`](#backup-dir1-and-backup-dir2) to retain a backup of the losing
version.)
- Unlike for `--conflict-resolve`, `--resync-mode none` is not a valid option
(or rather, it will be interpreted as "no resync", unless `--resync` has also
been specified, in which case it will be ignored.)
- Winners and losers are decided at the individual file-level only (there is
not currently an option to pick an entire winning directory atomically,
although the `path1` and `path2` options typically produce a similar result.)
- To maintain backward-compatibility, the `--resync` flag implies
`--resync-mode path1` unless a different `--resync-mode` is explicitly
specified. Similarly, all `--resync-mode` options (except `none`) imply
`--resync`, so it is not necessary to use both the `--resync` and
`--resync-mode` flags simultaneously -- either one is sufficient without the
other.
### --check-access
Access check files are an additional safety measure against data loss.
bisync will ensure it can find matching `RCLONE_TEST` files in the same places
in the Path1 and Path2 filesystems.
`RCLONE_TEST` files are not generated automatically.
For `--check-access` to succeed, you must first either:
**A)** Place one or more `RCLONE_TEST` files in both systems, or
**B)** Set `--check-filename` to a filename already in use in various locations
throughout your sync'd fileset. Recommended methods for **A)** include:
* `rclone touch Path1/RCLONE_TEST` (create a new file)
* `rclone copyto Path1/RCLONE_TEST Path2/RCLONE_TEST` (copy an existing file)
* `rclone copy Path1/RCLONE_TEST Path2/RCLONE_TEST --include "RCLONE_TEST"` (copy multiple files at once, recursively)
* create the files manually (outside of rclone)
* run `bisync` once *without* `--check-access` to set matching files on both filesystems
will also work, but is not preferred, due to potential for user error
(you are temporarily disabling the safety feature).
Note that `--check-access` is still enforced on `--resync`, so `bisync --resync --check-access`
will not work as a method of initially setting the files (this is to ensure that bisync can't
[inadvertently circumvent its own safety switch](https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=3.%20%2D%2Dcheck%2Daccess%20doesn%27t%20always%20fail%20when%20it%20should).)
Time stamps and file contents for `RCLONE_TEST` files are not important, just the names and locations.
If you have symbolic links in your sync tree it is recommended to place
`RCLONE_TEST` files in the linked-to directory tree to protect against
bisync assuming a bunch of deleted files if the linked-to tree should not be
accessible.
See also the [--check-filename](--check-filename) flag.
### --check-filename
Name of the file(s) used in access health validation.
The default `--check-filename` is `RCLONE_TEST`.
One or more files having this filename must exist, synchronized between your
source and destination filesets, in order for `--check-access` to succeed.
See [--check-access](#check-access) for additional details.
### --compare
As of `v1.66`, bisync fully supports comparing based on any combination of
size, modtime, and checksum (lifting the prior restriction on backends without
modtime support.)
By default (without the `--compare` flag), bisync inherits the same comparison
options as `sync`
(that is: `size` and `modtime` by default, unless modified with flags such as
[`--checksum`](/docs/#c-checksum) or [`--size-only`](/docs/#size-only).)
If the `--compare` flag is set, it will override these defaults. This can be
useful if you wish to compare based on combinations not currently supported in
`sync`, such as comparing all three of `size` AND `modtime` AND `checksum`
simultaneously (or just `modtime` AND `checksum`).
`--compare` takes a comma-separated list, with the currently supported values
being `size`, `modtime`, and `checksum`. For example, if you want to compare
size and checksum, but not modtime, you would do:
```
--compare size,checksum
```
Or if you want to compare all three:
```
--compare size,modtime,checksum
```
`--compare` overrides any conflicting flags. For example, if you set the
conflicting flags `--compare checksum --size-only`, `--size-only` will be
ignored, and bisync will compare checksum and not size. To avoid confusion, it
is recommended to use _either_ `--compare` or the normal `sync` flags, but not
both.
If `--compare` includes `checksum` and both remotes support checksums but have
no hash types in common with each other, checksums will be considered _only_
for comparisons within the same side (to determine what has changed since the
prior sync), but not for comparisons against the opposite side. If one side
supports checksums and the other does not, checksums will only be considered on
the side that supports them.
When comparing with `checksum` and/or `size` without `modtime`, bisync cannot
determine whether a file is `newer` or `older` -- only whether it is `changed`
or `unchanged`. (If it is `changed` on both sides, bisync still does the
standard equality-check to avoid declaring a sync conflict unless it absolutely
has to.)
It is recommended to do a `--resync` when changing `--compare` settings, as
otherwise your prior listing files may not contain the attributes you wish to
compare (for example, they will not have stored checksums if you were not
previously comparing checksums.)
### --ignore-listing-checksum
When `--checksum` or `--compare checksum` is set, bisync will retrieve (or
generate) checksums (for backends that support them) when creating the listings
for both paths, and store the checksums in the listing files.
`--ignore-listing-checksum` will disable this behavior, which may speed things
up considerably, especially on backends (such as [local](/local/)) where hashes
must be computed on the fly instead of retrieved. Please note the following:
* As of `v1.66`, `--ignore-listing-checksum` is now automatically set when
neither `--checksum` nor `--compare checksum` are in use (as the checksums
would not be used for anything.)
* `--ignore-listing-checksum` is NOT the same as
[`--ignore-checksum`](/docs/#ignore-checksum),
and you may wish to use one or the other, or both. In a nutshell:
`--ignore-listing-checksum` controls whether checksums are considered when
scanning for diffs,
while `--ignore-checksum` controls whether checksums are considered during the
copy/sync operations that follow,
if there ARE diffs.
* Unless `--ignore-listing-checksum` is passed, bisync currently computes
hashes for one path
*even when there's no common hash with the other path*
(for example, a [crypt](/crypt/#modification-times-and-hashes) remote.)
This can still be beneficial, as the hashes will still be used to detect
changes within the same side
(if `--checksum` or `--compare checksum` is set), even if they can't be used to
compare against the opposite side.
* If you wish to ignore listing checksums _only_ on remotes where they are slow
to compute, consider using
[`--no-slow-hash`](#no-slow-hash) (or
[`--slow-hash-sync-only`](#slow-hash-sync-only)) instead of
`--ignore-listing-checksum`.
* If `--ignore-listing-checksum` is used simultaneously with `--compare
checksum` (or `--checksum`), checksums will be ignored for bisync deltas,
but still considered during the sync operations that follow (if deltas are
detected based on modtime and/or size.)
### --no-slow-hash
On some remotes (notably `local`), checksums can dramatically slow down a
bisync run, because hashes cannot be stored and need to be computed in
real-time when they are requested. On other remotes (such as `drive`), they add
practically no time at all. The `--no-slow-hash` flag will automatically skip
checksums on remotes where they are slow, while still comparing them on others
(assuming [`--compare`](#compare) includes `checksum`.) This can be useful when one of your
bisync paths is slow but you still want to check checksums on the other, for a more
robust sync.
### --slow-hash-sync-only
Same as [`--no-slow-hash`](#no-slow-hash), except slow hashes are still
considered during sync calls. They are still NOT considered for determining
deltas, nor or they included in listings. They are also skipped during
`--resync`. The main use case for this flag is when you have a large number of
files, but relatively few of them change from run to run -- so you don't want
to check your entire tree every time (it would take too long), but you still
want to consider checksums for the smaller group of files for which a `modtime`
or `size` change was detected. Keep in mind that this speed savings comes with
a safety trade-off: if a file's content were to change without a change to its
`modtime` or `size`, bisync would not detect it, and it would not be synced.
`--slow-hash-sync-only` is only useful if both remotes share a common hash
type (if they don't, bisync will automatically fall back to `--no-slow-hash`.)
Both `--no-slow-hash` and `--slow-hash-sync-only` have no effect without
`--compare checksum` (or `--checksum`).
### --download-hash
If `--download-hash` is set, bisync will use best efforts to obtain an MD5
checksum by downloading and computing on-the-fly, when checksums are not
otherwise available (for example, a remote that doesn't support them.) Note
that since rclone has to download the entire file, this may dramatically slow
down your bisync runs, and is also likely to use a lot of data, so it is
probably not practical for bisync paths with a large total file size. However,
it can be a good option for syncing small-but-important files with maximum
accuracy (for example, a source code repo on a `crypt` remote.) An additional
advantage over methods like [`cryptcheck`](/commands/rclone_cryptcheck/) is
that the original file is not required for comparison (for example,
`--download-hash` can be used to bisync two different crypt remotes with
different passwords.)
When `--download-hash` is set, bisync still looks for more efficient checksums
first, and falls back to downloading only when none are found. It takes
priority over conflicting flags such as `--no-slow-hash`. `--download-hash` is
not suitable for [Google Docs](#gdocs) and other files of unknown size, as
their checksums would change from run to run (due to small variances in the
internals of the generated export file.) Therefore, bisync automatically skips
`--download-hash` for files with a size less than 0.
See also: [`Hasher`](https://rclone.org/hasher/) backend,
[`cryptcheck`](/commands/rclone_cryptcheck/) command, [`rclone check
--download`](/commands/rclone_check/) option,
[`md5sum`](/commands/rclone_md5sum/) command
### --max-delete
As a safety check, if greater than the `--max-delete` percent of files were
deleted on either the Path1 or Path2 filesystem, then bisync will abort with
a warning message, without making any changes.
The default `--max-delete` is `50%`.
One way to trigger this limit is to rename a directory that contains more
than half of your files. This will appear to bisync as a bunch of deleted
files and a bunch of new files.
This safety check is intended to block bisync from deleting all of the
files on both filesystems due to a temporary network access issue, or if
the user had inadvertently deleted the files on one side or the other.
To force the sync, either set a different delete percentage limit,
e.g. `--max-delete 75` (allows up to 75% deletion), or use `--force`
to bypass the check.
Also see the [all files changed](#all-files-changed) check.
### --filters-file {#filters-file}
By using rclone filter features you can exclude file types or directory
sub-trees from the sync.
See the [bisync filters](#filtering) section and generic
[--filter-from](/filtering/#filter-from-read-filtering-patterns-from-a-file)
documentation.
An [example filters file](#example-filters-file) contains filters for
non-allowed files for synching with Dropbox.
If you make changes to your filters file then bisync requires a run
with `--resync`. This is a safety feature, which prevents existing files
on the Path1 and/or Path2 side from seeming to disappear from view
(since they are excluded in the new listings), which would fool bisync
into seeing them as deleted (as compared to the prior run listings),
and then bisync would proceed to delete them for real.
To block this from happening, bisync calculates an MD5 hash of the filters file
and stores the hash in a `.md5` file in the same place as your filters file.
On the next run with `--filters-file` set, bisync re-calculates the MD5 hash
of the current filters file and compares it to the hash stored in the `.md5` file.
If they don't match, the run aborts with a critical error and thus forces you
to do a `--resync`, likely avoiding a disaster.
### --conflict-resolve CHOICE {#conflict-resolve}
In bisync, a "conflict" is a file that is *new* or *changed* on *both sides*
(relative to the prior run) AND is *not currently identical* on both sides.
`--conflict-resolve` controls how bisync handles such a scenario. The currently
supported options are:
- `none` - (the default) - do not attempt to pick a winner, keep and rename
both files according to [`--conflict-loser`](#conflict-loser) and
[`--conflict-suffix`](#conflict-suffix) settings. For example, with the default
settings, `file.txt` on Path1 is renamed `file.txt.conflict1` and `file.txt` on
Path2 is renamed `file.txt.conflict2`. Both are copied to the opposite path
during the run, so both sides end up with a copy of both files. (As `none` is
the default, it is not necessary to specify `--conflict-resolve none` -- you
can just omit the flag.)
- `newer` - the newer file (by `modtime`) is considered the winner and is
copied without renaming. The older file (the "loser") is handled according to
`--conflict-loser` and `--conflict-suffix` settings (either renamed or
deleted.) For example, if `file.txt` on Path1 is newer than `file.txt` on
Path2, the result on both sides (with other default settings) will be `file.txt`
(winner from Path1) and `file.txt.conflict1` (loser from Path2).
- `older` - same as `newer`, except the older file is considered the winner,
and the newer file is considered the loser.
- `larger` - the larger file (by `size`) is considered the winner (regardless
of `modtime`, if any).
- `smaller` - the smaller file (by `size`) is considered the winner (regardless
of `modtime`, if any).
- `path1` - the version from Path1 is unconditionally considered the winner
(regardless of `modtime` and `size`, if any). This can be useful if one side is
usually more trusted or up-to-date than the other.
- `path2` - same as `path1`, except the path2 version is considered the
winner.
For all of the above options, note the following:
- If either of the underlying remotes lacks support for the chosen method, it
will be ignored and fall back to `none`. (For example, if `--conflict-resolve
newer` is set, but one of the paths uses a remote that doesn't support
`modtime`.)
- If a winner can't be determined because the chosen method's attribute is
missing or equal, it will be ignored and fall back to `none`. (For example, if
`--conflict-resolve newer` is set, but the Path1 and Path2 modtimes are
identical, even if the sizes may differ.)
- If the file's content is currently identical on both sides, it is not
considered a "conflict", even if new or changed on both sides since the prior
sync. (For example, if you made a change on one side and then synced it to the
other side by other means.) Therefore, none of the conflict resolution flags
apply in this scenario.
- The conflict resolution flags do not apply during a `--resync`, as there is
no "prior run" to speak of (but see [`--resync-mode`](#resync-mode) for similar
options.)
### --conflict-loser CHOICE {#conflict-loser}
`--conflict-loser` determines what happens to the "loser" of a sync conflict
(when [`--conflict-resolve`](#conflict-resolve) determines a winner) or to both
files (when there is no winner.) The currently supported options are:
- `num` - (the default) - auto-number the conflicts by automatically appending
the next available number to the `--conflict-suffix`, in chronological order.
For example, with the default settings, the first conflict for `file.txt` will
be renamed `file.txt.conflict1`. If `file.txt.conflict1` already exists,
`file.txt.conflict2` will be used instead (etc., up to a maximum of
9223372036854775807 conflicts.)
- `pathname` - rename the conflicts according to which side they came from,
which was the default behavior prior to `v1.66`. For example, with
`--conflict-suffix path`, `file.txt` from Path1 will be renamed
`file.txt.path1`, and `file.txt` from Path2 will be renamed `file.txt.path2`.
If two non-identical suffixes are provided (ex. `--conflict-suffix
cloud,local`), the trailing digit is omitted. Importantly, note that with
`pathname`, there is no auto-numbering beyond `2`, so if `file.txt.path2`
somehow already exists, it will be overwritten. Using a dynamic date variable
in your `--conflict-suffix` (see below) is one possible way to avoid this. Note
also that conflicts-of-conflicts are possible, if the original conflict is not
manually resolved -- for example, if for some reason you edited
`file.txt.path1` on both sides, and those edits were different, the result
would be `file.txt.path1.path1` and `file.txt.path1.path2` (in addition to
`file.txt.path2`.)
- `delete` - keep the winner only and delete the loser, instead of renaming it.
If a winner cannot be determined (see `--conflict-resolve` for details on how
this could happen), `delete` is ignored and the default `num` is used instead
(i.e. both versions are kept and renamed, and neither is deleted.) `delete` is
inherently the most destructive option, so use it only with care.
For all of the above options, note that if a winner cannot be determined (see
`--conflict-resolve` for details on how this could happen), or if
`--conflict-resolve` is not in use, *both* files will be renamed.
### --conflict-suffix STRING[,STRING] {#conflict-suffix}
`--conflict-suffix` controls the suffix that is appended when bisync renames a
[`--conflict-loser`](#conflict-loser) (default: `conflict`).
`--conflict-suffix` will accept either one string or two comma-separated
strings to assign different suffixes to Path1 vs. Path2. This may be helpful
later in identifying the source of the conflict. (For example,
`--conflict-suffix dropboxconflict,laptopconflict`)
With `--conflict-loser num`, a number is always appended to the suffix. With
`--conflict-loser pathname`, a number is appended only when one suffix is
specified (or when two identical suffixes are specified.) i.e. with
`--conflict-loser pathname`, all of the following would produce exactly the
same result:
```
--conflict-suffix path
--conflict-suffix path,path
--conflict-suffix path1,path2
```
Suffixes may be as short as 1 character. By default, the suffix is appended
after any other extensions (ex. `file.jpg.conflict1`), however, this can be
changed with the [`--suffix-keep-extension`](/docs/#suffix-keep-extension) flag
(i.e. to instead result in `file.conflict1.jpg`).
`--conflict-suffix` supports several *dynamic date variables* when enclosed in
curly braces as globs. This can be helpful to track the date and/or time that
each conflict was handled by bisync. For example:
```
--conflict-suffix {DateOnly}-conflict
// result: myfile.txt.2006-01-02-conflict1
```
All of the formats described [here](https://pkg.go.dev/time#pkg-constants) and
[here](https://pkg.go.dev/time#example-Time.Format) are supported, but take
care to ensure that your chosen format does not use any characters that are
illegal on your remotes (for example, macOS does not allow colons in
filenames, and slashes are also best avoided as they are often interpreted as
directory separators.) To address this particular issue, an additional
`{MacFriendlyTime}` (or just `{mac}`) option is supported, which results in
`2006-01-02 0304PM`.
Note that `--conflict-suffix` is entirely separate from rclone's main
[`--sufix`](/docs/#suffix-suffix) flag. This is intentional, as users may wish
to use both flags simultaneously, if also using
[`--backup-dir`](#backup-dir1-and-backup-dir2).
Finally, note that the default in bisync prior to `v1.66` was to rename
conflicts with `..path1` and `..path2` (with two periods, and `path` instead of
`conflict`.) Bisync now defaults to a single dot instead of a double dot, but
additional dots can be added by including them in the specified suffix string.
For example, for behavior equivalent to the previous default, use:
```
[--conflict-resolve none] --conflict-loser pathname --conflict-suffix .path
```
### --check-sync
Enabled by default, the check-sync function checks that all of the same
files exist in both the Path1 and Path2 history listings. This _check-sync_
integrity check is performed at the end of the sync run by default.
Any untrapped failing copy/deletes between the two paths might result
in differences between the two listings and in the untracked file content
differences between the two paths. A resync run would correct the error.
Note that the default-enabled integrity check locally executes a load of both
the final Path1 and Path2 listings, and thus adds to the run time of a sync.
Using `--check-sync=false` will disable it and may significantly reduce the
sync run times for very large numbers of files.
The check may be run manually with `--check-sync=only`. It runs only the
integrity check and terminates without actually synching.
Note that currently, `--check-sync` **only checks listing snapshots and NOT the
actual files on the remotes.** Note also that the listing snapshots will not
know about any changes that happened during or after the latest bisync run, as
those will be discovered on the next run. Therefore, while listings should
always match _each other_ at the end of a bisync run, it is _expected_ that
they will not match the underlying remotes, nor will the remotes match each
other, if there were changes during or after the run. This is normal, and any
differences will be detected and synced on the next run.
For a robust integrity check of the current state of the remotes (as opposed to just their listing snapshots), consider using [`check`](commands/rclone_check/)
(or [`cryptcheck`](/commands/rclone_cryptcheck/), if at least one path is a `crypt` remote) instead of `--check-sync`,
keeping in mind that differences are expected if files changed during or after your last bisync run.
For example, a possible sequence could look like this:
1. Normally scheduled bisync run:
```
rclone bisync Path1 Path2 -MPc --check-access --max-delete 10 --filters-file /path/to/filters.txt -v --no-cleanup --ignore-listing-checksum --disable ListR --checkers=16 --drive-pacer-min-sleep=10ms --create-empty-src-dirs --resilient
```
2. Periodic independent integrity check (perhaps scheduled nightly or weekly):
```
rclone check -MvPc Path1 Path2 --filter-from /path/to/filters.txt
```
3. If diffs are found, you have some choices to correct them.
If one side is more up-to-date and you want to make the other side match it, you could run:
```
rclone sync Path1 Path2 --filter-from /path/to/filters.txt --create-empty-src-dirs -MPc -v
```
(or switch Path1 and Path2 to make Path2 the source-of-truth)
Or, if neither side is totally up-to-date, you could run a `--resync` to bring them back into agreement
(but remember that this could cause deleted files to re-appear.)
*Note also that `rclone check` does not currently include empty directories,
so if you want to know if any empty directories are out of sync,
consider alternatively running the above `rclone sync` command with `--dry-run` added.
See also: [Concurrent modifications](#concurrent-modifications), [`--resilient`](#resilient)
### --resilient
***Caution: this is an experimental feature. Use at your own risk!***
By default, most errors or interruptions will cause bisync to abort and
require [`--resync`](#resync) to recover. This is a safety feature, to prevent
bisync from running again until a user checks things out. However, in some
cases, bisync can go too far and enforce a lockout when one isn't actually
necessary, like for certain less-serious errors that might resolve themselves
on the next run. When `--resilient` is specified, bisync tries its best to
recover and self-correct, and only requires `--resync` as a last resort when a
human's involvement is absolutely necessary. The intended use case is for
running bisync as a background process (such as via scheduled [cron](#cron)).
When using `--resilient` mode, bisync will still report the error and abort,
however it will not lock out future runs -- allowing the possibility of
retrying at the next normally scheduled time, without requiring a `--resync`
first. Examples of such retryable errors include access test failures, missing
listing files, and filter change detections. These safety features will still
prevent the *current* run from proceeding -- the difference is that if
conditions have improved by the time of the *next* run, that next run will be
allowed to proceed. Certain more serious errors will still enforce a
`--resync` lockout, even in `--resilient` mode, to prevent data loss.
Behavior of `--resilient` may change in a future version. (See also:
[`--recover`](#recover), [`--max-lock`](#max-lock), [Graceful
Shutdown](#graceful-shutdown))
### --recover
If `--recover` is set, in the event of a sudden interruption or other
un-graceful shutdown, bisync will attempt to automatically recover on the next
run, instead of requiring `--resync`. Bisync is able to recover robustly by
keeping one "backup" listing at all times, representing the state of both paths
after the last known successful sync. Bisync can then compare the current state
with this snapshot to determine which changes it needs to retry. Changes that
were synced after this snapshot (during the run that was later interrupted)
will appear to bisync as if they are "new or changed on both sides", but in
most cases this is not a problem, as bisync will simply do its usual "equality
check" and learn that no action needs to be taken on these files, since they
are already identical on both sides.
In the rare event that a file is synced successfully during a run that later
aborts, and then that same file changes AGAIN before the next run, bisync will
think it is a sync conflict, and handle it accordingly. (From bisync's
perspective, the file has changed on both sides since the last trusted sync,
and the files on either side are not currently identical.) Therefore,
`--recover` carries with it a slightly increased chance of having conflicts --
though in practice this is pretty rare, as the conditions required to cause it
are quite specific. This risk can be reduced by using bisync's ["Graceful
Shutdown"](#graceful-shutdown) mode (triggered by sending `SIGINT` or
`Ctrl+C`), when you have the choice, instead of forcing a sudden termination.
`--recover` and `--resilient` are similar, but distinct -- the main difference
is that `--resilient` is about _retrying_, while `--recover` is about
_recovering_. Most users will probably want both. `--resilient` allows retrying
when bisync has chosen to abort itself due to safety features such as failing
`--check-access` or detecting a filter change. `--resilient` does not cover
external interruptions such as a user shutting down their computer in the
middle of a sync -- that is what `--recover` is for.
### --max-lock
Bisync uses [lock files](#lock-file) as a safety feature to prevent
interference from other bisync runs while it is running. Bisync normally
removes these lock files at the end of a run, but if bisync is abruptly
interrupted, these files will be left behind. By default, they will lock out
all future runs, until the user has a chance to manually check things out and
remove the lock. As an alternative, `--max-lock` can be used to make them
automatically expire after a certain period of time, so that future runs are
not locked out forever, and auto-recovery is possible. `--max-lock` can be any
duration `2m` or greater (or `0` to disable). If set, lock files older than
this will be considered "expired", and future runs will be allowed to disregard
them and proceed. (Note that the `--max-lock` duration must be set by the
process that left the lock file -- not the later one interpreting it.)
If set, bisync will also "renew" these lock files every `--max-lock minus one
minute` throughout a run, for extra safety. (For example, with `--max-lock 5m`,
bisync would renew the lock file (for another 5 minutes) every 4 minutes until
the run has completed.) In other words, it should not be possible for a lock
file to pass its expiration time while the process that created it is still
running -- and you can therefore be reasonably sure that any _expired_ lock
file you may find was left there by an interrupted run, not one that is still
running and just taking awhile.
If `--max-lock` is `0` or not set, the default is that lock files will never
expire, and will block future runs (of these same two bisync paths)
indefinitely.
For maximum resilience from disruptions, consider setting a relatively short
duration like `--max-lock 2m` along with [`--resilient`](#resilient) and
[`--recover`](#recover), and a relatively frequent [cron schedule](#cron). The
result will be a very robust "set-it-and-forget-it" bisync run that can
automatically bounce back from almost any interruption it might encounter,
without requiring the user to get involved and run a `--resync`. (See also:
[Graceful Shutdown](#graceful-shutdown) mode)
### --backup-dir1 and --backup-dir2
As of `v1.66`, [`--backup-dir`](/docs/#backup-dir-dir) is supported in bisync.
Because `--backup-dir` must be a non-overlapping path on the same remote,
Bisync has introduced new `--backup-dir1` and `--backup-dir2` flags to support
separate backup-dirs for `Path1` and `Path2` (bisyncing between different
remotes with `--backup-dir` would not otherwise be possible.) `--backup-dir1`
and `--backup-dir2` can use different remotes from each other, but
`--backup-dir1` must use the same remote as `Path1`, and `--backup-dir2` must
use the same remote as `Path2`. Each backup directory must not overlap its
respective bisync Path without being excluded by a filter rule.
The standard `--backup-dir` will also work, if both paths use the same remote
(but note that deleted files from both paths would be mixed together in the
same dir). If either `--backup-dir1` and `--backup-dir2` are set, they will
override `--backup-dir`.
Example:
```
rclone bisync /Users/someuser/some/local/path/Bisync gdrive:Bisync --backup-dir1 /Users/someuser/some/local/path/BackupDir --backup-dir2 gdrive:BackupDir --suffix -2023-08-26 --suffix-keep-extension --check-access --max-delete 10 --filters-file /Users/someuser/some/local/path/bisync_filters.txt --no-cleanup --ignore-listing-checksum --checkers=16 --drive-pacer-min-sleep=10ms --create-empty-src-dirs --resilient -MvP --drive-skip-gdocs --fix-case
```
In this example, if the user deletes a file in
`/Users/someuser/some/local/path/Bisync`, bisync will propagate the delete to
the other side by moving the corresponding file from `gdrive:Bisync` to
`gdrive:BackupDir`. If the user deletes a file from `gdrive:Bisync`, bisync
moves it from `/Users/someuser/some/local/path/Bisync` to
`/Users/someuser/some/local/path/BackupDir`.
In the event of a [rename due to a sync conflict](#conflict-loser), the
rename is not considered a delete, unless a previous conflict with the same
name already exists and would get overwritten.
See also: [`--suffix`](/docs/#suffix-suffix),
[`--suffix-keep-extension`](/docs/#suffix-keep-extension)
## Operation
### Runtime flow details
bisync retains the listings of the `Path1` and `Path2` filesystems
from the prior run.
On each successive run it will:
- list files on `path1` and `path2`, and check for changes on each side.
Changes include `New`, `Newer`, `Older`, and `Deleted` files.
- Propagate changes on `path1` to `path2`, and vice-versa.
### Safety measures
- Lock file prevents multiple simultaneous runs when taking a while.
This can be particularly useful if bisync is run by cron scheduler.
- Handle change conflicts non-destructively by creating
`.conflict1`, `.conflict2`, etc. file versions, according to
[`--conflict-resolve`](#conflict-resolve), [`--conflict-loser`](#conflict-loser), and [`--conflict-suffix`](#conflict-suffix) settings.
- File system access health check using `RCLONE_TEST` files
(see the `--check-access` flag).
- Abort on excessive deletes - protects against a failed listing
being interpreted as all the files were deleted.
See the `--max-delete` and `--force` flags.
- If something evil happens, bisync goes into a safe state to block
damage by later runs. (See [Error Handling](#error-handling))
### Normal sync checks
Type | Description | Result | Implementation
--------------|-----------------------------------------------|--------------------------|-----------------------------
Path2 new | File is new on Path2, does not exist on Path1 | Path2 version survives | `rclone copy` Path2 to Path1
Path2 newer | File is newer on Path2, unchanged on Path1 | Path2 version survives | `rclone copy` Path2 to Path1
Path2 deleted | File is deleted on Path2, unchanged on Path1 | File is deleted | `rclone delete` Path1
Path1 new | File is new on Path1, does not exist on Path2 | Path1 version survives | `rclone copy` Path1 to Path2
Path1 newer | File is newer on Path1, unchanged on Path2 | Path1 version survives | `rclone copy` Path1 to Path2
Path1 older | File is older on Path1, unchanged on Path2 | _Path1 version survives_ | `rclone copy` Path1 to Path2
Path2 older | File is older on Path2, unchanged on Path1 | _Path2 version survives_ | `rclone copy` Path2 to Path1
Path1 deleted | File no longer exists on Path1 | File is deleted | `rclone delete` Path2
### Unusual sync checks
Type | Description | Result | Implementation
--------------------------------|---------------------------------------|------------------------------------|-----------------------
Path1 new/changed AND Path2 new/changed AND Path1 == Path2 | File is new/changed on Path1 AND new/changed on Path2 AND Path1 version is currently identical to Path2 | No change | None
Path1 new AND Path2 new | File is new on Path1 AND new on Path2 (and Path1 version is NOT identical to Path2) | Conflicts handled according to [`--conflict-resolve`](#conflict-resolve) & [`--conflict-loser`](#conflict-loser) settings | default: `rclone copy` renamed `Path2.conflict2` file to Path1, `rclone copy` renamed `Path1.conflict1` file to Path2
Path2 newer AND Path1 changed | File is newer on Path2 AND also changed (newer/older/size) on Path1 (and Path1 version is NOT identical to Path2) | Conflicts handled according to [`--conflict-resolve`](#conflict-resolve) & [`--conflict-loser`](#conflict-loser) settings | default: `rclone copy` renamed `Path2.conflict2` file to Path1, `rclone copy` renamed `Path1.conflict1` file to Path2
Path2 newer AND Path1 deleted | File is newer on Path2 AND also deleted on Path1 | Path2 version survives | `rclone copy` Path2 to Path1
Path2 deleted AND Path1 changed | File is deleted on Path2 AND changed (newer/older/size) on Path1 | Path1 version survives |`rclone copy` Path1 to Path2
Path1 deleted AND Path2 changed | File is deleted on Path1 AND changed (newer/older/size) on Path2 | Path2 version survives | `rclone copy` Path2 to Path1
As of `rclone v1.64`, bisync is now better at detecting *false positive* sync conflicts,
which would previously have resulted in unnecessary renames and duplicates.
Now, when bisync comes to a file that it wants to rename (because it is new/changed on both sides),
it first checks whether the Path1 and Path2 versions are currently *identical*
(using the same underlying function as [`check`](commands/rclone_check/).)
If bisync concludes that the files are identical, it will skip them and move on.
Otherwise, it will create renamed duplicates, as before.
This behavior also [improves the experience of renaming directories](https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=Renamed%20directories),
as a `--resync` is no longer required, so long as the same change has been made on both sides.
### All files changed check {#all-files-changed}
If _all_ prior existing files on either of the filesystems have changed
(e.g. timestamps have changed due to changing the system's timezone)
then bisync will abort without making any changes.
Any new files are not considered for this check. You could use `--force`
to force the sync (whichever side has the changed timestamp files wins).
Alternately, a `--resync` may be used (Path1 versions will be pushed
to Path2). Consider the situation carefully and perhaps use `--dry-run`
before you commit to the changes.
### Modification times
By default, bisync compares files by modification time and size.
If you or your application should change the content of a file
without changing the modification time and size, then bisync will _not_
notice the change, and thus will not copy it to the other side.
As an alternative, consider comparing by checksum (if your remotes support it).
See [`--compare`](#compare) for details.
### Error handling {#error-handling}
Certain bisync critical errors, such as file copy/move failing, will result in
a bisync lockout of following runs. The lockout is asserted because the sync
status and history of the Path1 and Path2 filesystems cannot be trusted,
so it is safer to block any further changes until someone checks things out.
The recovery is to do a `--resync` again.
It is recommended to use `--resync --dry-run --verbose` initially and
_carefully_ review what changes will be made before running the `--resync`
without `--dry-run`.
Most of these events come up due to an error status from an internal call.
On such a critical error the `{...}.path1.lst` and `{...}.path2.lst`
listing files are renamed to extension `.lst-err`, which blocks any future
bisync runs (since the normal `.lst` files are not found).
Bisync keeps them under `bisync` subdirectory of the rclone cache directory,
typically at `${HOME}/.cache/rclone/bisync/` on Linux.
Some errors are considered temporary and re-running the bisync is not blocked.
The _critical return_ blocks further bisync runs.
See also: [`--resilient`](#resilient), [`--recover`](#recover),
[`--max-lock`](#max-lock), [Graceful Shutdown](#graceful-shutdown)
### Lock file
When bisync is running, a lock file is created in the bisync working directory,
typically at `~/.cache/rclone/bisync/PATH1..PATH2.lck` on Linux.
If bisync should crash or hang, the lock file will remain in place and block
any further runs of bisync _for the same paths_.
Delete the lock file as part of debugging the situation.
The lock file effectively blocks follow-on (e.g., scheduled by _cron_) runs
when the prior invocation is taking a long time.
The lock file contains _PID_ of the blocking process, which may help in debug.
Lock files can be set to automatically expire after a certain amount of time,
using the [`--max-lock`](#max-lock) flag.
**Note**
that while concurrent bisync runs are allowed, _be very cautious_
that there is no overlap in the trees being synched between concurrent runs,
lest there be replicated files, deleted files and general mayhem.
### Return codes
`rclone bisync` returns the following codes to calling program:
- `0` on a successful run,
- `1` for a non-critical failing run (a rerun may be successful),
- `2` for a critically aborted run (requires a `--resync` to recover).
### Graceful Shutdown
Bisync has a "Graceful Shutdown" mode which is activated by sending `SIGINT` or
pressing `Ctrl+C` during a run. Once triggered, bisync will use best efforts to
exit cleanly before the timer runs out. If bisync is in the middle of
transferring files, it will attempt to cleanly empty its queue by finishing
what it has started but not taking more. If it cannot do so within 30 seconds,
it will cancel the in-progress transfers at that point and then give itself a
maximum of 60 seconds to wrap up, save its state for next time, and exit. With
the `-vP` flags you will see constant status updates and a final confirmation
of whether or not the graceful shutdown was successful.
At any point during the "Graceful Shutdown" sequence, a second `SIGINT` or
`Ctrl+C` will trigger an immediate, un-graceful exit, which will leave things
in a messier state. Usually a robust recovery will still be possible if using
[`--recover`](#recover) mode, otherwise you will need to do a `--resync`.
If you plan to use Graceful Shutdown mode, it is recommended to use
[`--resilient`](#resilient) and [`--recover`](#recover), and it is important to
NOT use [`--inplace`](/docs/#inplace), otherwise you risk leaving
partially-written files on one side, which may be confused for real files on
the next run. Note also that in the event of an abrupt interruption, a [lock
file](#lock-file) will be left behind to block concurrent runs. You will need