Syncing large repository #440

mathewmeconry · 2013-11-13T08:18:04Z

Hi there,

I've a repository with over 115.8GB and it's not syncing. The initial sync worked and from time to time some files get synced. Some of the files are huge (.iso files). I've also some vmdk's which should be skipped by the syncclient and it is configured with the .seafile-ignore.txt.
Also the seafile-data folder is quite huge with 55GB.

I tried to sync over this night but nothing happend.

Attached you've the ccnet-logfile and the seafile-logfile and a screenshot of the syncfolder and the seafile-data.

Thanks

Folder to sync:

seafile-data

ccnet logfile:
[11/12/13 20:18:53] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1741, 500 Shutdown
[11/12/13 20:47:48] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1875, 500 Shutdown
[11/12/13 20:53:56] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1905, 500 Shutdown
[11/12/13 20:58:25] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1926, 500 Shutdown
[11/12/13 21:01:08] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1938, 500 Shutdown
[11/12/13 21:03:23] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1950, 500 Shutdown
[11/12/13 21:05:40] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1961, 500 Shutdown
[11/12/13 21:07:54] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1973, 500 Shutdown
[11/12/13 21:10:07] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1985, 500 Shutdown
[11/12/13 21:12:15] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1996, 500 Shutdown
[11/12/13 21:14:33] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2008, 500 Shutdown
[11/12/13 21:17:10] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2020, 500 Shutdown
[11/12/13 21:19:35] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2032, 500 Shutdown
[11/12/13 21:22:18] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2044, 500 Shutdown
[11/12/13 21:24:47] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2056, 500 Shutdown
[11/12/13 21:27:06] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2068, 500 Shutdown
[11/12/13 21:29:18] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 2079, 500 Shutdown
[11/12/13 22:36:27] ../common/peer.c(942): Local peer down
[11/12/13 22:36:27] ../common/peer.c(942): Local peer down
[11/12/13 22:36:27] ../common/processor.c(219): [Proc] Shutdown processor rpcserver-proc(-1002) for bad update: 515 peer down
[11/12/13 22:36:27] ../common/processor.c(219): [Proc] Shutdown processor service-proxy-proc(-1001) for bad update: 515 peer down
[11/12/13 22:36:27] ../common/processor.c(219): [Proc] Shutdown processor service-stub-proc(1001) for bad update: 515 peer down
[11/12/13 22:36:27] ../common/processor.c(219): [Proc] Shutdown processor mqserver-proc(-1001) for bad update: 515 peer down
[11/12/13 22:36:31] ../common/session.c(336): Exit at Tue Nov 12 22:36:31 2013

[11/12/13 22:37:06] ccnet-daemon.c(168): starting ccnet client 2.0.5
[11/12/13 22:37:06] ccnet-daemon.c(170): ccnet source code version fc7228e80057e1e49892edb539099d900702b90d
[11/12/13 22:37:06] ../common/session.c(385): Listen on 127.0.0.1 13419
[11/12/13 22:37:06] ../common/session.c(257): Update pubinfo file
[11/12/13 22:37:07] ../common/session.c(365): Accepted a local client
[11/12/13 22:37:07] ../common/session.c(365): Accepted a local client
[11/12/13 22:37:07] ../common/session.c(365): Accepted a local client
[11/12/13 22:37:07] ../common/session.c(365): Accepted a local client
[11/12/13 22:37:07] ../common/session.c(365): Accepted a local client
[11/12/13 22:37:07] ../common/session.c(365): Accepted a local client
[11/12/13 22:38:38] ../common/processors/rcvcmd-proc.c(519): Add server 31c365ef sch-network.zapto.org:10001
[11/12/13 22:38:40] ../common/connect-mgr.c(364): [Conn] Start outgoing connect to (null)(31c365efdb) 46.126.232.231:10001
[11/12/13 22:38:41] ../common/connect-mgr.c(289): [Conn] Peer (null) (31c365efdb) connected
[11/12/13 22:41:46] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1014, 500 Shutdown
[11/12/13 22:49:51] ../common/peer.c(942): Local peer down
[11/12/13 22:49:51] ../common/peer.c(942): Local peer down
[11/12/13 22:49:51] ../common/processor.c(219): [Proc] Shutdown processor rpcserver-proc(-1002) for bad update: 515 peer down
[11/12/13 22:49:51] ../common/processor.c(219): [Proc] Shutdown processor service-proxy-proc(-1001) for bad update: 515 peer down
[11/12/13 22:49:51] ../common/processor.c(219): [Proc] Shutdown processor service-stub-proc(1001) for bad update: 515 peer down
[11/12/13 22:49:51] ../common/processor.c(219): [Proc] Shutdown processor mqserver-proc(-1001) for bad update: 515 peer down
[11/12/13 22:49:55] ../common/session.c(336): Exit at Tue Nov 12 22:49:55 2013

[11/12/13 23:36:49] ccnet-daemon.c(168): starting ccnet client 2.0.5
[11/12/13 23:36:49] ccnet-daemon.c(170): ccnet source code version fc7228e80057e1e49892edb539099d900702b90d
[11/12/13 23:36:49] ../common/session.c(385): Listen on 127.0.0.1 13419
[11/12/13 23:36:49] ../common/session.c(257): Update pubinfo file
[11/12/13 23:36:50] ../common/session.c(365): Accepted a local client
[11/12/13 23:36:50] ../common/session.c(365): Accepted a local client
[11/12/13 23:36:50] ../common/session.c(365): Accepted a local client
[11/12/13 23:36:50] ../common/session.c(365): Accepted a local client
[11/12/13 23:36:50] ../common/session.c(365): Accepted a local client
[11/12/13 23:36:50] ../common/session.c(365): Accepted a local client
[11/12/13 23:38:08] ../common/processors/rcvcmd-proc.c(519): Add server 31c365ef sch-network.zapto.org:10001
[11/12/13 23:38:10] ../common/connect-mgr.c(364): [Conn] Start outgoing connect to (null)(31c365efdb) 46.126.232.231:10001
[11/12/13 23:38:10] ../common/connect-mgr.c(289): [Conn] Peer (null) (31c365efdb) connected
[11/12/13 23:48:44] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1032, 500 Shutdown
[11/13/13 00:56:34] ccnet-daemon.c(168): starting ccnet client 2.0.5
[11/13/13 00:56:34] ccnet-daemon.c(170): ccnet source code version fc7228e80057e1e49892edb539099d900702b90d
[11/13/13 00:56:34] ../common/session.c(385): Listen on 127.0.0.1 13419
[11/13/13 00:56:34] ../common/session.c(257): Update pubinfo file
[11/13/13 00:56:35] ../common/session.c(365): Accepted a local client
[11/13/13 00:56:35] ../common/session.c(365): Accepted a local client
[11/13/13 00:56:35] ../common/session.c(365): Accepted a local client
[11/13/13 00:56:35] ../common/session.c(365): Accepted a local client
[11/13/13 00:56:35] ../common/session.c(365): Accepted a local client
[11/13/13 00:56:35] ../common/session.c(365): Accepted a local client
[11/13/13 00:56:47] ../common/processors/rcvcmd-proc.c(519): Add server 31c365ef sch-network.zapto.org:10001
[11/13/13 00:56:47] ../common/connect-mgr.c(364): [Conn] Start outgoing connect to (null)(31c365efdb) 46.126.232.231:10001
[11/13/13 00:56:48] ../common/connect-mgr.c(289): [Conn] Peer (null) (31c365efdb) connected
[11/13/13 06:01:08] ../common/peer.c(940): libevent got an error! what=33, errno=104 (Connection reset by peer)
[11/13/13 06:01:08] ../common/peer.c(948): [Net Error] Peer seafile (31c365efdb) down
[11/13/13 06:01:08] ../common/processor.c(293): [Proc] peer 31c365efdb, Shutdown processor service-stub-proc(2398) for bad response: 515 peer down
[11/13/13 06:01:08] ../common/processor.c(293): [Proc] peer b910df591b, Shutdown processor service-proxy-proc(-2398) for bad response: 515 peer down
[11/13/13 06:01:08] ../common/processor.c(293): [Proc] peer 31c365efdb, Shutdown processor service-stub-proc(2397) for bad response: 515 peer down
[11/13/13 06:01:08] ../common/processor.c(293): [Proc] peer b910df591b, Shutdown processor service-proxy-proc(-2397) for bad response: 515 peer down
[11/13/13 06:01:08] ../common/processor.c(293): [Proc] peer 31c365efdb, Shutdown processor service-stub-proc(2396) for bad response: 515 peer down
[11/13/13 06:01:08] ../common/processor.c(293): [Proc] peer b910df591b, Shutdown processor service-proxy-proc(-2396) for bad response: 515 peer down
[11/13/13 06:01:08] ../common/processor.c(219): [Proc] Shutdown processor keepalive2-proc(-1006) for bad update: 515 peer down
[11/13/13 06:01:08] ../common/processors/keepalive2-proc.c(476): [Keepalive] Error from peer 515 peer down
[11/13/13 06:01:17] ../common/connect-mgr.c(289): [Conn] Peer seafile (31c365efdb) connected
[11/13/13 07:48:38] ../common/peer.c(942): Local peer down
[11/13/13 07:48:38] ../common/peer.c(942): Local peer down
[11/13/13 07:48:38] ../common/processor.c(219): [Proc] Shutdown processor rpcserver-proc(-1002) for bad update: 515 peer down
[11/13/13 07:48:38] ../common/processor.c(219): [Proc] Shutdown processor service-proxy-proc(-1001) for bad update: 515 peer down
[11/13/13 07:48:38] ../common/processor.c(219): [Proc] Shutdown processor service-stub-proc(1001) for bad update: 515 peer down
[11/13/13 07:48:38] ../common/processor.c(219): [Proc] Shutdown processor mqserver-proc(-1001) for bad update: 515 peer down
[11/13/13 07:48:42] ../common/session.c(336): Exit at Wed Nov 13 07:48:42 2013

[11/13/13 08:28:33] ccnet-daemon.c(168): starting ccnet client 2.0.5
[11/13/13 08:28:33] ccnet-daemon.c(170): ccnet source code version fc7228e80057e1e49892edb539099d900702b90d
[11/13/13 08:28:33] ../common/session.c(385): Listen on 127.0.0.1 13419
[11/13/13 08:28:33] ../common/session.c(257): Update pubinfo file
[11/13/13 08:28:34] ../common/session.c(365): Accepted a local client
[11/13/13 08:28:34] ../common/session.c(365): Accepted a local client
[11/13/13 08:28:34] ../common/session.c(365): Accepted a local client
[11/13/13 08:28:34] ../common/session.c(365): Accepted a local client
[11/13/13 08:28:34] ../common/session.c(365): Accepted a local client
[11/13/13 08:28:34] ../common/session.c(365): Accepted a local client
[11/13/13 08:29:50] ../common/processors/rcvcmd-proc.c(519): Add server 31c365ef sch-network.zapto.org:10001
[11/13/13 08:29:50] ../common/connect-mgr.c(364): [Conn] Start outgoing connect to (null)(31c365efdb) 46.126.232.231:10001
[11/13/13 08:29:51] ../common/connect-mgr.c(289): [Conn] Peer (null) (31c365efdb) connected
[11/13/13 08:35:23] ../common/peer.c(763): Delayed update from local-2(b910df59), id is 1025, 500 Shutdown

seafile logfile:
[11/13/13 07:45:09] type: W, status: M, name: ABU/4. Lehrjahr/VA/mscherer/Einsatzuebersicht_Baden.xcf, ce mtime: 1383859154, ce size: 1929409, file mtime: 1383859154, file size: 1929409
[11/13/13 07:46:39] Failed to build cache tree[11/13/13 07:46:39] sync-mgr.c(589): Failed to commit unclean worktree.
[11/13/13 07:46:39] sync-mgr.c(624): [sync mgr] Worktree is not clean. Skip merging repo BBB(87a2fc08).
[11/13/13 07:46:39] sync-mgr.c(496): Repo 'BBB' sync state transition from merging to 'error': 'Files changed in local folder, skip merge.'.
[11/13/13 07:46:40] sync-mgr.c(425): Repo 'BBB' sync state transition from 'initializing' to 'merging'.
[11/13/13 07:46:53] Worktree changes (at most 5 files are shown):
[11/13/13 07:46:53] Cannot find diff entry VM's in index.
[11/13/13 07:46:53] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_Unternehmensmodellierung.docx, ce mtime: 1383724858, ce size: 444383, file mtime: 1383724870, file size: 444383
[11/13/13 07:46:53] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_1.0.docx, ce mtime: 1383724853, ce size: 140252, file mtime: 1384106966, file size: 251774
[11/13/13 07:46:53] Cannot find diff entry ABU/4. Lehrjahr/VA/mscherer/VA.docx in index.
[11/13/13 07:46:53] type: W, status: M, name: ABU/4. Lehrjahr/VA/mscherer/Einsatzuebersicht_Baden.xcf, ce mtime: 1383859154, ce size: 1929409, file mtime: 1383859154, file size: 1929409
[11/13/13 07:48:27] Failed to build cache tree[11/13/13 07:48:27] sync-mgr.c(589): Failed to commit unclean worktree.
[11/13/13 07:48:27] sync-mgr.c(624): [sync mgr] Worktree is not clean. Skip merging repo BBB(87a2fc08).
[11/13/13 07:48:27] sync-mgr.c(496): Repo 'BBB' sync state transition from merging to 'error': 'Files changed in local folder, skip merge.'.
[11/13/13 07:48:28] sync-mgr.c(425): Repo 'BBB' sync state transition from 'initializing' to 'merging'.
[11/13/13 07:48:38] ccnet_processor_handle_update: [Proc] Shutdown processor rpcserver-proc(-1001) for bad update: 515 peer down
[11/13/13 07:48:40] Worktree changes (at most 5 files are shown):
[11/13/13 07:48:40] Cannot find diff entry VM's in index.
[11/13/13 07:48:40] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_Unternehmensmodellierung.docx, ce mtime: 1383724858, ce size: 444383, file mtime: 1383724870, file size: 444383
[11/13/13 07:48:40] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_1.0.docx, ce mtime: 1383724853, ce size: 140252, file mtime: 1384106966, file size: 251774
[11/13/13 07:48:40] Cannot find diff entry ABU/4. Lehrjahr/VA/mscherer/VA.docx in index.
[11/13/13 07:48:40] type: W, status: M, name: ABU/4. Lehrjahr/VA/mscherer/Einsatzuebersicht_Baden.xcf, ce mtime: 1383859154, ce size: 1929409, file mtime: 1383859154, file size: 1929409
[11/13/13 08:28:37] seaf-daemon.c(467): starting seafile client 2.0.5
[11/13/13 08:28:37] seaf-daemon.c(469): seafile source code version 4a0d0c8
[11/13/13 08:28:37] ../common/mq-mgr.c(60): [mq client] mq cilent is started
[11/13/13 08:28:38] ../common/mq-mgr.c(106): [mq mgr] publish to hearbeat mq: seafile.heartbeat
[11/13/13 08:28:45] ../common/gc-core.c(317): GC started. Total block number is 18734.
[11/13/13 08:28:45] ../common/gc-core.c(45): GC index size is 9367 Byte.
[11/13/13 08:28:45] ../common/gc-core.c(331): Populating index.
[11/13/13 08:28:45] ../common/gc-core.c(176): Populating index for repo aa1a122f.
[11/13/13 08:28:45] ../common/gc-core.c(243): Traversed 0 commits, 0 blocks.
[11/13/13 08:28:45] ../common/gc-core.c(176): Populating index for repo 87a2fc08.
[11/13/13 08:29:48] ../common/gc-core.c(243): Traversed 63 commits, 62930 blocks.
[11/13/13 08:29:48] ../common/gc-core.c(176): Populating index for repo 13e0733d.
[11/13/13 08:29:48] ../common/gc-core.c(243): Traversed 0 commits, 0 blocks.
[11/13/13 08:29:48] ../common/gc-core.c(371): Scanning and deleting unused blocks.
[11/13/13 08:29:50] ../common/gc-core.c(391): GC finished. 18734 blocks total, about 62930 reachable blocks, 291 blocks are removed.
[11/13/13 08:30:42] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:30:42] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:30:42] sync-mgr.c(425): Repo 'Bilder' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:30:42] sync-mgr.c(425): Repo 'Bilder' sync state transition from 'committing' to 'initializing'.
[11/13/13 08:30:46] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'committing' to 'initializing'.
[11/13/13 08:33:08] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:35:16] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'committing' to 'initializing'.
[11/13/13 08:35:17] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'initializing' to 'uploading'.
[11/13/13 08:35:17] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'init') --> ('normal', 'check')
[11/13/13 08:35:18] processors/check-tx-v3-proc.c(260): protocol version is 5.
[11/13/13 08:35:18] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'check') --> ('normal', 'commit')
[11/13/13 08:35:18] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'commit') --> ('normal', 'fs')
[11/13/13 08:35:21] Failed to build cache tree[11/13/13 08:35:21] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 08:35:21] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'fs') --> ('normal', 'check-blocks')
[11/13/13 08:35:21] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 08:35:23] sync-mgr.c(425): Repo 'BBB' sync state transition from 'initializing' to 'merging'.
[11/13/13 08:35:23] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'check-blocks') --> ('normal', 'get-chunk-server')
[11/13/13 08:35:23] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'get-chunk-server') --> ('normal', 'data')
[11/13/13 08:35:23] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'data') --> ('normal', 'update-branch')
[11/13/13 08:35:23] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'update-branch') --> ('finished', 'finished')
[11/13/13 08:35:29] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'uploading' to 'synchronized'.
[11/13/13 08:35:29] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:37:38] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'committing' to 'initializing'.
[11/13/13 08:37:38] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'initializing' to 'uploading'.
[11/13/13 08:37:38] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'init') --> ('normal', 'check')
[11/13/13 08:37:38] processors/check-tx-v3-proc.c(260): protocol version is 5.
[11/13/13 08:37:38] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'check') --> ('normal', 'commit')
[11/13/13 08:37:38] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'commit') --> ('normal', 'fs')
[11/13/13 08:37:41] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'fs') --> ('normal', 'update-branch')
[11/13/13 08:37:41] transfer-mgr.c(502): Transfer repo 'aa1a122f': ('normal', 'update-branch') --> ('finished', 'finished')
[11/13/13 08:37:47] sync-mgr.c(425): Repo 'Dokumente' sync state transition from 'uploading' to 'synchronized'.
[11/13/13 08:37:53] Worktree changes (at most 5 files are shown):
[11/13/13 08:37:53] Cannot find diff entry VM's in index.
[11/13/13 08:37:53] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_Unternehmensmodellierung.docx, ce mtime: 1383724858, ce size: 444383, file mtime: 1383724870, file size: 444383
[11/13/13 08:37:53] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_1.0.docx, ce mtime: 1383724853, ce size: 140252, file mtime: 1384106966, file size: 251774
[11/13/13 08:37:53] Cannot find diff entry ABU/4. Lehrjahr/VA/mscherer/VA.docx in index.
[11/13/13 08:37:53] type: W, status: M, name: ABU/4. Lehrjahr/VA/mscherer/Einsatzuebersicht_Baden.xcf, ce mtime: 1383859154, ce size: 1929409, file mtime: 1383859154, file size: 1929409
[11/13/13 08:39:30] Failed to build cache tree[11/13/13 08:39:30] sync-mgr.c(589): Failed to commit unclean worktree.
[11/13/13 08:39:30] sync-mgr.c(624): [sync mgr] Worktree is not clean. Skip merging repo BBB(87a2fc08).
[11/13/13 08:39:30] sync-mgr.c(496): Repo 'BBB' sync state transition from merging to 'error': 'Files changed in local folder, skip merge.'.
[11/13/13 08:39:32] sync-mgr.c(425): Repo 'BBB' sync state transition from 'initializing' to 'merging'.
[11/13/13 08:39:44] Worktree changes (at most 5 files are shown):
[11/13/13 08:39:44] Cannot find diff entry VM's in index.
[11/13/13 08:39:44] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_Unternehmensmodellierung.docx, ce mtime: 1383724858, ce size: 444383, file mtime: 1383724870, file size: 444383
[11/13/13 08:39:44] type: W, status: M, name: EBW/Modul 254/254_AA_7413_Prozesslandkarte_1.0/254_AA_7413_Prozesslandkarte_1.0.docx, ce mtime: 1383724853, ce size: 140252, file mtime: 1384106966, file size: 251774
[11/13/13 08:39:44] Cannot find diff entry ABU/4. Lehrjahr/VA/mscherer/VA.docx in index.
[11/13/13 08:39:44] type: W, status: M, name: ABU/4. Lehrjahr/VA/mscherer/Einsatzuebersicht_Baden.xcf, ce mtime: 1383859154, ce size: 1929409, file mtime: 1383859154, file size: 1929409
[11/13/13 08:42:04] Failed to build cache tree[11/13/13 08:42:04] sync-mgr.c(589): Failed to commit unclean worktree.
[11/13/13 08:42:04] sync-mgr.c(624): [sync mgr] Worktree is not clean. Skip merging repo BBB(87a2fc08).
[11/13/13 08:42:04] sync-mgr.c(496): Repo 'BBB' sync state transition from merging to 'error': 'Files changed in local folder, skip merge.'.
[11/13/13 08:42:04] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:44:16] Failed to build cache tree[11/13/13 08:44:16] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 08:44:16] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 08:44:17] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:48:07] Failed to build cache tree[11/13/13 08:48:07] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 08:48:07] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 08:48:07] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:52:43] Failed to build cache tree[11/13/13 08:52:43] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 08:52:43] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 08:52:44] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:56:59] Failed to build cache tree[11/13/13 08:56:59] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 08:56:59] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 08:57:00] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 08:59:33] Failed to build cache tree[11/13/13 08:59:33] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 08:59:33] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 08:59:34] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:02:04] Failed to build cache tree[11/13/13 09:02:04] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:02:04] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:02:04] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:04:55] Failed to build cache tree[11/13/13 09:04:55] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:04:55] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:04:55] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:07:01] Failed to build cache tree[11/13/13 09:07:01] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:07:01] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:07:01] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:09:08] Failed to build cache tree[11/13/13 09:09:08] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:09:08] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:09:09] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:11:23] Failed to build cache tree[11/13/13 09:11:23] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:11:23] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:11:24] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:13:40] Failed to build cache tree[11/13/13 09:13:40] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:13:40] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:13:40] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.
[11/13/13 09:15:33] Failed to build cache tree[11/13/13 09:15:33] sync-mgr.c(930): [Sync mgr] Failed to commit to repo BBB(87a2fc08).
[11/13/13 09:15:33] sync-mgr.c(496): Repo 'BBB' sync state transition from committing to 'error': 'Failed to index files.'.
[11/13/13 09:15:34] sync-mgr.c(425): Repo 'BBB' sync state transition from 'synchronized' to 'committing'.

The text was updated successfully, but these errors were encountered:

killing · 2013-11-13T10:06:24Z

Unfortunately there is not enough information in the log for debugging the problem. Do you mind running a special debug package to produce more log messages?
The .seafile-data folder is used for cache and will be cleaned up once the sync is done. We'll also reduce the cache space usage in the future.

mathewmeconry · 2013-11-13T12:04:41Z

Sure I do. Seafile is much better then owncloud ;)
Just send me the instructions and I'm glad to help.

By the way I'm using linux mint as client and ubuntu server as server.

killing · 2013-11-15T05:49:07Z

Hi,

The packages are here:
64bit: https://seacloud.cc/f/0a1cf9a6d0/
32bit: https://seacloud.cc/f/2ab7a3ec6a/

Just install the package and send the seafile.log file to me (xjqkilling AT gmail dot com). I need to complete log to analyse the cause.

mathewmeconry · 2013-11-15T07:51:30Z

I sent you an email with the logfiles ccnet and seafile

killing · 2013-11-18T05:37:59Z

Seems the Vm's directory is the cause of problem. Can you tell me what have you done about this directory?

mathewmeconry · 2013-11-18T09:08:04Z

In this directroy are all my vmdk files of the VirtualMachines which I
have. This folder sync only once to the seafile server and is now excluded
by the seafile-ignore.txt
The files look's as shown below.

seafile-ignore.txt:
VM's/

2013/11/18 Jiaqiang Xu notifications@github.com

Seems the Vm's directory is the cause of problem. Can you tell me what
have you done about this directory?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28676527
.

killing · 2013-11-18T09:11:00Z

I just found a bug in handling empty folder and seafile-ignore.txt. Your problem is very likely related to it.
Can you check on the server, whether there are two "VM's" folder?

mathewmeconry · 2013-11-18T09:16:25Z

As you can see the folder is not empty on the sever and on the client.

There is also only one VM's folder on the hole. I only found a folder called VMs without an '.

Server:

Local:

Searchresults for VM in the repositoryfolder local:

killing · 2013-11-18T09:23:31Z

What's the content of seafile-ignore.txt?

killing · 2013-11-18T09:24:22Z

And do you sync between two machine? If so, what's the other client's version?

mathewmeconry · 2013-11-18T09:24:45Z

VM's/

mathewmeconry · 2013-11-18T09:25:23Z

No I only sync with my notebook. I access the files also over seahub and the android app

killing · 2013-11-18T09:31:41Z

But from your log, I can observe the client downloaded from the server, after this download, it started to fail. Can you tell what's downloaded?

mathewmeconry · 2013-11-18T09:34:40Z

It started downloading the VM's folder. Because after I created the seafile-ignore.txt I deleted the VM's folder on the server to safe storage. But after that the client also deleted the folder and so I was pushed move the folder out of trash on the server and starting syncing them down.

But It looks like the sync download is already finished

killing · 2013-11-18T12:01:57Z

I have no luck in reproducing your problem. But it's easy to fix your problem. You just need to unsync the library and sync it again with the same folder.

killing · 2013-11-18T12:04:39Z

Also if you want to delete the VM's folder from the server, you should do that after unsync.

mathewmeconry · 2013-11-18T12:14:39Z

but when I resync the repository the client needs to index the local repository again?

I ask this because it's a huge repository and need over 12 hours or more to index

killing · 2013-11-18T12:19:10Z

Is the biggest part is the VM images? As long as you keep the seafile-ignore.txt file, those images will be not be indexed.

mathewmeconry · 2013-11-18T12:20:46Z

not really. its nearly half half. VM's 70GB and the rest 50GBs.

Another question. Does seafile follow simlinks in the local folder to another folder, which is not synced?

killing · 2013-11-18T12:31:24Z

We can also change the code to fix the problem for you. Then you don't need to unsync.
Yes, Seafile follows symlinks and sync the contents in them.

killing · 2013-11-18T13:07:46Z

Just come up with another easier way to fix your problem.

Move the VM's folder out of this library
After the library get synced, move back the VM's to library
This should get it sync again.

mathewmeconry · 2013-11-18T15:16:28Z

Ok. I resync the library..

thank's for your help

mathewmeconry · 2013-11-19T13:37:40Z

Sorry when I reopen the ticket but resync doesn't help anything.

always when I reindex the folder it fails. Also with another encrypted repo.
When I try this to resync it's successful with indexing files but always fails at uploading.

I sent you already an email attached the log file

thanks for your help

killing · 2013-11-19T13:57:32Z

The log is full of errors like:
Failed to open tmp file /mnt/mscherer/data/.seafile-data/fs/66/57ba26ee1037f61849e108fbad60959aa26fde.EIUN6W Input/output error.
Can you check the folder?

mathewmeconry · 2013-11-19T15:23:06Z

this folder is empty

killing · 2013-11-20T06:33:37Z

It seems to be a problem on /mnt/mscherer. Is it properly mounted? Seems a lot of files in this partition can't be accessed.

mathewmeconry · 2013-11-20T06:52:50Z

yes. it's a ntfs mounted partition. ok creazy. when I try to opwn them it
works
On Nov 20, 2013 7:33 AM, "Jiaqiang Xu" notifications@github.com wrote:

It seems to be a problem on /mnt/mscherer. Is it properly mounted? Seems a
lot of files in this partition can't be accessed.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28866814
.

killing · 2013-11-20T07:01:23Z

It's not recommended to use external storage for Seafile's internal data. You can move the location of the .seafile-data folder by:

Move .seafile-data to an internal drive
Edit ~/.ccnet/seafile.ini to update the location

mathewmeconry · 2013-11-20T07:02:36Z

the problem is that I have not enough space on my ssd
On Nov 20, 2013 8:01 AM, "Jiaqiang Xu" notifications@github.com wrote:

It's not recommended to use external storage for Seafile's internal data.
You can move the location of the .seafile-data folder by:

Move .seafile-data to an internal drive

Edit ~/.ccnet/seafile.ini to update the location

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28867607
.

killing · 2013-11-20T07:06:50Z

Yes I understand.
The .seafile-data folder is just for cache. When you upload a large folder
for the first time, it can take a lot cache space. But after the upload
finishes, and the client restarts, the cache will be cleaned up. You can
then move .seafile-data to your ssd.
This drawback will be fixed next version though.

2013/11/20 mathewmeconry notifications@github.com

the problem is that I have not enough space on my ssd
On Nov 20, 2013 8:01 AM, "Jiaqiang Xu" notifications@github.com wrote:

It's not recommended to use external storage for Seafile's internal
data.
You can move the location of the .seafile-data folder by:

Move .seafile-data to an internal drive

Edit ~/.ccnet/seafile.ini to update the location

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867607>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28867649
.

mathewmeconry · 2013-11-20T07:08:10Z

the ssd has not enough space fpr the first upload
On Nov 20, 2013 8:07 AM, "Jiaqiang Xu" notifications@github.com wrote:

Yes I understand.
The .seafile-data folder is just for cache. When you upload a large folder
for the first time, it can take a lot cache space. But after the upload
finishes, and the client restarts, the cache will be cleaned up. You can
then move .seafile-data to your ssd.
This drawback will be fixed next version though.

2013/11/20 mathewmeconry notifications@github.com

the problem is that I have not enough space on my ssd
On Nov 20, 2013 8:01 AM, "Jiaqiang Xu" notifications@github.com
wrote:

It's not recommended to use external storage for Seafile's internal
data.
You can move the location of the .seafile-data folder by:

Move .seafile-data to an internal drive

Edit ~/.ccnet/seafile.ini to update the location

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867607>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867649>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28867765
.

killing · 2013-11-20T07:13:32Z

I mean you still use NFS partition for the first upload. And then move back
to the ssd later.

2013/11/20 mathewmeconry notifications@github.com

the ssd has not enough space fpr the first upload
On Nov 20, 2013 8:07 AM, "Jiaqiang Xu" notifications@github.com wrote:

Yes I understand.
The .seafile-data folder is just for cache. When you upload a large
folder
for the first time, it can take a lot cache space. But after the upload
finishes, and the client restarts, the cache will be cleaned up. You can
then move .seafile-data to your ssd.
This drawback will be fixed next version though.

2013/11/20 mathewmeconry notifications@github.com

the problem is that I have not enough space on my ssd
On Nov 20, 2013 8:01 AM, "Jiaqiang Xu" notifications@github.com
wrote:

It's not recommended to use external storage for Seafile's internal
data.
You can move the location of the .seafile-data folder by:

Move .seafile-data to an internal drive

Edit ~/.ccnet/seafile.ini to update the location

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867607>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867649>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867765>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28867803
.

mathewmeconry · 2013-11-20T07:38:11Z

ok. but during the first upload the client fails ...

killing · 2013-11-20T14:20:05Z

Try unsync first.

mathewmeconry · 2013-11-20T16:57:58Z

already unsync but now unable to sync the repository again
On Nov 20, 2013 8:13 AM, "Jiaqiang Xu" notifications@github.com wrote:

I mean you still use NFS partition for the first upload. And then move
back
to the ssd later.

2013/11/20 mathewmeconry notifications@github.com

the ssd has not enough space fpr the first upload
On Nov 20, 2013 8:07 AM, "Jiaqiang Xu" notifications@github.com
wrote:

Yes I understand.
The .seafile-data folder is just for cache. When you upload a large
folder
for the first time, it can take a lot cache space. But after the
upload
finishes, and the client restarts, the cache will be cleaned up. You
can
then move .seafile-data to your ssd.
This drawback will be fixed next version though.

2013/11/20 mathewmeconry notifications@github.com

the problem is that I have not enough space on my ssd
On Nov 20, 2013 8:01 AM, "Jiaqiang Xu" notifications@github.com
wrote:

It's not recommended to use external storage for Seafile's
internal
data.
You can move the location of the .seafile-data folder by:

Move .seafile-data to an internal drive

Edit ~/.ccnet/seafile.ini to update the location

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867607>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867649>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867765>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-28867803>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-28867976
.

killing · 2013-11-21T12:08:28Z

What does the log say when it fails?
Perhaps an easier way for you to sync is to create multiple libraries for your large set of data. Keep each library under an acceptable size for your ssd size. This way you can place your .seafile-data folder in your internal drive.

pfefffer · 2013-12-29T12:35:20Z

Hi!

killing wrote
"It's not recommended to use external storage for Seafile's internal data."
What ist the reason for this?
Is this a bug in Linux (not mounting everything correctly)?
Would it be ok to have all - including the server software - on an external storage?
Or is it a special problem using NTFS in Linux?

Thanks!
Pfeffer.

mathewmeconry · 2013-12-29T13:11:08Z

Hi

The server (ubtuntu server) application and the server data are on the same
partition and in the same directory.

On the client (linux mint) are some repositories and the seafile-data
folder on a mounted ntfs partiton.

Greez
Mathew
On Dec 29, 2013 1:35 PM, "pfefffer" notifications@github.com wrote:

Hi!

killing wrote
"It's not recommended to use external storage for Seafile's internal data."
What ist the reason for this?
Is this a bug in Linux (not mounting everything correctly)?
Would it be ok to have all - including the server software - on an
external storage?
Or is it a special problem using NTFS in Linux?

Thanks!
Pfeffer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-31316241
.

pfefffer · 2013-12-29T15:31:59Z

ahh - the problem was not caused on the server side but on client side?

mathewmeconry · 2013-12-29T15:35:59Z

I don't know really.
The issue is that the client doesn't sync. Up- and download
On Dec 29, 2013 4:32 PM, "pfefffer" notifications@github.com wrote:

ahh - the problem was not caused on the server side but on client side?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-31318959
.

pfefffer · 2013-12-30T00:39:53Z

but you said when you "try to opwn them it works".
What do you exactly mean by "open them"?
Is it enough to "cat" some file from that partition before starting the client?

killing · 2013-12-30T02:16:47Z

The problem, IIRC, is that he placed the Seafile folder in a network
partition. Once the network is disconnected, Seafile fails to sync files.

2013/12/30 pfefffer notifications@github.com

but you said when you "try to opwn them it works".
What do you exactly mean by "open them"?
Is it enough to "cat" some file from that partition before starting the
client?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-31329537
.

mathewmeconry · 2013-12-30T09:05:00Z

Okey guys. I'm happy that you help me.

With open files I mean I am able before and during the sync to open files (
yes with cat I'm also able to open files ) on the partition.

There is no network folder nothing. Just a ntfs partition mounted into my
linux mint client machine
On Dec 30, 2013 3:17 AM, "Jiaqiang Xu" notifications@github.com wrote:

The problem, IIRC, is that he placed the Seafile folder in a network
partition. Once the network is disconnected, Seafile fails to sync files.

2013/12/30 pfefffer notifications@github.com

but you said when you "try to opwn them it works".
What do you exactly mean by "open them"?
Is it enough to "cat" some file from that partition before starting the
client?

—
Reply to this email directly or view it on GitHub<
https://github.com/haiwen/seafile/issues/440#issuecomment-31329537>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/440#issuecomment-31331063
.

mathewmeconry · 2014-01-21T10:54:51Z

So now the issue is fixed:

Reinstalled server and renewed the portforwardings on the router.
Initial sync is now running fine and I hope it should work also in the the future.

Many thanks for your help and keep going

Good software ;)

mathewmeconry closed this as completed Nov 18, 2013

mathewmeconry reopened this Nov 19, 2013

mathewmeconry closed this as completed Jan 21, 2014

Syncing large repository #440

Syncing large repository #440

Comments

mathewmeconry commented Nov 13, 2013

killing commented Nov 13, 2013

mathewmeconry commented Nov 13, 2013

killing commented Nov 15, 2013

mathewmeconry commented Nov 15, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

killing commented Nov 18, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

killing commented Nov 18, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

killing commented Nov 18, 2013

killing commented Nov 18, 2013

mathewmeconry commented Nov 18, 2013

mathewmeconry commented Nov 19, 2013

killing commented Nov 19, 2013

mathewmeconry commented Nov 19, 2013

killing commented Nov 20, 2013

mathewmeconry commented Nov 20, 2013

killing commented Nov 20, 2013

mathewmeconry commented Nov 20, 2013

killing commented Nov 20, 2013

mathewmeconry commented Nov 20, 2013

killing commented Nov 20, 2013

mathewmeconry commented Nov 20, 2013

killing commented Nov 20, 2013

mathewmeconry commented Nov 20, 2013

killing commented Nov 21, 2013

pfefffer commented Dec 29, 2013

mathewmeconry commented Dec 29, 2013

pfefffer commented Dec 29, 2013

mathewmeconry commented Dec 29, 2013

pfefffer commented Dec 30, 2013

killing commented Dec 30, 2013

mathewmeconry commented Dec 30, 2013

mathewmeconry commented Jan 21, 2014