Repository navigation
Allow sync to single ics file (per calendar) #48
Description
Activity
Does orage use a single file as a storage format?
It's rather unlikely that i will want to support this. The hack used for supporting single-file HTTP resources is awful enough.
Orage uses only one own calendar in one ics file, but it allows to add "remote" files (on local FS) and thus allows to use more calendars (and ics files as storage) at once. The remote files can be additionally marked as RO, but can be RW too. You can read more here http://www.kolumbus.fi/~w408237/orage/orage_doc_4_7.html#exchange-foreign-tab
IMO if you have one-file support, it can be simplest, in mean that there is not needing to start from scratch. I am not experienced Pythonist, but merging seems to be simple task - i did some initial playing with the icalendar module. There can be any VEVENT/VTODO/VJOURNAL simple added into one calendar object instead putting into separate file. If you have HTTP one file implemented, thus this must be the same, only by opposed direction.
The simplest way seems to be to store internally in vdir as usual and add some pre/posprocessing to achieve synchronization of the separate file as result and then reuse the most of existing code. But i cannot help, then it is your decision.
The icalendar module is not as useful as it might seem, as simply extracting events, tasks and journal components is not enough -- timezones that appear outside of them have to be copied into each of them. Furthermore, starting to depend on icalendar would only solve the calendar stuff, addressbooks are obviously not parsable with icalendar, which means that we still have to maintain the custom hack.
I don't have a personal motivation to implement this, but everybody is welcome to try it themselves. A proper implementation would have its own module in
vdirsyncer.storage, subclassvdirsyncer.storage.base.Storageand usesplit_collectionto load the file and yet another hack to save the file back. I'd be delighted if somebody finds a way of not loading the whole file into memory, but i don't think anybody cares to try that hard.This is actually a lot easier than i thought since one can use
icalendar.cal.Componentto parse anything vobject-like, even if it's a vcf.Btw you could use radicale additionally to vdirsyncer for now. It is a caldav server which stores its stuff in a single ics file.
For now i have my own radicale on my own server, with ssh access, then i use rsync to sync from server to local machine (only one way) and this synced .ics as RO remote file in Orage. Do you mean to use two radicale servers with vdirsyncer between them?
I wasn't aware that you already use radicale on the server side. In that case your approach works fine. You might want to check out Unison, which is like rsync but syncs in both directions. However, it can't handle the case where the file changes on both sides -- which is why a local radicale installation might still be helpful to let vdirsyncer figure out the conflict resolution. But for now i'd just swap rsync for unison, i am going to try to implement your feature request in the near future i think.
Thanks for willingness to implement this – that will be one small step for man; but one giant leap for XFCE's mankind :-)
I will learn more about unison and try differences, thanks to point to it
Can you try the
singlefilestoragebranch?pip install --user --upgrade --force-reinstall git://github.com/untitaker/vdirsyncer.git#singlefilestorageOr
git clone git://github.com/untitaker/vdirsyncer.git git checkout singlefilestorage pip install --user --upgrade --force-reinstall .Eh, i should actually make this usable from the CLI...
Okay, now. Usage:
[storage foo] path = ~/path/to/file.icsOK, tested it and it seems as working. I was success to modify the (one) VTODO and then sync it back to the server. I encouter one small error while testing (and trying), i can post more detailed report about configuration - there are more changes required in storage configuration (i did by try and fail).
I have only one suggestion now – it seems, that while syncing back to the server there was synced all calendar components, not only the one which was changed – is it intended behavior? The "LAST-MODIFIED" property was changed only in one item (as expected), but for any item there was a line:
Copying (updating) item 86359880-7eb1-4d71-a2fd-8c2de5bf9a4d to <CaldavStorage(**{'username': 'xxx', 'url': 'https://calend.slavino.sk/xxx/hlavny.ics/That is interesting... I'll try and see if i can reproduce this error.
I don't seem to be able to reproduce this error. It might be that icalendar does some inappropriate reformatting which leads to this at the beginning, but there is little i can do about it.
24 remaining items
Eh, scratch that, i found it.
Okay, this is one of the last shots... it really should be working now.
OK, the second problem solved! Thanks! Now are both b's etags always the same. Now the a's etag lefts. I tried three situations:
1, I tried from scratch (deleted all local things) and do first sync, immediate after this i did second sync and all local etag differs, then all items are uploaded to remote. All next syncs (without local or remote changes) are not uploaded, nor to remote nor to local.
2, Then i itry do remote change (via thunderbird). The situation repeats – remote changes is updated to local, next sync it is updated to remote and all next syncs no update happen.
3, At last i tried local change (via txt editor) – it is uploaded to remote and all next syncs no update heappens.It seems, that there is small problem with initial creating of the etag, when item is synced to(created in) singlefile, but this is OK after syncing (upload) to remote. Latter i will try it with Orage...
I tried the local file with Orage, where i modified one item – all works as expected! I was worrying, if Oroge will not do more changes, but not – only one item was uploaded to server! I am happy!
Then only previous mentioned problem persists, when changes are downloaded from server, they are always first time uploaded back to server. It seems as harmlessly, but if the same item will be changed in remote 2 (or more) times between syncing, there can conflict happen... Eh, i am not able to describe it better in English, i hope that it is clear for you.
There is no way for vdirsyncer to tell how often a item changed on one side since the last sync, so i can't imagine how changing the item twice would lead to problems... note that changing an item and then reverting the changes still counts as a change, and i can do nothing about it.
Could you give me another step by step tutorial on how to reproduce this?
Sure, there are two situation, when this happens. First new storage:
1, setup new singlefile storage
2, do initial sync – all item are uploaded to local file
3, do next sync – all items are uploaded back to server
4, all next sync (without items change) are not uploaded by any directionThe same happens, when remote item is changed:
0, have synced singlefile storage
1, do some change in remote item
2, do first sync – the changed item is uploaded to local
3, do next sync – the same item is uploaded to remote
4, all next sync (without items change) are without upload by any directionBecause i have enabled the
a['etag']andstatus_etag_aprinting, i see that they differs in both step 2, but they are identical in step 3 in both above described situations.The problem can happen in this situation:
1, do change in remote storage
2, do sync – the change is uploaded to local
3, do another change in the same remote item
4, do sync – because above mentioned behavior, the vdirsyncer can think, that the change happens on both ends, in the remote (real) and in the local (above behavior) too, then there can be conflict.Now it should FINALLY WORK.
Yes, i can confirm. I tested all previous situations and there are not needed uploads. Thanks!
Great!
Thanks for patient and useful collaboration!
please, add check and exception handling, if the path (directories) to local file doesn't exists.
Should work now.
I consider this case closed. From now on, please file new issues for everything.
Please, add option to sync into one (single) ics file, which then can be used as foreign file for Orage (the XFCE's light calendar) - there is a lack of CalDAV clients in Linux, if one don't want to use Thunderbird or Evolution, then working with shared calendars can be problematic ;)
I think initialy about merging synced ics files into one and this seems to be trivial task, but split them back can be more trickly...