Syncing fails on an Android/termux as os.link is not available

[michitux]

Vdirsyncer fails syncing on Android in termux due to os.link not being available in the python build of termux/Android (see termux/termux-packages#29 for details). os.link is used in the atomicwrites library as it can be seen in the traceback below:

Copying (uploading) item e9213c2c-7cf4-4728-b1c4-45781d92267f to my_calendar_local/contact_birthdays
error: Unknown error occured for my_calendar/contact_birthdays: module 'os' has no attribute 'link'
error: Use `-vdebug` to see the full traceback.
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/cli/utils.py", line 74, in handle_cli_error
debug:     raise e
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/sync/__init__.py", line 153, in sync
debug:     partial_sync
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/sync/__init__.py", line 177, in run
debug:     self._run_impl(a, b)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/sync/__init__.py", line 204, in _run_impl
debug:     href, etag = self.dest.storage.upload(self.item)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/storage/base.py", line 15, in inner
debug:     return f(self, *args, **kwargs)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/storage/filesystem.py", line 126, in upload
debug:     fpath, etag = self._upload_impl(item, href)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/storage/filesystem.py", line 149, in _upload_impl
debug:     return fpath, get_etag_from_file(f)
debug:   File "/data/data/com.termux/files/usr/lib/python3.6/contextlib.py", line 88, in __exit__
debug:     next(self.gen)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/atomicwrites/__init__.py", line 152, in _open
debug:     self.commit(f)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/atomicwrites/__init__.py", line 179, in commit
debug:     move_atomic(f.name, self._path)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/atomicwrites/__init__.py", line 101, in move_atomic
debug:     return _move_atomic(src, dst)
debug:   File "/data/data/com.termux/files/home/.local/venvs/vdirsyncer/lib/python3.6/site-packages/atomicwrites/__init__.py", line 49, in _move_atomic
debug:     os.link(src, dst)

I'm not sure if there is a proper solution for this or if this simply means vdirsyncer cannot be used on Android. If the latter is the case it would be nice if vdirsyncer could fail early and wouldn't try syncing every single item.

[untitaker]

I don't understand what you are trying to do by using vdirsyncer is this setup. Could you elaborate?

Filesystem support for Android seems impossible to do safely, so I'd rather have people avoid such a setup in the first place.

[untitaker]

@michitux any update on this?

[michitux]

Sorry for the late reply. My main goal was to backup the contents from one nextcloud instance and then I wanted to sync with another nextcloud instance. I wanted to do this on my phone in termux as this was what I had at hand. In the meantime I did the same (successfully) on my laptop. I guess using vdirsyncer under termux on a real Android phone as I did is not really useful apart from very special cases. However, on a tablet or Chromebook with a keyboard it might make more sense.

Concerning the filesystem: this is the internal /data directory, on my phone this is ext4. It's just that hard links have been disabled. Just out of curiosity I have also tried the master breach - it compiles but fails then probably for the same reason (the traceback is not that clear here).

As I have said I can fully understand if you don't want to support this but then an earlier warning/error message would be nice.

[untitaker]

I suppose you get something like "permission denied"? I am not sure how to test that beforehand. But we can definetly make the error message nicer.

[michitux]

I've just noticed that somehow the linebreaks in the traceback above were completely lost. I've just tried restoring them, now the error should be easier to read I hope. In Python the check seems to be quite simple: test if os.link exists. If it doesn't exist, then support for (hard)links is missing. Note that I'm getting the error posted above once for every entry in the address book and calendar. With the rust version the error is Permission denied (os error 13). Again, I get this error once for every entry.

[untitaker]

I agree that a preliminary check in Python would be very simple, but my plan is to eventually port the entire codebase to Rust due to various reasons. Outside of Python I have nothing to go by other than the "permission denied" error. Bailing out early after the first error is generally undesirable for storage actions ("Copying to ..", "Deleting ..."), but I might have to do that for just IO errors and nothing else.

[dnastase]

Is it there an update on this ?
Is it stated somewhere that if hard linking is not supported then the function should just be missing entirely (ie: as opposed to just raising an exception) ?

[untitaker]

@dnastase no that's also part of the issue. As far as I can tell this Python port just violates the lang spec