Need to continuosly restart the sync server to cope with different errors

[polettix]

Hi! I have a setup with many moving parts: one instance of taskwarrior in Linux for my use, one "read only" instance to send notificatios via ntfy, taskdroid on the phone, and taskchampion -sync-server with Postgresql backend behind nginx to sync across them all. The sync server is running in a Docker container orchestrated by Dokku, which also takes care of the Postgresql instance.

The whole thing has been setup in the past two-three days so everything is up to date.

I consistently and continuosly end up in one or several clients complaining about not being able to sync, e.g. after I add/modify/mark tasks i taskdroid it refuses to sync with an error (the log is not too interesting as I think it's the local component name). This happens in the Linux instance too, again after doing some modifications, this time either with error 410 "Gone" (what is gone?) or sometimes with error 500 (associated to some logs in the sync server related to the database).

nginx logs show those error codes sent back from the sync server, plus error code 409 (still from the sync server) when I try to sync from taskdroid.

I assume it's a taskchampion-sync-server issue because if I restart it then all clients are able to sync again, whatever the problem (409, 410, 500, etc.). Until... the next time, I mean. It's a bit strange because HTTP errors 4XX usually indicate some error on the client side, so it's puzzling that they are solved with an action on the server side.

Do you have any suggestions for collecting logs around that can actually be of some help with this "distributed" problem? I'd like to avoid restarting the sync server every 10 minutes or so.

Thanks!

[djmitche]

Can you say more about what the sync errors say?

When it is working, are all replicas able to sync with one another? That is, does it work at all? My suspicion would be that there is some history divergence between these replicas and whichever one talks to the sync server first gets to push its changes, then everything else gets errors.

When you restart the server, are you also erasing the postgres database?

Any logs would be useful. Also looking at select client_id, version_id, parent_version_id from versions and select client_id, latest_version_id on the server and select * from sync_meta on the clients.

Have a look at https://gothenburgbitfactory.org/taskchampion/sync.html for the model in use here. Basically the server should have a single chain of versoins leading to the latest_version_id, and the replicas should have a base_version somewhere on that chain.

410 means that the replica requested a version that doesn't exist. That suggests its base_version is pointing to a different change from the one on the server.

409 can happen when a replica uploads changes but another replica has already uploaded changes. In this case the replica "rebases" onto those already-uploaded changes and tries again.

500's suggest an error in the database.