Sync patient data from server to user

[ssrihari]

• Get new/updated patients from server
• Save the timestamp of the latest record synced with server
• Log validation errors
• Mark patients with validation errors as invalid
• Finish unit tests
• Save phone numbers

We need to think this through
We need to batch records that need to be sent from server to user. Do we use pagination mechanisms? How do we define last_sync_at? Should we record server_updated_at in addition to device_updated_at?

[ssrihari]

Further thoughts:

• We should store updated_on_server_at during the POST request, so that we can compute the diff of records to send to each device during the sync.
• Sync all records first time. First time syncing should be indicated by absence of updated_on_server_at or a more specific field in the request.
• We should separate the notions of request and response timestamps from updated_at timestamps of the records. To compute the diff, we should only ever use timestamps of records, not request/response timestamps.
• At the end of the GET request, we should compute the latest_record_timestamp, and send that to the device with the response. The device can send this value with the next request to receive the next batch of records. This makes the pulls batch-able as a stream of independent requests, without needing to keep a track of when the sync 'ends'. Do not use last_sync_at for this reason.

Things the device/front-end should do:

• Devices should store latest_record_updated_at per API endpoint.
• Devices should send a value denoting the number of records, or batch_size they want. This can change per request.
• Devices should push before they pull

Things we're not doing right now:

• If we want to send back only the diff of changed records per user, we should store last_updated_by_user_id
• If we use last_sync_at, it should be set at the beginning of the GET request, not the end.
• We could include a margin of error in the time range we use to retrieve records to sync, to account for some race conditions or time differences across devices/servers.

Patterns for office mobile syncing architecture: https://www.dre.vanderbilt.edu/~schmidt/PDF/PatternPaperv11.pdf

[ssrihari]

@timcheadle: @govindkrjoshi and I are in agreement about the above. Thoughts?