3717 om supply central backup kdd #3924
Conversation
github-actions
bot
added
enhancement
There was a problem hiding this comment.
very nice! thanks @andreievg that's great.
have popped in some spelling nits that stood out while reading - while it's a bit late in the day to comment sensibly, I'm leaning to option 3. I like the idea of being able to configure backup schedule within oms, and it's a straightforward option for support
| ### (B) Other areas of consideration | ||
|
|
||
| 1. When data is stored remotely, it needs to be safe from prying eye (encrypted) | ||
| 2. We also have Grafana installed on most central instances, and I think we've been using the same postgres instance for both omSupply central and Grafana |
There was a problem hiding this comment.
Note that the grafana.db SQLite database is generally backed up along with the mSupply backup - added into the additional files and bundled with
andreievg
changed the title
I agree it's nice to build a solution that doesn't rely on mSupply. |
| _Pros:_ | ||
|
|
||
| - Should be reasonably simple | ||
| - A.3 is handled automatically (using the same mechanism as mSupply) |
There was a problem hiding this comment.
I thought this would fail A.3 as unless the backup happens at the exact same time as mSupply and pauses activity on both MC and OMC, otherwise the cloud backups (A.3) aren't guaranteed to be in sync with each other?
| |:---------:|:--------------------:|:----------------------:|:---------------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | B.3.1 | old | recent | recent | Loss of data while restore of OMR data (via sync initialisation), due to missing data for that site on MC. OMC and OMR may miss some central data being added, since their cursor for that data would be ahead of central_change_log on MC. Some OMR remote records that are pushed to OMC may be related to central records on MC that are not present anymore, thus when OMR is re-initialised some constraints may fail. | | ||
| | B.3.2 | recent | old | recent | Loss of data while restore of OMR (via sync initialisation), due to missing data for that site on OMC. Some data being pushed OMC from OMR may not be integrated due to missing relation dependencies (that were pushed previously but are now missing). Some new OMC central data may not travel to OMR since the cursor of OMR for central data would be ahead of current OMC | | ||
| | B.3.3 | recent | recent | old | This would require some restore of backup on omSupply remote site, which is not something we recommend or support (if remote site data is lost, you should re-initialise) | |
There was a problem hiding this comment.
I have one key question - isn't B.3.3 essentially what we're trying to achieve? That's basically what we rely on for current MC prior to OMS2.0? But the complication I see is: don't we require both MC and OMC to backed up together at precisely the same time? Otherwise we're constantly going to end up with B.3.1 and B.3.2? At least for the cloud files anyway
Currently MC pauses all processes while it executes the backup process, including sync APIs. Would we not need to do the same with OMC at the same time?
Co-authored-by: Mark Prins <9192912+mark-prins@users.noreply.github.com>
Co-authored-by: Mark Prins <9192912+mark-prins@users.noreply.github.com>
Co-authored-by: Mark Prins <9192912+mark-prins@users.noreply.github.com>
Co-authored-by: Mark Prins <9192912+mark-prins@users.noreply.github.com>
Co-authored-by: Mark Prins <9192912+mark-prins@users.noreply.github.com>
Co-authored-by: Mark Prins <9192912+mark-prins@users.noreply.github.com>
Fixes #3717
Didn't really deal with B.3 use cases. Started thinking about how cursors on both sides can help re-integrate data from remote to central etc, but realised it's a bit of a rabbit hole, so I didn't want to get too deep into it. I did want to outline some problems that result with backup recency mismatch.
Also do I need to get into more details about the actual backup and restore operations for postgres ?