Consistent snapshots for unlocked UUIDs

[stuarteberg]

To update the body column for a snapshot, we currently need the following:

• A cached the synapse point table, taken from an earlier UUID
  • including rows for sv and body
• A copy of the mutation log (to determine which sv entries from the cache are out-of-date)
• A copy of the complete in-memory map (to update the body column based on the sv column)

For locked UUIDs, there is no issue. But for unlocked UUIDs, there are race conditions at play.

Since the mutation log and the in-memory map both take several minutes to download, there's a good chance that there will be ongoing mutations while they're downloading. That means they may not be consistent with each other. In fact, the downloaded in-memory-map might be inconsistent with itself, since we don't use .../mappings?consistent=true.

Options:

• To at least obtain consistent mappings, I could use consistent=true
• I could also minimize the time between fetching the mappings and fetching the mutations if I carefully fetched the last UUID of mutations first, and fetch the rest of the dag afterward.
  • Or I could rewrite fetch_bodies_for_many_points() to use DVID's built endpoint for fetching supervoxel split IDs, which is very fast but doesn't cover the whole DAG. (In most cases, that doesn't matter.)
• To obtain guaranteed consistency between the mapping and the mutation log, I would have to start with the mappings from the most recent locked UUID and then "play forward" the mutations from the mutation log. That is possible, but an efficient implementation would not be trivial to write.
• Maybe a "simple" option for consistent mappings. This will result in a fully consistent mapping, but it doesn't let you pick the exact timepoint you'll end up with.
  • Record the current time.
  • Fetch the complete mappings from DVID (with consistent=true)
  • Fetch recent mutations. Have any mutations occurred since we started fetching mappings?
  • [loop head] If so, fetch the supervoxels for the affected bodies, and update the mapping only for those bodies.
  • Now fetch recent mutations again. Have any mutaitons occurred since we updated the mappings?
  • jump to [loop head]

Another issue is that the point-annotations (for soma points, etc.) also rely on consistent labelmaps, which adds an additional opportunity for inconsistencies to arise. Probably the right thing to do is to fetch their supervoxels immediately before or after fetching the mapping, and update them in the same way that synapses are updated.