Entity follow up work

[seadowg]

Release blockers (prioritised):

• Make sure that entity lists are consistent between follow up forms #6029
• Prevent updates to entities that don't appear in entity list #6031
• Delete local entities that have been deleted/archived/unassigned from server #6011
• Allow entities to be deleted/cleared locally #6012
• Optimize load times and memory footprints of form entry with datasets #5623
• There's most likely forms that don't meet the spec that can be filled (create without label for instance) or will cause the app to crash

To improve:

• No log of changes to entities
• Datasets are downloaded and stored for every form
• Online entities are not shown in browser
• Form loading takes longer for entity follow up forms (because of building combined instance) than normal secondary instance forms
  • The is quite severe due to n^2 lookups required for deduplicating
• Collect should not attempt to redownload media file if it has the same hash as the last download #6030
• Send submissions in oldest first order #6128
• Entities can be loaded half way through being updated (need to lock on form load and mutations OR mutate it transactionally)

[lognaturel]

Datasets are downloaded and stored for every form

Should this consideration be part of #6029?

Currently I think the only case that a download would happen for multiple forms is if an entity list changed part way through a match exactly sync. Otherwise, there would be a server request, but the server would immediately respond with a 304 and the entity list would be returned from cache. But yes, each entity list is stored once for each form.

[seadowg]

Should this consideration be part of #6029?

I think it probably gets ticked off as part of that, but it's a different problem so I'd like to keep it called out separately.