feat(datasource/graphene) auto reloading graphene manifest after an edit

[chrisj]

Replaces #450

With our current infrastructure, the only way for us to get mesh updates is to continuously poll the manifest endpoint to see if it changes, we can't rely on 404 responses as previously thought. Eventually we want to get push messages on the client to replace this logic.

I was asked to try to reduce the flicker that occurs when we change the manifest of an existing cell. I have partial "fix" with this commit 80d95e1. The issue is that the ManifestChunk arrives before the first FragmentChunk.

The easiest solution that mostly works is to prevent ManifestChunks from triggering redraws. I was surprised that ManfestChunk gets a GPU_MEMORY state. It still can cause issues.

Do you have any suggestions on how we could make it smoother. I was asked to go even further by keeping existing fragments from previous cells around until the new cell is at least partially meshed. For example, with a merge, it is very easy to just combine the fragments from each segment.

[fcollman]

@jbms any reason we can't get this one merged?

[jbms]

This should somehow ensure that it is associated with this graph --- can have more than one datasource in a UserLayer.

[chrisj]

Would this solve that? It looks like I could either do loadState.dataSource.subsources or loadState.subsources

getMeshSource() {
    const {layer} = this;
    for (const dataSource of layer.dataSources) {
      const {loadState} = dataSource;
      if (loadState instanceof LoadedLayerDataSource) {
        const {subsources} = loadState.dataSource;
        const graphSubsource = subsources.filter(subsource => subsource.id === 'graph')[0];
        if (graphSubsource?.subsource.segmentationGraph !== this.graph) {
          continue;
        }
        const meshSubsource = subsources.filter(subsource => subsource.id === 'mesh')[0];
        if (meshSubsource) {
          return meshSubsource.subsource.mesh;
        }
      }
    }
    return undefined;
  }

[jbms]

This shouldn't be done by name, and also skipping the notification doesn't really fix the problem in general, because any other chunk update would also trigger the notification. Additionally it will still redraw on the first fragment that is received so there will still be flicker if there is more than one fragment. I think a different strategy is needed to avoid flicker.

Will some of the existing fragments be referenced by the new manifest, if they haven't changed, or does the fragment key depend on the root id?

[jbms]

Sorry for the delay in reviewing, I added some comments.

[chrisj]

@jbms I added a response about getMeshSource

I will remove the attempt at fixing flicker until we have a good solution

[chrisj]

@jbms I pushed the change I talked about and removed the flicker reduction code so it should be ready to be merged.