MAPSJS-2983 Write to the console if we need too many attempts getting… #2284

nzjony · 2021-08-19T14:50:42Z

… the image from the cache

Some failed attempts is ok, because there are cases where the DataSource is attached and ready
and the map has started to render, but the theme isn't fully set.

There are other ways to fix the problem, but they are generally more invasive and could introduce
regressions, some options are:

In the TileGeometryCreator::createAllGeometries method, await on the getTheme() Promise
Don't allow new TextElements to be added while the theme is updating
Investigate and remove all calls to update the mapview while the method addDataSource is called
and see if this can be removed, because no update calls means we don't render and request new tiles
which may contain references to POI's that don't yet exist on the cache.

But, for the sake of simplicity, this is the preferred method.

@here/harp-mapview/lib/poi/PoiRenderer.ts

codecov · 2021-08-20T10:05:58Z

Codecov Report

Merging #2284 (715bad5) into master (fbba18f) will increase coverage by 0.30%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##           master    #2284      +/-   ##
==========================================
+ Coverage   67.95%   68.25%   +0.30%     
==========================================
  Files         315      315              
  Lines       27766    27763       -3     
  Branches     6215     6214       -1     
==========================================
+ Hits        18869    18951      +82     
+ Misses       8897     8812      -85

Impacted Files	Coverage Δ
@here/harp-mapview/lib/poi/PoiRenderer.ts	`72.59% <83.33%> (+29.36%)`	⬆️
@here/harp-mapview/lib/poi/PoiManager.ts	`14.28% <0.00%> (+1.42%)`	⬆️
@here/harp-mapview/lib/image/MipMapGenerator.ts	`95.91% <0.00%> (+2.04%)`	⬆️
@here/harp-mapview/lib/image/Image.ts	`93.33% <0.00%> (+2.22%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fbba18f...715bad5. Read the comment docs.

@here/harp-mapview/lib/poi/PoiRenderer.ts

atomicsulfate · 2021-08-23T08:30:57Z

@here/harp-mapview/lib/poi/PoiRenderer.ts

+    // There is a texture missing in the cache, we attempt again, and then error out.
+    const missingTextureCount = missingTextureName.get(imageTextureName);
+    missingTextureName.set(imageTextureName, missingTextureCount ? missingTextureCount + 1 : 0);
+    if (missingTextureName.get(imageTextureName)! >= SEARCH_CACHE_ATTEMPTS) {


We could log the error just once to avoid cluttering the console. Not sure if it makes sense to do it after n attempts or at the first one. It's difficult to say when "n attempts" is enough, cause it depends on how many POIs use that same image.

I would argue that if there is an error after 5 attempts, there is something fundamentally wrong and a new approach should be taken (like the ones I mentioned in the commit message). At some point, trying to present the error in a nice way is counter productive.

In saying that, I can reduce cluttering the console by changing the >= to be just =, then we avoid any complex state management.

@here/harp-mapview/test/PoiBatchRegistryTest.ts

atomicsulfate

I left some comments.

atomicsulfate · 2021-08-23T12:08:45Z

@here/harp-mapview/lib/poi/PoiRenderer.ts

@@ -396,11 +396,10 @@ function findImageItem(
    // There is a texture missing in the cache, we attempt again, and then error out.
    const missingTextureCount = missingTextureName.get(imageTextureName);
    missingTextureName.set(imageTextureName, missingTextureCount ? missingTextureCount + 1 : 0);
-    if (missingTextureName.get(imageTextureName)! >= SEARCH_CACHE_ATTEMPTS) {
+    if (missingTextureName.get(imageTextureName)! === SEARCH_CACHE_ATTEMPTS) {


minor: we could do missingTextureName.get(imageTextureName) only once.

Yes, but I would need to then do the +1 twice. What is preferred?

atomicsulfate · 2021-08-23T12:11:19Z

@here/harp-mapview/test/PoiRendererTest.ts

+        const webGLRenderer = {
+            capabilities: {
+                isWebGL2: false
+            }
+        } as THREE.WebGLRenderer;
+        const mapViewStub = sinon.createStubInstance(MapView);
+        const poiManager = new PoiManager((mapViewStub as any) as MapView);
+        const testImageName = "testImage";
+        const mapEnv = new Env();


move this to a beforeEach() function. Each test should start with a clean slate.

Yes, the MapEnv is actually never used, but I will do it to make the test at least easy to extend in future.

atomicsulfate · 2021-08-23T12:18:16Z

@here/harp-mapview/test/PoiRendererTest.ts

+        ) => {
+            // Note, the images must be unique, otherwise the test will fail, because the internal
+            // caching mechanism of the ImageCache will have already loaded the image.
+            return testCache.addImage(name, `https://picsum.photos/${x++}/${y++}`, load);


Unit tests must not have external dependencies. We can instead stub testCache's findImageByName method and make it return what we want whenever we want in the test.

Done, I could use a local resource so I don't need to stub.

… the image from the cache Some failed attempts are ok, because there are cases where the DataSource is attached and ready and the map has started to render, but the theme isn't fully set. There are other ways to fix the problem, but they are generally more invasive and could introduce regressions / performance issues, some options are: - In the TileGeometryCreator::createAllGeometries method, await on the getTheme() Promise - Don't allow new TextElements to be added while the theme is updating - Investigate and remove all calls to update the mapview while the method `addDataSource` is called and see if this can be removed, because no update calls means we don't render and request new tiles which may contain references to POI's that don't yet exist on the cache. For the sake of simplicity, this is the preferred method. Change-Id: I441a0ff5f63fd3d2a403b43b422c3044c902c7a6 Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

…source Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

#2284) Some failed attempts are ok, because there are cases where the DataSource is attached and ready and the map has started to render, but the theme isn't fully set. There are other ways to fix the problem, but they are generally more invasive and could introduce regressions / performance issues, some options are: - In the TileGeometryCreator::createAllGeometries method, await on the getTheme() Promise - Don't allow new TextElements to be added while the theme is updating - Investigate and remove all calls to update the mapview while the method `addDataSource` is called and see if this can be removed, because no update calls means we don't render and request new tiles which may contain references to POI's that don't yet exist on the cache. For the sake of simplicity, this is the preferred method. Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com> Signed-off-by: Fischer, Thomas <thomas.fischer@here.com>

nzjony requested a review from atomicsulfate August 19, 2021 14:50

nzjony force-pushed the MAPSJS-2983_error_if_too_many_attempts branch from 70907f7 to 117c40c Compare August 19, 2021 14:58

nzjony commented Aug 19, 2021

View reviewed changes

@here/harp-mapview/lib/poi/PoiRenderer.ts Outdated Show resolved Hide resolved

atomicsulfate reviewed Aug 23, 2021

View reviewed changes

@here/harp-mapview/lib/poi/PoiRenderer.ts Show resolved Hide resolved

atomicsulfate reviewed Aug 23, 2021

View reviewed changes

@here/harp-mapview/test/PoiBatchRegistryTest.ts Show resolved Hide resolved

atomicsulfate suggested changes Aug 23, 2021

View reviewed changes

nzjony force-pushed the MAPSJS-2983_error_if_too_many_attempts branch from 7b43bd8 to 7e51b07 Compare August 23, 2021 09:51

nzjony requested a review from atomicsulfate August 23, 2021 10:18

nzjony closed this Aug 23, 2021

nzjony reopened this Aug 23, 2021

atomicsulfate suggested changes Aug 23, 2021

View reviewed changes

nzjony requested a review from atomicsulfate August 23, 2021 13:46

nzjony force-pushed the MAPSJS-2983_error_if_too_many_attempts branch from de21974 to e0aaff6 Compare August 23, 2021 13:53

nzjony force-pushed the MAPSJS-2983_error_if_too_many_attempts branch from e0aaff6 to 68c1f11 Compare August 23, 2021 14:11

atomicsulfate previously approved these changes Aug 23, 2021

View reviewed changes

nzjony added 4 commits August 23, 2021 16:13

Update @here/harp-mapview/lib/poi/PoiRenderer.ts

02c4b0b

Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

MAPSJS-2983 Fix broken test

449318a

Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

MAPSJS-2983 Code review comments

5a44545

Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

code review fixes: move initialization to beforeEach and use local re…

715bad5

…source Signed-off-by: Jonathan Stichbury <2533428+nzjony@users.noreply.github.com>

nzjony dismissed atomicsulfate’s stale review via 715bad5 August 23, 2021 14:14

nzjony force-pushed the MAPSJS-2983_error_if_too_many_attempts branch from 68c1f11 to 715bad5 Compare August 23, 2021 14:14

atomicsulfate self-requested a review August 23, 2021 14:16

atomicsulfate approved these changes Aug 23, 2021

View reviewed changes

nzjony merged commit bccab08 into master Aug 23, 2021

nzjony deleted the MAPSJS-2983_error_if_too_many_attempts branch August 23, 2021 14:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAPSJS-2983 Write to the console if we need too many attempts getting… #2284

MAPSJS-2983 Write to the console if we need too many attempts getting… #2284

nzjony commented Aug 19, 2021

codecov bot commented Aug 20, 2021 •

edited

Loading

atomicsulfate Aug 23, 2021

nzjony Aug 23, 2021

nzjony Aug 23, 2021

atomicsulfate left a comment

atomicsulfate Aug 23, 2021

nzjony Aug 23, 2021

atomicsulfate Aug 23, 2021

nzjony Aug 23, 2021

atomicsulfate Aug 23, 2021

nzjony Aug 23, 2021

MAPSJS-2983 Write to the console if we need too many attempts getting… #2284

MAPSJS-2983 Write to the console if we need too many attempts getting… #2284

Conversation

nzjony commented Aug 19, 2021

codecov bot commented Aug 20, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atomicsulfate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 20, 2021 •

edited

Loading