Images not loading since v5.0.12

[laurensbl]

Since 5.0.12 images aren't loading/displayed anymore. When I select a folder containing images the load animation plays indefinitely and the log states a lot of entries like this one:

[PHP] Error: exif_read_data(): Unable to open file at /srv/www-local/nextcloud/apps/gpxpod/lib/Service/ProcessService.php#940

This occurs in both new and old interface. Tracks are loaded/displayed correctly in both interfaces.

[julien-nc]

Hey, thanks for the bug report. Was it working fine with 5.0.11?

This is weird because:

• I can't reproduce the issue on both dev and production environments 😁
• The related code was not changed between 5.0.11 and 5.0.12

At ProcessService.php#940 (which is pointed by your error log), exif_read_data is used in a tricky way to directly pass the content of the file instead of giving it a path (that makes no sense in this context).

• Which version of Php are you using?
• Was there any change in your server host system recently (close to the GpxPod upgrade)?
• Could you try to downgrade GpxPod to 5.0.11 and check if the issue is still there?

[julien-nc]

Is there any error in nextcloud.log?

I guess the part where GpxPod falls back to IMagick to get geo data from pictures could be more robust and handle exif_read_data crashes. For now, it only falls back if exif_read_data does not crash and does not find geo data.

[laurensbl]

Hmmm, a few things are mixing here, so it seems. Actually, I did recently change OS and PHP version: I switched to Ubuntu 22.04 LTS (from Debian 11). And with it from PHP 7.4 to 8.1.

But the issue no longer throws an error, it now just doesn't display the images on the map like it does in the old interface.

What changed? I dunno. This afternoon, when trying out the new version 5.0.12 I had an rather long re-indexing (command line) for the Memories app running. I can't imagine that interfered, but I mention it nevertheless. Not on the same images as used for GpxPod btw, but another folder. After that I rebooted the container. And I downgraded to 5.0.11, did some testing and upgraded again to 5.0.12.

Well, when testing with 5.0.11 all images where displayed correctly on the map in the old interface, but the new threw a fatal error when switching, so I couldn't test that.

Anyway, no error anymore but nothing on the map either. I will send you an email with some screenshots and the log message from the fatal error under 5.0.11. If you need more info let me know, I'll try to provide it as best as I can.

[laurensbl]

Update: Today I updated and rebooted the host (Proxmox VE) and all is OK now. When selecting a directory in the left pane the tracks/images load much faster now and tracks and images are displayed on the map.

Sorry for bothering, seems that the problems weren't caused by GpxPod.

[julien-nc]

Nice timing, I was about to investigate soon. If this is all good then 🎉
Thanks for the feedback.

I'm interested to know what you think about the new "2 panels" view.

[laurensbl]

I'm interested to know what you think about the new "2 panels" view.

I like it. Most screens nowadays are widescreen and it makes good use of that.
To me, navigation feels intuitive, and it's cool that the track list is dynamic. Making zoom to bounds on double-clicking a folder in the left pane would be a nice addition imho.

It can be sluggish though. I tried some messing around and found that toggling all tracks for instance makes Nextcloud very slow. I toggled all on the topmost folder and it seems to be recursive cause a lot of tracks were drawn on the map. I'm not sure about this however, as I also toggled all on the other folders in quick succession and I can't stop it, toggling again doesn't seem to have effect and changing to the old interface or another Nextcloud app is very slow and as soon as I return to GpxPod new interface it start drawing again. The whole app became quite unresponsive. I have to wait until all tracks are drawn to be able to untoggle all.

I have some large tracks and those load slow, and toggle all seems to get GpxPod/Nextcloud to it's knees with those tracks.
It's just a home PC Nextcloud is running on btw, so I don't expect lightning speed. But a 2 x Intel(R) Core(TM) i3-8100 CPU should be able to handle it better, I guess. Nextcloud is running in a Ubuntu container under Proxmox VE, no other serious activity going on while testing GpxPod.

Select all / Deselect all on the old interface is much faster, albeit Deselect all doesn't stop selecting all either but only deselects already selected tracks while Select all keeps going, It's much more responsive though.

I emailed my tracks folder to you earlier, feel free to test with those tracks.

Also, I miss the ESRI Aerial map in the new interface. It's that map I continue using the old interface for.

Edit: Tracks are drawn now, so I can toggle again. I now see toggling doesn't do anything on the topmost parent map. I guess that's because it contains no tracks, only images. The toggle icon can't be changed anymore however, it stays black / on.

[julien-nc]

Thanks a lot for the feedback.

You can add ESRI Aerial as a custom raster tile server in the new interface. The URL is https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}.

I'll check if it's possible to improve the interface responsiveness while loading tracks. I remember I made a lot of optimizations back then in the old interface.

[julien-nc]

Hey there, 2 things that should improve the speed when loading a lot of tracks at once:

• server side geojson generation is now cached
• saving the enabled state of all tracks is now done in one single request instead of one per track

In the old interface, there was no geojson conversion, the gpx content was parsed by the frontend to generate Leaflet drawing objects. It was reasonably fast.
The new map library (maplibre-gl) only supports displaying geojson so we need to convert either in the backend or the frontend. I took the decision to convert in the backend because I found a nice and simple Php library to do so. Problem is this conversion is done each time the frontend requests a track to the backend. There are 2 ways to improve performance there:

• store the geojson in the database
  • pros: we ensure the conversion only happens once for each track, requesting is then very easy, we get the geojson from the DB
  • cons: it makes the database grow by a quite significant amount, the geojson payload being at least as big as the GPX one
• cache the geojson on the server side
  • pros: No DB access: even faster than storing in the DB. The DB does not grow.
  • cons: Cached values expire at some point, we need to recompute from time to time. Also, some NC instances have no cache system configured. No performance gain in this case...

Bottom line: I decided to use caching. The cons are not that bad and enabling a lot of tracks at once is not so common IMO.

It would be nice if you could try this. It's available in the main branch. If you can't clone and build GpxPod on your side, you can wait for the next release. In any case, you need Nextcloud >= 28 (which is coming very soon) to use a recent GpxPod version.
To be clear: A track's geojson is cached after the track has been obtained at least once by the frontend. So the protocol to observe the performance gain is:

• have a directory with a lot of tracks, add it to GpxPod
• disable all tracks in this directory
• reload GpxPod's page
• directory's context menu: toggle all (this one should be slow)
• disable all tracks by using "toggle all" again
• reload the page to clean the frontend data (if you stay in the same page, the frontend already caches the tracks that were displayed)
• toggle all (this one should be faster)

[laurensbl]

Hi, I will have a look at it as soon as NC is upgraded to v28. Met vriendelijke groet, Laurens

…

-----Original message----- From: Julien ***@***.***> Sent: Sunday 3rd December 2023 23:46 To: julien-nc/gpxpod ***@***.***> Cc: Laurens Blanckenborg ***@***.***>; Author ***@***.***> Subject: Re: [julien-nc/gpxpod] Images not loading since v5.0.12 (Issue #28) Hey there, 2 things that should improve the speed when loading a lot of tracks at once: * server side geojson generation is now cached * saving the enabled state of all tracks is now done in one single request instead of one per track In the old interface, there was no geojson conversion, the gpx content was parsed by the frontend to generate Leaflet drawing objects. It was reasonably fast. The new map library (maplibre-gl) only supports displaying geojson so we need to convert either in the backend or the frontend. I took the decision to convert in the backend because I found a nice and simple Php library to do so. Problem is this conversion is done each time the frontend requests a track to the backend. There are 2 ways to improve performance there: * store the geojson in the database * pros: we ensure the conversion only happens once for each track, requesting is then very easy, we get the geojson from the DB * cons: it makes the database grow by a quite significant amount, the geojson payload being at least as big as the GPX one * cache the geojson on the server side * pros: No DB access: even faster than storing in the DB. The DB does not grow. * cons: Cached values expire at some point, we need to recompute from time to time. Also, some NC instances have no cache system configured. No performance gain in this case... Bottom line: I decided to use caching. The cons are not that bad and enabling a lot of tracks at once is not so common IMO. It would be nice if you could try this. It's available in the main branch. If you can't clone and build GpxPod on your side, you can wait for the next release. In any case, you need Nextcloud >= 28 (which is coming very soon) to use a recent GpxPod version. To be clear: A track's geojson is cached after the track has been obtained at least once by the frontend. So the protocol to observe the performance gain is: * have a directory with a lot of tracks, add it to GpxPod * disable all tracks in this directory * reload GpxPod's page * directory's context menu: toggle all (this one should be slow) * disable all tracks by using "toggle all" again * reload the page to clean the frontend data (if you stay in the same page, the frontend already caches the tracks that were displayed) * toggle all (this one should be faster) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[laurensbl]

Hello there, Btw, I agree using cache is the better option. Best to be selective with what to store in the db imho, should be as few as possible and certainly not expendable data. When the first minor update of 28 is out I will update. I always wait for the second release when upgrading a major version, so that the most severe bugs -if any- are solved already. Better to be on the save side, I use NC for work so I try to minimize the risk of downtime as much as possible. Also, mostly I have to wait for some app updates to be released for compatibility with the new major version. After upgrading I'll test asap, I'll let you know. Met vriendelijke groet, Laurens

…

___________________________ L.T. Blanckenborg Bentemaden 5 8431 RB Oosterwolde The Netherlands T: +31 516 512605 M: +31 6 48077459

-----Original message----- From: Julien ***@***.***> Sent: Sunday 3rd December 2023 23:46 To: julien-nc/gpxpod ***@***.***> Cc: Laurens Blanckenborg ***@***.***>; Author ***@***.***> Subject: Re: [julien-nc/gpxpod] Images not loading since v5.0.12 (Issue #28) Hey there, 2 things that should improve the speed when loading a lot of tracks at once: * server side geojson generation is now cached * saving the enabled state of all tracks is now done in one single request instead of one per track In the old interface, there was no geojson conversion, the gpx content was parsed by the frontend to generate Leaflet drawing objects. It was reasonably fast. The new map library (maplibre-gl) only supports displaying geojson so we need to convert either in the backend or the frontend. I took the decision to convert in the backend because I found a nice and simple Php library to do so. Problem is this conversion is done each time the frontend requests a track to the backend. There are 2 ways to improve performance there: * store the geojson in the database * pros: we ensure the conversion only happens once for each track, requesting is then very easy, we get the geojson from the DB * cons: it makes the database grow by a quite significant amount, the geojson payload being at least as big as the GPX one * cache the geojson on the server side * pros: No DB access: even faster than storing in the DB. The DB does not grow. * cons: Cached values expire at some point, we need to recompute from time to time. Also, some NC instances have no cache system configured. No performance gain in this case... Bottom line: I decided to use caching. The cons are not that bad and enabling a lot of tracks at once is not so common IMO. It would be nice if you could try this. It's available in the main branch. If you can't clone and build GpxPod on your side, you can wait for the next release. In any case, you need Nextcloud >= 28 (which is coming very soon) to use a recent GpxPod version. To be clear: A track's geojson is cached after the track has been obtained at least once by the frontend. So the protocol to observe the performance gain is: * have a directory with a lot of tracks, add it to GpxPod * disable all tracks in this directory * reload GpxPod's page * directory's context menu: toggle all (this one should be slow) * disable all tracks by using "toggle all" again * reload the page to clean the frontend data (if you stay in the same page, the frontend already caches the tracks that were displayed) * toggle all (this one should be faster) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[laurensbl]

Dear Julien, Now that the issues with most of the installed plugins are solved, I could finally upgrade to NC v28. Which means I can do the testing you asked me to do. So I just tested loading the tracks following the steps as described below. The loading is indeed much faster after loading all tracks once before. The difference is very notable. Any idea about how long before that cache will expire? The speed gain is also very convenient when browsing tracks one by one. Might it be an idea to create some sort of cron job for generating cache? A bit like the preview generator app for images, or the indexing in the Memories app. Met vriendelijke groet, Laurens

…

-----Original message----- From: Julien ***@***.***> Sent: Sunday 3rd December 2023 23:46 To: julien-nc/gpxpod ***@***.***> Cc: Laurens Blanckenborg ***@***.***>; Author ***@***.***> Subject: Re: [julien-nc/gpxpod] Images not loading since v5.0.12 (Issue #28) Hey there, 2 things that should improve the speed when loading a lot of tracks at once: * server side geojson generation is now cached * saving the enabled state of all tracks is now done in one single request instead of one per track In the old interface, there was no geojson conversion, the gpx content was parsed by the frontend to generate Leaflet drawing objects. It was reasonably fast. The new map library (maplibre-gl) only supports displaying geojson so we need to convert either in the backend or the frontend. I took the decision to convert in the backend because I found a nice and simple Php library to do so. Problem is this conversion is done each time the frontend requests a track to the backend. There are 2 ways to improve performance there: * store the geojson in the database * pros: we ensure the conversion only happens once for each track, requesting is then very easy, we get the geojson from the DB * cons: it makes the database grow by a quite significant amount, the geojson payload being at least as big as the GPX one * cache the geojson on the server side * pros: No DB access: even faster than storing in the DB. The DB does not grow. * cons: Cached values expire at some point, we need to recompute from time to time. Also, some NC instances have no cache system configured. No performance gain in this case... Bottom line: I decided to use caching. The cons are not that bad and enabling a lot of tracks at once is not so common IMO. It would be nice if you could try this. It's available in the main branch. If you can't clone and build GpxPod on your side, you can wait for the next release. In any case, you need Nextcloud >= 28 (which is coming very soon) to use a recent GpxPod version. To be clear: A track's geojson is cached after the track has been obtained at least once by the frontend. So the protocol to observe the performance gain is: * have a directory with a lot of tracks, add it to GpxPod * disable all tracks in this directory * reload GpxPod's page * directory's context menu: toggle all (this one should be slow) * disable all tracks by using "toggle all" again * reload the page to clean the frontend data (if you stay in the same page, the frontend already caches the tracks that were displayed) * toggle all (this one should be faster) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>