-
-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Opening new My Tracks very slow with larger number of tracks #17805
Comments
With S10: |
It's already been noted a number of times, e.g. here #17257, and I believe often before the statistics display was added. Also, I see the same effect under Android 11, even when I use a build which targets Android 10, and/or using the "All files access" Android permission, which circumvents the new file access mechanisms. So chances are the root cause for this may be in how we do things different in our code now vs. the previous (collapsable list) MyPlaces>Tracks screen (which was comparably instantaneous), and not necessarily in the stat calculation or in the new google storage access limitations... |
17257 -> vshcherb closed this as completed on May 29. |
This issue should be checked on latest build as it could be already fixed. Aside from this issue, there was another one that is related to SmartFolders and affected opening My places -> tracks. It is fixed in #2159 |
Link reports error 404, not found After using again my folders with 767 gpx files |
sorry for link - turn put it's closed. 1994f23 |
Unfortunately, I have the same experience, there is no improvement with latest builds. It is still very slow to open either the My Tracks or the Visible Tracks screen. Something we do there seems to be extraordinarily resource intensive. (Interestingly, while it is always slow, times can vary from 3 to 15 sec, Android 11, ~3500 tracks in the folder). What's worse is #18077, which is a showstopper for me using our 4.6.x releases at all, I am forced to stay with 4.5 for all my work. |
Refactoring is needed with GPX Database.
|
I wonder how we did it with the old, expandable list view. That one was comparatively lightning- fast... |
Fun fact: When you create a smart folder with filter condition "Visible on map", that one opens essentially instantaneously. But opening the built-in "Visible tracks" screen creates the long wait, every time. |
@sonora VERY fun fact! |
I tested a new build with the GPX db mprovement merge of 2 weeks ago. The effect was that upon the first app start I received a black, empty map screen with no visible activity, I believe for 2, perhaps 3 minutes. Knowing that the GPX database was probably being updated, I waited things out, and eventually the map appeared and the app functioned normally. Then testing MyPlaces/Tracks and Visible Tracks, there is no decisive speed-up. It still takes about 12 seconds watching the spinner every time I open one of the 2 screens, or go from one to the other. So also this supports my observations that we did not introduce this bug by adding the statistics functionality (bug was here definitely before that), and perhaps anything else which can be fixed in the db, but we added it when we changed from the prior collapsible view to the new paged view. So it looks like we still need an analysis which process or lookup we call every time when opening one of the 2 screens and which scales so badly with increasing number of tracks on the device? (Android 11, OsmAnd build targeting Android 10 and having 'All files access' permission, so no file access limitation in play for this test at all.) |
PS: When I start the app specifically to check out a track, i.e. when after a fresh app start and immediately after the map becomes visible (but presumably the app is still busy with a lot of other initialization tasks), I pull up the My Tracks screen, that takes up to 40 seconds of watching the spinner. |
Work is still in progress, probably this week there will be UI fixes that should improve speed with my places - tracks screen |
@vshcherb Here are some benchmark figures for the time it takes to display the Tracks screen (or again for the Visible tracks screen):
So the time to display the screen seems to grow overproportionately with the number of total tracks present in OsmAnd's track folder. This is e.g. typical of sorting mechanisms. But in any case the code strategy seems flawed in the following aspects: (1) The time to open the Tracks screen should not depend on the total number of tracks in all folders, but merely on the number of items to be shown on the screen to be opened. (In my case simply 70 subfolders and zero tracks on the top level screen) (2) Similarly, there should not have to be another wait time (and every time) when going from the MyPlaces/Tracks screen to the "Visible Tracks" screen. It seems that latest at this time there should already be a table of all visible tracks present in memory, rather than going through the long wait time again (it seems like the complete flat list of all tracks is searched on every occasion?) (3) I suggest to disable the manual pull-down list-refresh functionality on these screens. The list of tracks should not change while users are on these screens, so a pull-down refresh makes no sense (but causes 12 sec delays for me time and time again when I scroll to the top of the list). Initially I thought we may simply have to introduce some caching here of whatever the code produces during these long wait times, so they are not needed upon every (even repeated) opening of the screens. But now I am fairly convinced the code strategy is flawed: Whatever the app keeps doing on these occasions should not be needed via such costly operations at all, for the simple purpose of producing a mostly rather short inventory list on a dialog. -> I would appreciate if whoever implemented the "New Tracks screen" code could take a look at these findings, and at least explain what the code does during these wait periods, and why that would be required upon every opening of the screen or changing between them? |
@Chumva I believe that this is the culprit:
and it leads to this
where the problem may be: We always sort the entire list of all files recursively (regardless of which folder they are in), which is a very costly operation, and probably totally unnecessary? Because almost never do we display the entire flat list, so the sorting should not be applied in the beginning to everything when cataloging all file/folder items, but in the end and only to the much fewer files / folders which are to be shown on the screen's current view? |
PS: A considerable portion of the wait time (about half) is spent for this line, even if I have no Smart Folders created;
|
This obvious fix at least brings some noticeable improvement whenever no smart folders are present: #19228. In my setup this PR speeds things up from 12-14 sec to 5-9 sec. There is more potential, but merging this is a good start. |
More low hanging fruits:
|
It's not correct ... Time depends on the number of tracks that's totally fine. Loading & sorting each time your open a screen is fine as well to create a structure especially on new load. What's not fine that 3500 is a very small number! Loading 3500 records from db and sorting 3500 numbers is about 0.1 seconds should be.
It looks totally fine that it's linear dependency but 0.2 millisec per track is too much. Did you try release versions ? https://download.osmand.net/releases/net.osmand-4.7.2.apk |
But the actual time I am waiting for the list to appear on screens seems even longer (often like twice that), not sure where the extra time is spent. (PS: I find that using a pull-down refresh on the Tracks screen often is quicker than when I re-enter the screen from the menu, also that seems weird.) an, of course, as stated above, I believe there is no need to again re-load the very same the tracks list when changing to the "Visible on maps" screen. And yes, you are right, reading a file list table of 3500 files should be peanuts, that's what I am saying all along ... ;) |
Tonight (or later this week) we decided finally switch nightly builds to release builds. We just tested that startup time is 5 times faster on release builds |
Speed up tracks screen when no Smart Folders are used (partial fix for #17805)
@Chumva Some more code questions:
|
@Chumva I believe we can now merge #19239 , unless you see anything I overlook. This beats the previous fix, the time needed now dropped to 4 - 6 sec for my setup, and I see no drawback. For up to ~1500 tracks I am now at 1 sec. It looks like most of the now remaining time is used on reading the GPX database and creating the stats. |
Today was new build released that's not debug and signed as release apk... So everything needs to be reinstalled actually. I tested it still doesn't work as fast as release but this one became 20-50% faster. |
Actually I was wrong ... Now is build officially released, so it's ready to test. |
yes I can confirm current nightly build 3x-5x faster and a lot faster actually |
Ok, thanks, I am out on a trip and will not be able to re-install my environment for a test, hope someone else can help out here. Looks like you have not merged #19239 yet, it's definitely worth it in any case, though. |
Yes assigned to review & merge |
@vshcherb Just out of curiosity: Using e.g. assembleAndroidFullOpenglFatRelease should produce a "fast" build, no matter if we sign it with our debug or our release key, or am I missing anything here? EDIT: see #18168 (comment) |
Small success story here: After quite some work on streamlining the code, removing recursions, and removing surplus track and folder analysis calculations, @Chumva and I were able to reduce the time it takes to load the 'MyPlaces>Tracks' screen by an order of magnitude. On my setup from once ~40 sec to now <3 sec. And also the logcats now reflect significant improvement:
Still, 3sec feels a bit 'laggyy', of course, that's what we still have #19683 for, I guess. |
Description
With new My tracks my phone S10 is very slow, but not my Tab with Android 10.
I use two devices, which are nearly complete identicaly in OsmAnd folders and files.
+Samsung S 10, Android 12, installed on internal, Android/media
+Samsung Tab S5e, Android 10, installed on Sd card, Android/obb
The S10 is normally much faster, example: install of osmand
S10 needs 5-8 sec,
Tab needs more than 20 sec.
BUT opening my tracks needs (total files in tracks subfolders ~500)
S10 5-7 sec
Tab ~1 sec.
I suppose the problem is the file access restrictions in Android 12 which uses extended file acces when creating statistics (tracks, distance, uphill, downhill, duration, size) which are at bottom line in tracks page.
proposed change
Please add an option/checkbox to decide whether this automatic calculation of statstics should be done. I think this will not only help a lot of users with new Android versions > 10, but make them really happy !
Problem 2
Opening My tracks / "Visible on map" with S10, Android 12 needs so much time, that i often get a message from system, whether to continue or abort the app.
With Tab it is also very slow, 5-10 sec, but no system warning. Visible files with both devices max 5, total files in tracks subfolders ~500.
Environment
OsmAnd Version: OsmAnd~ 4.6.0#537m, veröffentlicht: 2023-08-02
Android version: 12
Device model: Samsung S 10
The text was updated successfully, but these errors were encountered: