Offer performant Data storage folder option for Android 11

[sonora]

As a result of #10453, I wonder if we should not under OsmAnd settings / Data storage folder offer the additional pre-defined option

• /storage/XXXX-XXXX/osmand

where we should auto-discover XXXX-XXXX to be the ident of the external SD card (like we already do for the option "Manually specified" today).

The background is that under Android 11 this seems to be the only large external storage option creating a performant user experience (search, map indexing,opening the MyPlaces/Tracks screen, etc.). I have tested this on several Android 11 devices now, and it seems to work like a charm. But for the inexperienced user it is near impossible to even think of this

• unless we offer this as an explicit option (outside "Manually specified"),
• and provide respective documentation when to select this option.

[sonora]

It should be noted that in order for this to work the OsmAnd app needs to have the "Storage" permission ("Allow management of all files"), or else you will see an I/O error toast when OsnAnd tries to move the files.

(It seems to me that our current implementation does not request or re-request this permission at this time if once denied.)

[vshcherb]

Worth to try for 4.0 release, if it really works on Android 11.

[sonora]

Tested to work on Samsung S10, Samsung Xcover 4s, and a number of latest gen Samsung tablets with Android 11 (Samsung OneUI 3.0 or 3.1).

Maybe users can report more devices.

I think this should work on any Android 11 (not 10, though, I guess), with our current nightly, i.e. at least as long as we target Android 10.

[vshcherb]

https://stackoverflow.com/questions/50132764/getting-the-path-to-sd-card

[EugeneZmeuk]

For my Samsung S10e (OsmAnd nightly 4.0.0#32379m , 2021-07-19):
I took SD signature in OsmAnd Settings:

After I created "osmand' folder in my SD-card.
Next, I created the pass /storage/11A5-B232/osmand in "Manually specified" option.

I test download maps and rendering.
I did not notice any loss in rendering speed, download maps speed, and routing.

[vshcherb]

@sonora it doesn't look that performance is different between external storage 2 which is well documented and works fine

[sonora]

@EugeneZmeuk There should be no need to create the folder separately. Just in OsmAnd in the data folder setting specifying '/storage/11A5-B232/osmand' as 'Manual' should do the trick. In the dialog afterwards you can select to move your files, that method works for me.

@vshcherb On the devices I have tested there seems a very noticeable performance loss:

• After starting the app, it takes 20-30 sec until the map becomes visible
• After a download, if takes >1 minute to index the maps
• Entering the MyTracks screen and listing my about 30 subfolders takes about 30 sec.
• All of these are nearly instantaneous after moving things fro External storage 2 to just 'osmand'.

In addition, the latter is accessible in the file system, while Android 11 now hides everything under SD/Android/ (just in case you need to manually add tracks etc.).

[vshcherb]

Good point @sonora that it's not accessible to users. I think we can suggest that storage and in case its not accessible to write then we don't use it. It's a bit tricky of course cause every update could kill access to that storage anyway.

[Pastim]

I'm using a Samsung A41. Since updating to Android 11 It has been very slow indeed getting maps to display, let alone updating them. However, this morning (July 21st) there were some other updates, the link to start the app failed, so I reinstalled it. It now seems quite a lot quicker, although still not as fast as it was on Android 10.

Looking at the Data storage folder, it's set on External Storage 2, /storage/2614-1710/Android/data/net.osmand.plus/files. The Manually specified link below is exactly the same, so I can't imagine it behaves differently.

[vshcherb]

@Pastim did you try /storage/2614-1710/osmand?

[Pastim]

@vshcherb I'm just trying a new map download and it is still pretty slow.

I will try creating the folder as suggested, although I'm not sure if I can. Assuming I can, will I be able to access tracks from a USB to PC connection? If not that will be a problem for me. I'll comment here once I've tried things out.

[Pastim]

I can create the folder, and start to set it manually, when when I ask to move the files it says something like:

I/O error /storage/2614-1710/osmand/tiles/OsmAnd metainfo open failed ENOENT.

I can create the osmand folder from my PC using USB, and can move files there, but OSMAND doesn't seem to be able to create the folders.

Edit: I managed to find the extra permissions, so have moved the files. Definitely quicker downloading a new map. Thanks very much indeed!

[sonora]

In my experience, the I/O error under Android 11 appears exactly if the OsmAnd app has not been granted the "Storage" permission ("Allow management of all files"). Once granted, it suffices to use the 'Manual storage' menu item in OsmAnd, specify the correct version of /storage/XXXX-XXXX/osmand/ (OsmAnd will create the folder), then when asked select to 'move' the files.

The operation may take some minutes depending on how many files you have.

No need to create the folder via a PC or anything. (But this folder is of course fully accessible again from a PC or any file manager software, whereas all our other external paths are not).

Drawback is this only works under Android 11. Not yet under 9 or 10 (whete it is not needed). And I cannot yet say what will happen under Android 12, or once we up our target SDK to 11... :(

But I think it is mandarory we offer this as the only reasonable and performant configuration to our users under Android 11, or better even default to it.

[Pastim]

@sonora - agreed. Even finding out how to set the permissions took me a little while, so some help to users would be good, and the benefits are really good.

[xelayz]

I do not understand. I have Android 11, too and osmand is very slow, unuseable. But I do not have a SD card, so I can't fix this? Because I need to enter the manual path with "XXXX-XXXX" which link to the sd card.

[sonora]

@xelayz Maybe try using a folder "osmand" in the root of your internal memory, I have never tested.

[xelayz]

@xelayz Maybe try using a folder "osmand" in the root of your internal memory, I have never tested.

I tried. Not working. But its crazy, I thought the most phones do not have a sd slot anymore and and it looks like I'm the only one without an SD card and Android 11, how can that be? So osm is not really usable on Android 11 without SD card, really bad.

[julienw]

Specifying /storage/XXXX-XXXX/osmand/ as my data folder solved the issue for me !
Samsung S10e, Android 11 from July 1st 2021.

[sonora]

@julienw That's what I expected, thanks for the confirmation!

@xelayz On phones without external SD card slot, OsmAnd's data storage folder menu likely does not show an "External storage 2" option.. But you could try this to use a custom folder on the internal SD card: Note down the path displayed under "External storage 1". Specify that string under "Manually specified", but replace everying after (and including) /Android/data... by simply /osmand. Chances are that could work.

[vshcherb]

@sonora
Looks like this issue will be outdated cause since November 2021 we only can target Android 11. And with targetting Android 11 it's impossible to use these shared directories, only directories named specific with /Android/data/net.osmand

[sonora]

My suggestion is definitely a very noticeable improvement for anybody running any of our recent years' releases, dev builds, or nightly builds on Android 11.

It seems this results from still using Android:targetSdkVersion=26 or below. I understand we are not currently forced to change that on any non-GPlay version, and I am not sure we should as of now?

For GPlay and once forced to target 30 it appears we have to address the cases

• perform a one time data migration for upgrade installs
• subsequently and for fresh installs use scoped storage or any of the alternatives like using a subfolder of Downloads for maps or of Documents for user generated data.
• testing the case of existing installs and users upgrading their OS to 11
• allowing for the fact that some justified OsmAnd use cases require file manager app access and/or PC USB access to the OsmAnd storage folder

I am a little afraid all this may be an initial disaster, and regarding performance maybe even one without a reasonable solution I can see so far...

[sonora]

PS: Let me suggest that as a quick and immediate action on this we implement asap a bit of code which checks if

• the system OS is 11
• targetSdkVersuon <30
• if an SD Card is present, and determines its path,

and if the answer to all this is 'yes'

• upon a fresh unstall: suggests in our initial configuration dialog to use /storage/XXXX-XXXX/osmand as the preferable storage folder
• and if the user on an existing install goes to Settings/Data storage folder/Manually specified: is presented the pre-filled suggestion /storage/XXXX-XXXX/osmand in the dialog (unless already set to some other manual folder).

[vshcherb]

targetSdkVersuon <30

Target sdk version will be 30 since November. Updates will be coming inevitably (restriction of Google Play), since November 1 we will continue to distribute version 4.0 with (minSdkVersion = Android 2.3 and targetSdkVersuon = Android 10 (29) ) but it will be for a limited audience.

Unfortunately due to these limitation I don't want to invest much time into it, especially because the issue was not reproducible on 2 our reference devices. 1 Particular - Samsung S10e with Sdcard - Android 11.

[julienw]

Particular - Samsung S10e with Sdcard - Android 11.

FTR I definitely reproduce on S10e with sdcard on Android 11, with osmand installed from the Play Store.

[kevinmcmurtrie]

Should re-open #10453 since, as of the latest beta, there is no longer any workaround for microSd. OsmAnd+ is literally dead on the next update for all phones not having a SAF bypass switch. Zooming out and panning across map regions disables OsmAnd for 10+ minutes. Indexing is 30 to 60 seconds per file. The battery won't last long enough to render a map after common operations. This is on an Xperia 1 III that runs fluidly without Google's mess.

It seems like the only option is moving out of Google Play for the paid version.

[Pastim]

Which beta version are you talking about? I'm on Android 11, with a separate osmand folder on my SD card, and all is working well at present for me on OsmAnd+ 4.0.8 2021-09-22. Are you saying that's not possible any more, or that it doesn't work?

What is a SAF bypass switch?

[vshcherb]

SAF bypass switch won't fix performance as we checked on our test devices, it's implemented already on many devices, so OsmAnd continues to work on SDcard with Android 11 (if you use External Storage 2 / Multiuser Storage 2). Speed degrades comparing to Internal Storage but it's as fast as working via SAF API

[vshcherb]

The folder that @sonora mentionned is not selectable by default any more on new Google Play release, so the issue will be closed

[Pastim]

I am none the wiser, because I don't understand some of the technical terms used. I don't know was SAF is, nor do I see "External Storage 2" or "Multiuser Storage 2". I just see SD card wen using fx, or Files apps. Sorry for being so ill informed.

So my question is, will I be able to continue to use OsmAnd+ on Android 11 (Samsung A41), even the beta version referred to above that is said to not work properly, by using a separate osmand folder on my SD card? It's not 'fast', but neither is it that slow - it's perfectly usable.

[sonora]

@vshcherb Addendum: It seems that starting with OsmAnd version 4.1.11, we could offer

• /storage/XXXX-XXXX/Download/osmand

as a storage path option which is both performant and is generally accessible via file managers and USB?

EDIT: Needs testing though. Many users report this works fine (#13254 (comment)), but for me any file placed there by other apps or via USB remains invisible to OsmAnd...

[Pastim]

@vshcherb Addendum: It seems that starting with OsmAnd version 4.1.11, we could offer

• /storage/XXXX-XXXX/Download/osmand

as a storage path option which is both performant and is generally accessible via file managers and USB?

EDIT: Needs testing though. Many users report this works fine (#13254 (comment)), but for me any file placed there by other apps or via USB remains invisible to OsmAnd...

Oops - you are right! I can use USB or a file manager to put files in the tracks folder, but OsmAnd+ cannot see them. However. I can add gpx files to the SD Card Download directory and import tracks from there. I am not sure I'll be able to copy maps to OsmANd+.

[sonora]

I'll close this again, as thst's no valid solution then.

[vshcherb]

We can convert to discussion and mark as answered. Issue is something that we're going to implement as for now I don't see it as a generic solution and we didn't reproduce (or didn't notice it) on 1 of our device.

I'm pretty sure that after 1-2 year with Android 12-13 that will be non-relevant discussion but I could wrong

[pebogufi]

@julienw

Specifying /storage/XXXX-XXXX/osmand/ as my data folder solved the issue for me ! Samsung S10e, Android 11 from July 1st 2021.

Is your solution still running with Android 11 with current OsmAnd version, or other solution ?
With my S5e (now Android 10) I am prompted to update.