Problem loading Mapsforge .map files on Android 10 (api 29)

[julien-nc]

Issue Type

[ ] Question
[x] Bug
[ ] Improvement
[ ] Build system related
[ ] Performance
[ ] Documentation

Description and/or steps/code to reproduce the problem

I'm using Osmdroid in PhoneTrack-Android and everything works fine for Android <= 9.

I can't figure out a way to load Mapsforge .map file in Osmdroid on Android 10. I guess it's caused by the new way of managing storages in Android 10. I saw that things were fixed for tile caching (#1512 #1571).

For the "classic" online tiles providers, it seems the tiles cache is indeed written and red correctly.

So my problem is, first, that the old code sample (findMapFiles method) does not seem to explore the storages according to new Android storage management and fails to find .map files in the default place (/default-storage/osmdroid/*.map). It's looking in /storage/emulated/0/osmdroid/ and /mnt/media_rw/1501-3B0F/osmdroid/ on my virtual device and none of that exists or is accessible.

My second problem is that I can't find a way to "manually" add a .map file (as a workaround). Even if I create a file descriptor to a .map file using the absolute file path I can get from the adb shell, Osmdroid crashes when I try to display it :

org.mapsforge.map.reader.header.MapFileException: cannot read file: /storage/1501-3B0F/osmdroid/languedoc-roussillon.map
        at org.mapsforge.map.reader.MapFile.<init>(MapFile.java:247)
        at org.osmdroid.mapsforge.MapsForgeTileSource.<init>(MapsForgeTileSource.java:69)
        at org.osmdroid.mapsforge.MapsForgeTileSource.createFromFiles(MapsForgeTileSource.java:170)
        at net.eneiluj.nextcloud.phonetrack.android.activity.MapActivity.getMapsForgeTileProvider(MapActivity.java:391)
        at net.eneiluj.nextcloud.phonetrack.android.activity.MapActivity.setTileSource(MapActivity.java:1047)
        at net.eneiluj.nextcloud.phonetrack.android.activity.MapActivity.access$2300(MapActivity.java:110)
        at net.eneiluj.nextcloud.phonetrack.android.activity.MapActivity$13$1.onClick(MapActivity.java:1030)

while the file descriptor says the file exists.

I'm currently loosing time and energy trying to convert files Uri (obtained with ACTION_GET_CONTENT intents) to a File I could provide to Osmdroid. Probably a dead end 😁.

I'll be glad to provide more information if necessary.

Environment

Virtual Android device (API 29) without google services installed.

If it's a bug, version(s) of android this affects:

Android 10 (API level 29)

Version of osmdroid the issue relates to:

v6.1.6

[monsieurtanuki]

@eneiluj Quickly said and probably wrong: what about requestLegacyExternalStorage in the manifest?

[mlendl]

I have no problem to display Mapsforge maps in API 29. I have .map files in the app data storage.

[julien-nc]

@mlendl Nice to know it's possible, thanks for the answer. Could you give more details about how you do it?

Where exactly did you put the .map file? There is no Osmdroid-related directory in SDCARD/Android/data.

How do you get it and load it in your activity? Are you using StorageUtils.getStorageList(); ?

[mlendl]

I have my .map files in SDCARD/Android/data//files/. The file can be directly passed to Mapsforge tile source. Or you can use requestLegacyExternalStorage and put the .map files whereever, but only till august 2020, as Google declared.

[julien-nc]

Thanks for the answer.

I have my .map files in SDCARD/Android/data//files/

Did you forget something between data/ and /files?

The file can be directly passed to Mapsforge tile source

Do you mean you can get a file descriptor of the .map files and then pass them to MapsForgeTileSource.createFromFiles()?

Could you show me an example?

Or you can use requestLegacyExternalStorage

Yep I'd like to avoid that.

[mlendl]

It should be SDCARD/Android/data/my.app.package/files/
File[] maps = {new File(mapFile)};
MapsForgeTileSource.createFromFiles(maps, renderTheme, mapThemeName);

[julien-nc]

Ok thanks a lot.

In case other people have trouble managing .map files on Android 10+, here is my feedback:

The cleanest way I found to allow users to put .map files in the app data storage is to launch an ACTION_GET_CONTENT intent to let users choose a file anywhere on the device (and beyond, like in a Nextcloud storage), then to copy it to the app storage.

• use

File[] externalStorageVolumes = ContextCompat.getExternalFilesDirs(context, null);
File primaryExternalStorage = externalStorageVolumes[0];

to get your app storage.

• use an InputStream to read the .map file from its Uri
• use a FileOutputStream to write the destination file

InputStream inputStream = context.getContentResolver().openInputStream(fileUri);
FileOutputStream outputStream = new FileOutputStream(fdest);
try {
    byte[] buffer = new byte[4 * 1024]; 
    int read;
    while ((read = inputStream.read(buffer)) != -1) {
        outputStream.write(buffer, 0, read);
    }
    outputStream.flush();
} finally {
    inputStream.close();
}

It is then easy to scan the app storage to load all .map files with the findMapFiles method provided in Osmdroid Mapsforge tile provider code sample.

I guess the issue can be closed.
Thanks again.

[behnamKhan70]

Add requestlegacyexternalstorage="true" in manifest.xml
For Android 10

[hgoebl]

OK, does that mean that it's no longer possible to share a map between different apps (e.g. Locus, Orux, ...) in Android 10?
The requestLegacyExternalStorage won't help in the long run.
Are there ways to pass a DocumentFile to MapsForgeTileSource and use the "modern" mechanisms to access files in a more private fashion?

[monsieurtanuki]

I've just had a look at MapsForgeTileSource: we provide Files to the constructor of org.mapsforge.map.reader.MapFile, and that's all we do with Files, on the osmdroid side.
If we use something different from Files (as an option), that means we should use another constructor for mapsforge's MapFile. Would the FileInputStream version suit your needs?

[hgoebl]

Thanks @monsieurtanuki
I'm afraid this would be too tricky and might be disabled in future Android versions.
I'd have to cast the InputStream coming from DocumentFile to a FileInputStream so mapsforge can access the MapFile via the FileChannel behind the FileInputStream. I don't want to rely on that. Might break my app. User has to download and store mapfiles for each app - no way to share those big files. The evolution of Android file access is a shame, especially from 10 to 11. Not too much thinking/testing at Google in this regard.

[monsieurtanuki]

@hgoebl I've not understood 100% so maybe I'm wrong: perhaps you need to create an issue at mapsforge regarding your need to populate MapFile with DocumentFile. Once solved on the mapsforge side, we can do something on the osmdroid side.

[hgoebl]

@monsieurtanuki yes, would be a way. But DocumentFile doesn't expose a File, only an InputStream. mapsforge needs random access (somehow it's a database) and a stream cannot provide random access. So I stay with the solution that users have to reserve enough space so the map-files can be copied to private storage where File-based access will be supported in future releases of Android (most hopefully). It's only a caveat for users with multiple apps and the intention to share .map-files between those apps.

TL;DR your ideas are good, but in my opinion it's too risky regarding future Android releases.