Move the exportation folder to a location accessible with scoped storage

[GrazianoCapelli]

Starting in November 2021, app updates will be required to target API level 30 or above.
https://developer.android.com/distribute/best-practices/develop/target-sdk

The API level 30 force the Apps to switch to Scoped Storage.
https://developer.android.com/about/versions/11/privacy/storage

Apps that run on Android 11 but target Android 10 (API level 29, like GPS Logger today) can still request the requestLegacyExternalStorage attribute (GPS Logger does it). This flag allows apps to temporarily opt out of the changes associated with scoped storage, such as granting access to different directories and different types of media files. After you update your app to target Android 11, the system ignores the requestLegacyExternalStorage flag.

This means that getExternalStorageDirectory will be not usable anymore (like @WRPSoft rightly wrote on #130).
If I understood correctly, the root folder of the device storage is not freely accessible anymore.
We have to to migrate the app files to locations that are accessible with scoped storage.

In this period we prepared the app to easily move the app's folders in a different place.
For the Share and the View intents (that are using the Temp subfolder now) the solution is simple, because we already use the FileProvider: we'll use the private FilesDir folder. The Temp folder will not be directly accessible again, but it is a minor issue.
In the meanwhile we made some research in order to find a safer place for /GPSLogger folder (the public folder where the app exports the tracks to be picked by other apps or via USB), but we don't reached a solution yet.

Where could we move the exporting folder when we'll switch to API 30?
The folder should be visible by all other apps, including File Browsers, and should be present on all the devices.

If someone has an idea, we would be glad to receive some suggestion.

[WRPSoft]

Dear @GrazianoCapelli

I am also in the process of reading up on the (new) problem.
Unfortunately, it is not that trivial and not that well thought out by the Google developers.
The download folder, for example, can no longer be addressed as a whole with normal apps (ACTION_OPEN_DOCUMENT_TREE will be very limited now regarding this special folder). But it seems, that I can create a subfolder within the download folder, that can easily scanned/viewed via Intent (Intent.ACTION_OPEN_DOCUMENT_TREE) by all apps.

I don't really understand this, because if the user is informed of the dangers every time the folder selection dialog is called and the download folder will be chosen and the user must confirm, as it's been the case before, he should aware what he is doing.

This is also currently being discussed here too: osmdroid/osmdroid#1678 (comment)

Of course, selecting a single file via the file dialog seems to be no problem, but accessing a whole folder for browsing purposes is now a big task :(

[GrazianoCapelli]

I took some time to delve into the topic and to deepen what @WRPSoft suggested, and I got an idea of a possible approach to take.
We could let the user choose the folder for the exportation the first time it clicks the Export button (using Intent.ACTION_OPEN_DOCUMENT_TREE), in order to gain persistent read/write permission for the folder.
The user will be able to change the exportation folder also in Settings (we could add a new setting: "Exportation Folder").

As a bonus, the users will be able to export the files into any desired folder, including Clouds / FTP / SDCard and so on.

This approach works for Android 5.0+, because ACTION_OPEN_DOCUMENT_TREE is available on Android 5.0 (API level 21) and higher. On Android 4.X we can continue with the old method, hiding the selection of the folder in Settings.

Time for ideas, comments, and suggestions!

[GrazianoCapelli]

TO-DO LIST

• Create a new setting to choose the exporting folder. on Android 4 this setting will be hidden
• Fix the Toast "Track exported on ... folder"
• Take Persistable Uri Permission across the sessions. When the permission will be revoked by the system (or the user deletes the selected folder), the app will ask again the folder to use for exportation.
• Remove the request to delete exported files on track deletion; we take the opportunity to remove this ugly feature because it is rarely useful, and we received feedbacks from some users that accidentally deleted their exported tracks during the track deletion from Tracklist
• Release the Persistable Uri Permission for the old folders used for local exportation
• Manually load of EGM Grid from the set local exportation folder
• Check that the exported files are not deleted from the folder when the user uninstalls the app
• Check the behaviour on all the user cases (for example if the user deletes the folder)
• Manage the Android 4 systems, that don't support the Scoped Storage. The exportation still be done in /GPSLogger folder.
  No deed to ask to grant the storage permission, because Android 4 has not runtime permissions.
• Hide (or disable) the local exportation folder preference on Android 4
• Deep test on Android 4 systems
• Test the manual loading of EGM Grid on Android 4 systems

WISHLIST

• Add the support for Cloud services on Folder selection dialog:
  The folder picker dialog already supports clouds roots, but currently nor Dropbox and neither Google Drive support the SAF folder picker. In the future, when some cloud services will add the support, they will appear into the folder picker dialog.
• When at least one cloud service will support the SAF folder picker, we could also add an extra setting for automatic exportation on track finalization
• Apply the GPS Logger's theme to the Folder selection dialog:
  Currently the folder picker is not styleable, because it is not part of the app, but of the operating system

[GrazianoCapelli]

With the last commit I added the preference, the persistable permission and the fallback method for the old Android 4.X (I wouldn't drop it, the old Android 4 phones are very useful to be used as offline GPS loggers. And, last but not least, I have one of them :) ).
The fallback method is basically the old working method, with the exportation in /GPSLogger folder. To be tested and refined.

Currently if you select the Download folder the app crashes on Export (to be fixed).

I make some research, and I found that the support for Cloud services on Folder selection dialog is not yet supported by the cloud services like Drive or Dropbox. It will appear on dialog as soon as the cloud services will support it.

[GrazianoCapelli]

With the commit 0f4b118 I implemented the possibility to manually load the EGM Grid by placing it into the selected local exportation folder (the same way the user are used to).
I closed the issue #154.

[GrazianoCapelli]

The feature has been fully implemented, and will be deeply tested.
Time for comments, feedbacks, and suggestions.

[GrazianoCapelli]

The feature has been merged into the 'develop' branch, and will be included into the next v3.1.0.

[GrazianoCapelli]

dear @WRPSoft,

I don't really understand this, because if the user is informed of the dangers every time the folder selection dialog is called and the download folder will be chosen and the user must confirm, as it's been the case before, he should aware what he is doing.

These weeks I implemented the new exportation routine basing on Intent.ACTION_OPEN_DOCUMENT_TREE and I made a good amount of tests. In effect I noticed that on some Android versions the Download folder is accessible but not writeable with this method. Nevertheless the users can create a sub-folder of the Download folder with read/write permissions.
For my case I resolved the strange thing by checking if the file is writeable and, if not, aborting the exportation.

Another big limitation of Intent.ACTION_OPEN_DOCUMENT_TREE is due to the lack of support by cloud roots. For example Google Drive, Dropbox, and some other Cloud Services, support the selection of a single file but not the selection of a directory.
Here you can view an open letter to cloud providers on this topic. Dropbox is answering in a sort of evasive way.
If, in case of Dropbox, the reason could be somehow driven by business, I'm a bit worried about the lack of support from Google.

Anyway I don't see an alternative to that, thus Basic Air Data team is going straight with Scoped Storage, hoping on a better support in the future.

[GrazianoCapelli]

P.S: To date we made no test with Cloud Roots.
If someone knows a cloud service that supports the ACTION_OPEN_DOCUMENT_TREE please write us and, when possible, we'll test it.