Android 11, Can't access /storage/emulated/0/android/data

[SudosFTW]

App isn't fully capable of the scoped storage API calls for accessing said directory as a file manager.

• Device: Pixel 3a
• OS: Android 11
• Rooted: No
• Version: 3.5.0

I can't access files saved with apps like Telegram which store downloads in such a location. They don't even show up as recent files in anything else but the built-in file manager in Android (not google files, I have that disabled)

There should be a n actual API call specifically for file managers to access that location but as I'm not completely savvy on the inner workings of modern Android APIs I don't exactly know what it's called. This should probably be looked into heavily when building for Android 11 since the scoped storage API is mandatory now, and it should be seamless to Android 10 users already using the app.

[VishalNehra]

@SudosFTW I'd like to +1 to this... I'm expecting comments from @TranceLove here as he worked on the current implementation.

[TranceLove]

Best case scenario, you would want to ditch java.io.File at all and use content:// URI to even internal/external storages, so you won't need to maintain 2 sets of APIs.

But this is too a big task (that I may want to Kotlintize and refactor the whole HybridFile system) that perhaps too big to fit in 3.5 timeline as well.

[VishalNehra]

@TranceLove I mean, no need to ditch File API at all.. just ways to fix the Android/data/ directories not being accessible right now. Even when we're explicitly giving Read / write permissions with the Uri.

[TranceLove]

As stated in https://developer.android.com/about/versions/11/privacy/storage:

Access to app-specific directories on external storage

On Android 11, apps can no longer access files in any other app's dedicated, app-specific directory within external storage.

Although, there were cases claimed that java.io.File APIs can still access these storage without problems. We probably need some kind of fallback mechanism so that when accessing files using content:// URI fails, we may try using java.io.File instead.

Meanwhile, may try to see if adding android:requestLegacyExternalStorage="true" to AndroidManifest.xml can be a band-it fix.

Will try make a case for this. Though getting Android 11 scoped storage work with Amaze is still a long battle...

[TranceLove]

Tried some little tests today with Pixel 4 emulator running stock Android 11. Even with opt-out of scoped storage with the 2 methods

• no actions at all (compile with SDK 28, no extra flag in AndroidManifest.xml), that user will see Amaze is allowed to manage all files

• adding android:requestLegacyExternalStorage="true" and upgrade SDK to either 29 or 30 (which is required by setting this flag)

contents of data directory is still not visible. Not other apps too - only Android's own DocumentUI can get through.

Though some point in the future we shall be asking Google for MANAGE_EXTERNAL_STORAGE permission via app review anyway, I'm afraid users of Android 11 not using Google Play may be left behind unless Google reaches some kind of agreements with third-party app stores, since this permission shall only benefit apps distributed by Google Play.

Also check https://developer.android.com/training/data-storage for a quick table of file accessibility in Android 11.

[VishalNehra]

Related crashes

Rename:

2020-11-16 03:34:28.529 5634-5935/com.amaze.filemanager.debug E/ActivityThread: Rename recovery failed 
    android.system.ErrnoException: rename failed: EXDEV (Cross-device link)
        at libcore.io.Linux.rename(Native Method)
        at libcore.io.ForwardingOs.rename(ForwardingOs.java:190)
        at libcore.io.BlockGuardOs.rename(BlockGuardOs.java:350)
        at libcore.io.ForwardingOs.rename(ForwardingOs.java:190)
        at android.app.ActivityThread$AndroidOs.rename(ActivityThread.java:7581)
        at java.io.UnixFileSystem.rename(UnixFileSystem.java:368)
        at java.io.File.renameTo(File.java:1412)
        at com.amaze.filemanager.asynchronous.asynctasks.MoveFiles.doInBackground(MoveFiles.java:109)
        at com.amaze.filemanager.asynchronous.asynctasks.MoveFiles.doInBackground(MoveFiles.java:59)
        at android.os.AsyncTask$3.call(AsyncTask.java:394)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:923)

Copy:

2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err: java.io.FileNotFoundException: /storage/emulated/0/Android/data/DummyFile: open failed: EACCES (Permission denied)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at libcore.io.IoBridge.open(IoBridge.java:492)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at java.io.FileOutputStream.<init>(FileOutputStream.java:236)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.filesystem.FileUtil.isWritable(FileUtil.java:754)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.utils.MainActivityHelper.checkFolder(MainActivityHelper.java:420)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.asynchronous.asynctasks.PrepareCopyTask.finishCopying(PrepareCopyTask.java:325)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.asynchronous.asynctasks.PrepareCopyTask.onEndDialog(PrepareCopyTask.java:275)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.asynchronous.asynctasks.PrepareCopyTask.onEndDialog(PrepareCopyTask.java:269)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.asynchronous.asynctasks.PrepareCopyTask.onPostExecute(PrepareCopyTask.java:186)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.amaze.filemanager.asynchronous.asynctasks.PrepareCopyTask.onPostExecute(PrepareCopyTask.java:64)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.os.AsyncTask.finish(AsyncTask.java:771)
2020-11-16 03:34:56.761 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.os.AsyncTask.access$900(AsyncTask.java:199)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:788)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:106)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.os.Looper.loop(Looper.java:223)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:7656)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err: Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at libcore.io.Linux.open(Native Method)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at libcore.io.ForwardingOs.open(ForwardingOs.java:166)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at libcore.io.BlockGuardOs.open(BlockGuardOs.java:254)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at libcore.io.ForwardingOs.open(ForwardingOs.java:166)
2020-11-16 03:34:56.762 5634-5634/com.amaze.filemanager.debug W/System.err:     at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7542)
2020-11-16 03:34:56.763 5634-5634/com.amaze.filemanager.debug W/System.err:     at libcore.io.IoBridge.open(IoBridge.java:478)
2020-11-16 03:34:56.763 5634-5634/com.amaze.filemanager.debug W/System.err: 	... 17 more

[danielzgtg]

On Android 11

No app on Android 11 is supposed to be able to access /sdcard/Android/data/ no matter the mechanism. If you are able to, then you should report such a security vulnerability to Google.

only Android's own DocumentUI can get through

The built-in OS one might be able to read, but not even it has write access

Rooted: No

Unfortunately, I don't think it will ever be possible to access that without root.

I am in a rooted environment, and it still doesn't work normally. However, I found that a workaround is to use /data/media/0/Android/data instead of /sdcard/Android/data. It would be nice if Amaze could detect whether the device is on Android 11 and viewing /sdcard/Android/data and automatically use root to access that

[AndroidDeveloperLB]

If anyone wishes, it seems to only work for Android 11:
https://stackoverflow.com/questions/65967690/how-do-some-apps-reach-the-contents-of-android-sub-folders-on-android

Sadly it doesn't seem to work on Android 12, and I can't find any file manager app that can reach the folder anymore ...

[VishalNehra]

If anyone wishes, it seems to only work for Android 11:
https://stackoverflow.com/questions/65967690/how-do-some-apps-reach-the-contents-of-android-sub-folders-on-android

Sadly it doesn't seem to work on Android 12, and I can't find any file manager app that can reach the folder anymore ...

How can you say it's not working on android 12? Did you accept the permission for folder at Android/data from DocumentUI screen?

[AndroidDeveloperLB]

@VishalNehra I can say it because I have Pixel 4 with Android 12 beta 5, and it failed on all file managers that used to be able to allow it: Total Commander, X-plorer, and Mixplorer. I'm the one that created the post. I thought this workaround would stay for longer.
I was (sadly) also been informed it was "fixed".
:(

[EmmanuelMess]

@VishalNehra I can say it because I have Pixel 4 with Android 12 beta 5, and it failed on all file managers that used to be able to allow it: Total Commander, X-plorer, and Mixplorer. I'm the one that created the post. I thought this workaround would stay for longer.
I was (sadly) also been informed it was "fixed".
:(

Please open a new issue for Android 12.

[AndroidDeveloperLB]

@EmmanuelMess OK here:
https://issuetracker.google.com/issues/200699802

Please consider starring.

I've posted about it in the past, but sadly doesn't seem like Google will do it:
https://issuetracker.google.com/issues/192637587
https://issuetracker.google.com/issues/155215180

[VishalNehra]

@VishalNehra I can say it because I have Pixel 4 with Android 12 beta 5, and it failed on all file managers that used to be able to allow it: Total Commander, X-plorer, and Mixplorer. I'm the one that created the post. I thought this workaround would stay for longer.
I was (sadly) also been informed it was "fixed".
:(

Ok i tried on android 12 emulator. The document ui even doesn't show android/data or Android/obb for me to select them.
So this is a miss from Google's end, I'm assuming this will be fixed by the time 12 reaches production.

[AndroidDeveloperLB]

Wait, why is this re-opened?
Do you guys know of a workaround for Android 12 ?

[VishalNehra]

For android 12 will create a new issue later. Reopened this as I broke file / folder create functionality for normal folders while fixing for android/data. Will fix this as a part of this issue only.

[AndroidDeveloperLB]

@VishalNehra Do you think there is a solution for Android 12?

[TranceLove]

@AndroidDeveloperLB I cannot think of one, if others cannot do so without going through the Google Play review too.😕

[VishalNehra]

@VishalNehra Do you think there is a solution for Android 12?

I'll try to look for some way later, will update here. The android 12 images are quite laggy on my Macbook.

[VishalNehra]

Note to self - Our existing logic to create file / directory was, build newPath = oldPath + newFileName, create directory on this newPath.
This however, doesn't work in case of DocumentFile in Android 11, as when we try to iterate to newPath it'll throw an exception saying the newPath is not valid.
So we have to modify our workflow to include oldpath while creating file / directory instead of utilizing just the newPath (getting newPath.parent doesn't work either).

content://com.android.externalstorage.documents/tree/primary%3AAndroid%2Fdata/document/primary%3AAndroid%2Fdata%2Fcom.activision.callofduty.shooter%2Ffiles

content://com.android.externalstorage.documents/tree/primary%3AAndroid%2Fdata/com.activision.callofduty.shooter/files

[HasinduLanka]

If your phone updated from Android 10 to 11 recently and amaze can't access these folders, uninstalling amaze and installing it again would fix the issue