Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permission error, can't export DB and no debug log created #153

Closed
simonvanderveldt opened this issue Oct 26, 2015 · 16 comments

Comments

Projects
None yet
2 participants
@simonvanderveldt
Copy link

commented Oct 26, 2015

I wanted to export the DB to import to import it into a self hosted solution #49, but I get the following error message:

Error exporting DB: /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/files/ActivityDatabase:
open failed: EACCES (Permission Denied)

I also tried debugging data syncing before, but there was no debug log. Since no error was shown I didn't know why, but the above could very well be the reason. Maybe it would be a good idea to show a toast message just like when exporting the DB fails?

Seems like adding

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

to the manifest could solve it. See http://stackoverflow.com/a/20631589

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2015

Hm, without a stacktrace I don't know whether this is the source db file or the target db file causing the error. Does the given directory or file actually exist?

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2015

And do you mean, debug logging also failed due to the same permission error? Can you get a logcat with Android Studio or with adb? It should contain the same log output, but more of it (all apps, not just GB).

cpfeiffer added a commit that referenced this issue Oct 26, 2015

@simonvanderveldt

This comment has been minimized.

Copy link
Author

commented Oct 26, 2015

@cpfeiffer Thx for the quick reply!

It's a guess that debug logging doesnt't work because of the same reason, but since it's trying to write to the same location it seems logical that it's the cause.
The directory exists, owned by root:sd_card with permissions 775.

Here's a quick logcat when pressing Export DB:

u0_a75@jgedlte:/ $ su -
root@jgedlte:/ # logcat * | grep -i gadgetbridge
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886): Unable to export dbjava.io.FileNotFoundException: /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/files/ActivityDatabase: open failed: EACCES (Permission denied)
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at libcore.io.IoBridge.open(IoBridge.java:409) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at java.io.FileOutputStream.<init>(FileOutputStream.java:88) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at java.io.FileOutputStream.<init>(FileOutputStream.java:73) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at nodomain.freeyourgadget.gadgetbridge.util.FileUtils.copyFile(FileUtils.java:25) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at nodomain.freeyourgadget.gadgetbridge.database.DBHelper.exportDB(DBHelper.java:46) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity.exportDB(DebugActivity.java:196) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity.access$100(DebugActivity.java:36) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity$8.onClick(DebugActivity.java:143) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at android.view.View.performClick(View.java:4438) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at android.view.View$PerformClick.run(View.java:18422) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at android.os.Handler.handleCallback(Handler.java:733) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at android.os.Handler.dispatchMessage(Handler.java:95) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at android.os.Looper.loop(Looper.java:136) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at android.app.ActivityThread.main(ActivityThread.java:5034) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at java.lang.reflect.Method.invokeNative(Native Method) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at java.lang.reflect.Method.invoke(Method.java:515) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1270) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1086) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:132) ~[na:na]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at dalvik.system.NativeStart.main(Native Method) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886): Caused by: libcore.io.ErrnoException: open failed: EACCES (Permission denied)
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at libcore.io.Posix.open(Native Method) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at libcore.io.BlockGuardOs.open(BlockGuardOs.java:110) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     at libcore.io.IoBridge.open(IoBridge.java:393) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity(28886):     ... 19 common frames omitted
@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2015

As far as I understood, the app should always have access to its own "external" directory without the need for any required read or write permission.

Did Gadgetbridge create the directory or did you create it yourself? Could you try quitting Gadgetbridge, deleting the directory, starting it again and see if that makes a difference?

@simonvanderveldt

This comment has been minimized.

Copy link
Author

commented Oct 27, 2015

@cpfeiffer Yeah, that would make sense. Maybe Android doesn't see it as it's own directory?
I didn't create any directories myself, so I expect that Gadgetbridge created them.

I quit Gadgetbridge and deleted the entire /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge directory (apart from the files directory in it it was empty anyway). Not sure if it's relevant, but had to be root to be able to even see the /storage/emulated/0 directory.
This were the permissions

root@jgedlte:/ # ls -alR /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/
/storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/:
drwxrwxr-x root     sdcard_rw          2015-10-26 20:19 files
/storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge//files:

After opening Gadgetbridge the directory is not immediately created. When I do a Debug > Export DB I get the same error message: Error exporting DB: /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/files/ActivityDatabase: open failed: EACCES (Permission Denied).
These are the permissions of the new directory:

root@jgedlte:/ # ls -alR /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/
/storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/:
drwxrwxr-x root     sdcard_rw          2015-10-27 14:16 files
/storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge//files:

Logcat looks the same as well.

E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232): Unable to export dbjava.io.FileNotFoundException: /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/files/ActivityDatabase: open failed: EACCES (Permission denied)
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at libcore.io.IoBridge.open(IoBridge.java:409) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at java.io.FileOutputStream.<init>(FileOutputStream.java:88) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at java.io.FileOutputStream.<init>(FileOutputStream.java:73) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at nodomain.freeyourgadget.gadgetbridge.util.FileUtils.copyFile(FileUtils.java:25) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at nodomain.freeyourgadget.gadgetbridge.database.DBHelper.exportDB(DBHelper.java:46) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity.exportDB(DebugActivity.java:196) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity.access$100(DebugActivity.java:36) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity$8.onClick(DebugActivity.java:143) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at android.view.View.performClick(View.java:4438) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at android.view.View$PerformClick.run(View.java:18422) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at android.os.Handler.handleCallback(Handler.java:733) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at android.os.Handler.dispatchMessage(Handler.java:95) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at android.os.Looper.loop(Looper.java:136) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at android.app.ActivityThread.main(ActivityThread.java:5034) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at java.lang.reflect.Method.invokeNative(Native Method) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at java.lang.reflect.Method.invoke(Method.java:515) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1270) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1086) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:132) ~[na:na]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at dalvik.system.NativeStart.main(Native Method) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232): Caused by: libcore.io.ErrnoException: open failed: EACCES (Permission denied)
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at libcore.io.Posix.open(Native Method) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at libcore.io.BlockGuardOs.open(BlockGuardOs.java:110) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     at libcore.io.IoBridge.open(IoBridge.java:393) ~[na:0.0]
E/nodomain.freeyourgadget.gadgetbridge.activities.DebugActivity( 6232):     ... 19 common frames omitted

I also found a second directory for gadgetbridge at /storage/extSdCard/Android/data/nodomain.freeyourgadget.gadgetbridge/ this one also only had a files directory in it but had/has different permissions. I removed this directory as well and it was also recreated with the same permissions as before when trying to export the DB.

root@jgedlte:/ # ls -lR /storage/extSdCard/Android/data/nodomain.freeyourgadget.gadgetbridge/
/storage/extSdCard/Android/data/nodomain.freeyourgadget.gadgetbridge/:
drwxrwx--- u0_a156  sdcard_r          2015-09-22 20:53 files
/storage/extSdCard/Android/data/nodomain.freeyourgadget.gadgetbridge//files:

Btw. I'm on Android 4.4.4 on a Galaxy S4 (GPE), which has an external SD card.

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2015

This is very weird. I'm trying to understand what happens here:

  • the exception occurs upon creation of the FileOutputStream, that is, the creation of the destination file, not the reading of the source file
  • further, the destination file is supposed to be /storage/emulated/0/Android/data/nodomain.freeyourgadget.gadgetbridge/files/ActivityDatabase
  • that directory is supposed to be the "external files dir", see https://developer.android.com/reference/android/content/Context.html#getExternalFilesDir(java.lang.String)
  • in particular, every app (e.g. filemanager) having the READ_EXTERNAL_STORAGE permission should be able to read files in there
  • in my understanding, the directory should be owned by the app, not by root, and this is also how it is on my device

In that regard, the exception is actually correct, so we have to find out

  • why the directory is owned by root instead of gadgetbridge and
  • how to change or work around this

One workaround would be to add READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE permissions, but I'd like to avoid this. If you can build Gadgetbridge yourself, you could try if this indeed fixes the problem. If you can't build it, we could try to get e.g. jitpack.io to build it for you.

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

Could you also check the owner and permissions of other apps' directories?

@simonvanderveldt

This comment has been minimized.

Copy link
Author

commented Oct 28, 2015

I think it has something to do with that directory not actually being my ExternalFilesDir (SD Card). As far as I understand it so far there can be an emulated "external" storage, found under /storage/emulated/0 and an actual SD card, in my case found under /storage/extSdCard. The emulated storage is apparently there because of multi-user, the 0 in it is the default user's ID.

Google has some hints about this

getExternalStorageDirectory ()
...
Note: don't be confused by the word "external" here. This directory can better be thought as media/shared storage. It is a filesystem that can hold a relatively large amount of data and that is shared across all applications (does not enforce permissions). Traditionally this is an SD card, but it may also be implemented as built-in storage in a device that is distinct from the protected internal storage and can be mounted as a filesystem on a computer.
...
In devices with multiple shared/external storage directories, this directory represents the primary storage that the user will interact with. Access to secondary storage is available through getExternalFilesDirs(String), getExternalCacheDirs(), and getExternalMediaDirs().

Apparently we should use getExternalFilesDirs(java.lang.String) (notice the plural/s at the end) and if that contains more than 1 entry that 2nd entry should be used together with getExternalStorageState. Note that there is another hint about the uselessness of using /storage/emulated/0 vs the apps private directory gotten with `getFilesDir()

If a shared storage device is emulated (as determined by isExternalStorageEmulated(File)), it's contents are backed by a private user data partition, which means there is little benefit to storing data here instead of the private directories returned by getFilesDir(), etc.

For a bit more info see http://stackoverflow.com/a/23607358
I have no clue what to do when there are more than 2 entries and IMHO it's crazy that Google doesn't offer a proper way to get the app specific directory on real external storage.

If necessary I can still try to build Gadgetbridge myself, but that might take a couple of days since I haven't ever done an Android build :)

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

I'll try something with getExternalFilesDirs().

However using the emulated storage would actually be good, because of the private user partition. It would ensure that other users (not apps) on your mobile could not access that data, despite having READ_EXTERNAL_STORAGE, AFAIU.

I actually suspect that this is a specific issue with your device's OS.

cpfeiffer added a commit that referenced this issue Oct 28, 2015

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

This commit will prefer removable storage over external, but emulated storage. Can you test if this fixes the problem for you?

@simonvanderveldt

This comment has been minimized.

Copy link
Author

commented Oct 29, 2015

However using the emulated storage would actually be good, because of the private user partition. It would ensure that other users (not apps) on your mobile could not access that data, despite having READ_EXTERNAL_STORAGE, AFAIU.
I actually suspect that this is a specific issue with your device's OS.

What was actually the thing you wanted to do initially?
Write to the SD card or to the internal storage?

I actually suspect that this is a specific issue with your device's OS.

I'm running a GPE rom on a regular Galaxy S4, no other app has had this issue before, though that obviously doesn't tell the whole story ;)
I just tried a couple of apps with a directory in /storage/emulated/0/Android/data and all of them could write data there, but all of them also have the WRITE_EXTERNAL_STORAGE permission in their manifest.

See also this bug report against upstream Android https://code.google.com/p/android/issues/detail?id=81357

P.S. There seems to sth wrong with my rom indeed, as far as I understand it now the directories in /storage/emulated/0/Android/data should be owned by the UID of the app and not by root. I have found some things in my phone's init files that seem to be bugs but haven't been able to change them.

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 30, 2015

All we want to do is write to a location that can be accessed by the user, e.g. with a filebrowser. That includes the internal location. Emulated is better than SD card due to multi-user protection, so getExternalFilesDir(String) would be best.

See also this bug report against upstream Android https://code.google.com/p/android/issues/detail?id=81357

Yes, it looks like exactly your problem. Did you check http://blog.chrisolin.com/blog/the-dreaded-android-libcore-bug-from-hell-that-i-wasnt-actually-experiencing-it-was-really-selinux/ ?

P.S. There seems to sth wrong with my rom indeed, as far as I understand it now the directories in > /storage/emulated/0/Android/data should be owned by the UID of the app and not by root. I have found some things in my phone's init files that seem to be bugs but haven't been able to change them.

Exactly, yes. That's why I asked about the permissions of other apps' external folders.

If all fails, we will have to add WRITE_EXTERNAL_STORAGE, I'm afraid.

@simonvanderveldt

This comment has been minimized.

Copy link
Author

commented Oct 30, 2015

Yes, it looks like exactly your problem. Did you check http://blog.chrisolin.com/blog/the-dreaded-android-libcore-bug-from-hell-that-i-wasnt-actually-experiencing-it-was-really-selinux/ ?

I think I've found the culprit, it's not SELinux but the sdcard binary which does the FUSE mapping for external and removable storage is called with wrong arguments from the init system.

root@jgedlte:/ # ps | grep -i sdcard
media_rw  340   1     3216   1380  ffffffff b6f97304 S /system/bin/sdcard
media_rw  1885  1     4152   1352  ffffffff b6f21304 S /system/bin/sdcard
root@jgedlte:/ # cat /proc/340/cmdline                                         
/system/bin/sdcard/data/media/mnt/shell/emulated10231023
root@jgedlte:/ # cat /proc/1885/cmdline                                        
/system/bin/sdcard-u1023-g1023-w1023-d/mnt/media_rw/extSdCard/storage/extSdCard

The directories in /storage/extSdCard/Android/data/ all have a u0_a###owner, whereas the directories in /storage/emulated/0/Android/data/ all have root as owner.

I just built from master and with the changes you've made it created the DB file on the SD card in /storage/extSdCard/Android/data/nodomain.freeyourgadget.gadgetbridge/files.
The /storage/extSdCard/Android/data/nodomain.freeyourgadget.gadgetbridge/ also has the right permissions:

drwxrwx--- u0_a67   sdcard_r          2015-10-30 14:50 nodomain.freeyourgadget.gadgetbridgesimon

I'll try to fix my init system, but I'm not really sure what to do with this issue apart from that. It seems hacky to work around a broken phone by adding WRITE_EXTERNAL_STORAGE.
The current solution works and the data even though it's on the external SD card is owned by the app specific user, we're only missing the multi-user protection (My device doesn't even have the options for it) so maybe the current workaround is good enough?

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Oct 31, 2015

Thanks for digging through this. I'd prefer not to add WRITE_EXTERNAL_STORAGE, so it's good to know that the current workaround works.

I'll make sure that we try emulated directories first, and removable last. We then use the first one that is writable. If there is none, we will pop up an error dialog so that the user can send a bug report.

Then we can still add the extra permission if necessary.

@cpfeiffer cpfeiffer self-assigned this Oct 31, 2015

@simonvanderveldt

This comment has been minimized.

Copy link
Author

commented Oct 31, 2015

Np, learned a lot whilst figuring it out :)

FYI getExternalFilesDirs always returns the emulated/internal storage as the first item in the list, see this test in CTS.

It seems like the logic in 8920f5e can be a bit simplified (basically get the getExternalFilesDirs list and use the first one you can write to) but it works as it should on my phone. If someone with a correctly working /storage/emulated/0 can check it still works for them as well I think you've solved it in an elegant way :)

@cpfeiffer cpfeiffer closed this in d4f070f Nov 1, 2015

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Nov 1, 2015

Thanks for testing this -- I've simplified it now that we know that removable storage (in contrast to external, but emulated storage) works for you.

Now we prefer the primary external storage (typically emulated) if it's writable, and only use removable storage, when the former is not availabe/writable.

This should work with your mobile even with the wrong permissions and without the need to add WRITE_EXTERNAL_STORAGE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.