Error while getting your list from content

[mepreston]

using Google version on Android 10

i started getting this error every time i start the app after i
changed the "Default storage folder" over to my sd card.

Error while getting your list from content:
//com.android.externalstorage.documents/tree/primary%3A1list/document/primary%3A1list%2F.1list
Displaying cached one instead.

i get this for each list i have everytime i access the app.

using a file manager i see a folder with files under it
on the SD card, one for each list. the timestamp on the
files seems to imply they are being updated.

[chris-asdf]

I have the same/a similar problem on Android 11.

Looks like something during the update of the list files went wrong at some point, because the lists which cause the errors can't be reimported into a clean installation (well just cleared data and cache). It just throws me back to the main screen, no error nothing (didn't check adb log).

Additionally on a list with dots in the name (21.9.) the toast is shown minutes after creation of an empty list.

Thanks, Chris

Ps: Uh and btw. where are the cached versions? Even if I just clear the cache the toast persists...

[francisferras]

I have the same/a similar problem on Android 11.

Looks like something during the update of the list files went wrong at some point, because the lists which cause the errors can't be reimported into a clean installation (well just cleared data and cache). It just throws me back to the main screen, no error nothing (didn't check adb log).

Additionally on a list with dots in the name (21.9.) the toast is shown minutes after creation of an empty list.

Thanks, Chris

Ps: Uh and btw. where are the cached versions? Even if I just clear the cache the toast persists...

hey there.

Can you hep me get pass the same mistake?
the solution is clear data and cache?

[rancidfrog]

Is this issue related to Android 11+ lack of support of external storage, forcing apps to use only own folder under /sdcard/Android/data/ ?

Consequence of scoped storage

[chris-asdf]

Hi francisferras,

not really. If you clear your data and your lists reside in the app's private folder, then you'll fry them. Clearing the cache didn't change anything (at least for me)
The only thing which works, is keeping the lists in the app's private storage and completely remove all remaining old lists. And yes this means recreating all your lists by hand, since I don't know how the cached version can be accessed.

But this also means the option to synchronize your files is not existing anymore!

Hope this helps (somehow 🤔)
Chris

[chris-asdf]

@rancidfrog

I'm not sure, because you have to explicitly allow the app on a specific folder when you choose external storage for your lists.
As your message in the toasts reveal, your list is on 'external' storage, too.
If you look in your folder, you will find the files, but they are broken. So during the write, something bad is happening. A timeout or locking or permission problem? If such trivial things exist on Android 😉.
(Even worse, the writes succeed some time. Since it will work for a short while, when the lists are freshly created.)

Any ideas?

Chris

[rancidfrog]

The error occurred after I created a new list, and selecting a different storage location to private app storage:

1. Click on new list, and select 'app private storage'
   
2. Select choose folder, and I selected one on sdcard/documents
   

Then all of a suddentl I am getting this toast error
So,
I believe it is a problem with how new versions of android restrict access to files/storage/ etc.

Not sure how to solve. Maybe app is not asking permissions properly in regards to new standard implemented on newer android versions, 11+
Maybe devs need to ask permission to use selected folder each time user chooses a new location to store lists

[rancidfrog]

@chris-asdf
I just switched the list back to private app storage and toast disappears.
So, it seems the issue is with accessing files on storage, getting the right permissions, etc.

Long click on tab, name of the list creating the error:

In my case o,
Then select edit icon, on the top right
Select more options and switch back to app private storage
And toast error disappears

[rancidfrog]

@chris-asdf
No the list is not broken nor corrupted
It is saved in onelist format,

If you open with fx file explorer, it will show you the list in the stored format.
Accessing file seems to be the problem

[francisferras]

Hi again,

Thanks for the replies.

I've copy paste the tasks to other note app. Uninstalled the app, installed again and made new lists with the copied items.

I will keep things in the app private storage and hope no errors message will pop up.

I'll let you know if stays like this.

Question: maybe is this in settings? Not a dev here

[rancidfrog]

@francisferras
I do not think so.
I believe it is specifically due to scoped storage, external storage.
In the OP you were using sdcard / folder to store notes,

//com.android.externalstorage.documents/tree/primary%3A1list/document/primary%3A1list%2F.1list

.
All you needed to do is edit the list, and return using app private storage for the error to disappear, as mentioned with images in my post above.
Dev needs to take a look at how custom folders are being handled, and check for standard usage for android 11+

[lolo-io]

Hi guys.
I'm sorry I am really busy these days on professional projects, I don't have a lot of time for now to update the app.
As @rancidfrog mentioned, the bugs of import & save lists from/to external folders are all coming from scoped storage. During the development process of this feature I also had some bugs due to some devices models handling storage a different way.
Storage management is always changing on Android making it really difficult to maintain. When I first wrote the app I could easily get access to any folder and had read/write access to it. Now I have to deal with scoped contentResolvers and I'm not sure if it is possible to have a all time read & write access to a specific folder outside the app private folder.
App private folder is this one : "Internal storage\Android\data\com.lolo.io.onelist". You might not see it from your phone browser as its private but you ca see it if you plug your phone to your pc. I don't use this folder so it's empty.

The "cached lists" are actually stored in the app shared preferences as json.

As I was saying file persistence and all time read/write access to a folder causes a lot of trouble to maintain this feature. I was thinking of abandoning it, but I understand some people use it.

If anyone has suggestions or want to help, it will be accepted with great pleasure.

[chris-asdf]

@rancidfrog

• broken / corrupted lists

Well some lists I had were corrupted and couldn't be imported again. They looked fine to me, but I didn't check the syntax. 😉 Others were intact but outdated.

• switching location

That's good to know. I'll give it a try

@lolo-io
Is there anything to look for, so you might pinpoint the issue easier?

[RockyJE]

I'm not a developer, but an avid Android 'tinkerer' who loves the 1List app and will do what I can to help fix/find a work-around, or whatever, for this storage issue. FWIW, I have experience testing apps for devs.

I have 1List installed on:

• Android 10 Motorola phone with Resurrection Remix ROM, unrooted (the device on which I want to use 1List)
• Android 11 tablet with Samsung OneUI 3.1, rooted (installed only so I could look at file locations, but I can test whatever)

Let me know what I can do to help, if anything.

[rancidfrog]

@RockyJE
You need to check how other apps use scoped storage to solve issue.
Specifically, how they access selected storage and gain permission to use folder. How it persists...
Music apps, file browsers, etc.
However, file browsers are specific and might have exceptions

[RockyJE]

I really do appreciate your attention to this, and if you do nothing else with this app it is still the best 'Grocery List' app out there——believe me, I've looked.

The only change that would be nice to have is the ability to use FolderSync to have automated backups of my lists. Manually backing up each list seperately is not optimal, but works, and if I need to add 1List to my rigorous weekly backup schedule, so be it.

That being said, see below post for some, hopefully, useful info.

[RockyJE]

A10, unrooted
A11, rooted
Fresh install v1.3.1, default settings (with a list added)
In "storage/emulated/0/Android/data" I see folders/sub folders for all my other installed apps, but not 1list/com.lolo.io.onelist.

It is here
"/data/user/0/com.lolo.io.onelist"
On rooted device I am able to see my list edits in "/data/user/0/com.lolo.io.onelist/shared_prefs/MainActivity.xml"

I'll do some investigation about my other apps' storage access. Of course, on my unrooted it will be limited.

[RockyJE]

FWIW, I use the closed-source MiXplorer file browser, and the dev released a version specifically to address the scoped storage problem (MiXplorer_v6.58.4-API29_B22020920 is latest). There has been a lot of discussion about this in the app's xda thread (bottom link).

I need to do a bunch more research on this, but in the meantime here's a bit of info from the MiXplorer User Manual thread https://forum.xda-developers.com/t/mixplorer-q-a-and-faq-user-manual.3308582/post-85904825

To access [External Storage]/Android/data on Android 11 (and possibly some other situations).

• "Android 11 with target API-30 doesn't provide files inside those directories. Use the other MiXplorer build with API-29..."

MAIN APP THREAD with downloadable versions
Thread '[APP][2.2+] MiXplorer v6.x Released (fully-featured file manager)' https://forum.xda-developers.com/t/app-2-2-mixplorer-v6-x-released-fully-featured-file-manager.1523691/

[lrq3000]

There is a new library called SimpleStorage that may help solve this issue: https://github.com/anggrayudi/SimpleStorage#request-storage-access-pick-folder--files-request-create-file-etc

[lrq3000]

And here's potentially another helpful hands-on tutorial, with snippets to check permissions: https://medium.com/swlh/sample-for-android-storage-access-framework-aka-scoped-storage-for-basic-use-cases-3ee4fee404fc

Example sourcecode: https://github.com/PavelBorze/SAF_sample

[lrq3000]

I have made an attempt in my own fork, with semi-success:

https://github.com/lrq3000/OneList

Using SimpleStorage, I am able to get the permissions to create files in the user selected folder, but now I'm just having some difficulties with the saveList() function in PersistenceHelper.kt because list.path is always null for some reason I don't understand.

@lolo-io Maybe you can help me understand how the paths are stored for each list? What is the workflow for the saving process? I am new to Kotlin so this doesn't help, and I didn't code in Java for Android since a long time, but I can get around with some tinkering.

As a workaround in case I can't succeed, my fork also implements a Share All Lists button, which serves as a quick and dirty export backup function of all lists as a single text file. There is no Import All Lists function, but it can always be coded later by a more experienced Kotlin dev than me, and it will be retrocompatible with past backups. The Share All Lists function is implemented cleanly in this commit that can be cherry-picked: lrq3000@6893e30

[lrq3000]

For those interested in using the Share All Lists function, here is a debug APK: https://github.com/lrq3000/OneList/releases/tag/shareall

IMPORTANT NOTE: make sure to first backup all your current lists one by one, by using the original Share One List button, before uninstalling your OneList app to install this debug APK.

Side-note: an alternative to SimpleStorage, but made by Google devs but still in alpha, is ModernStorage, but it has a MUCH more complicated API IMHO, its purpose seems to be sligthly different (more complete but hence less simple): https://google.github.io/modernstorage/permissions/

[lrq3000]

I successfully wrote a plain text file on the emulated SD Card on Android 11 using SimpleStorage, this commit shows how:

lrq3000@aaa1793

Found the last bit of solution (makeFile() helper function) in: anggrayudi/SimpleStorage#71

To store the URI in the preferences, another project from the same author as SimpleStorage shows how to do it: https://github.com/anggrayudi/MaterialPreference/blob/5cd9b8653c71fae0314fa2bbf7f71c4c8c8f4104/materialpreference/src/main/java/com/anggrayudi/materialpreference/FolderPreference.kt

@lolo-io May I ask if it would not be too bothering for you to implement the real patch from my proof of concept? I really am not experienced enough in Kotlin to write clean code, so even if I do finish my own implementation, it very likely won't be clean enough for you to merge anyway :-/

[lolo-io]

Hi guys, sorry for not being active a lot on this project.

Thank you everyone for trying to address this issue.

This is a really complex one because when writing a fix I have to make sure this doesn't break lists for people running previous versions of the app and updating it.

Having permanent write permission on a whole folder on a SD Card works with some devices and not with some others, and I also have to take into account previous Android versions.
Sometimes the foxes will work until the user reboots the device and then the permissions are denied, even using takePersistableUriPermission.
Having a temporary right to export a file to a location and editing it is ok, but requesting permanent permission (read AND write) on a file that can be located anywhere is really challenging.

@lrq3000 I will test your patch :)

[lolo-io]

I do not myself use this feature anymore as it is not stable 😄.
I'd like to find a simple way to replace it with some kind of syncing.

[lrq3000]

Don't worry, it's already awesome you made this app, it's very kind of you to try to follow up! Yes file permissions have become way too complicated, it used to be a lot easier! I didn't try if the permissions still worked after closing and restarting the app because i am having issues saving the path (it saves it but it seems we can't use it, likely one of SimpleStorage helper functions must be used to convert back to a uri but documentation is not great). About older android versions, it should work fine, you already wrote code that handles past versions differently, i kept the same idea to only use SimpleStorage with newer Android versions. SimpleStorage can work from Android API 19 upwards, but targeting API 31 (Android 10 iirc /EDIT: for Android 10 it's API 29) should be adequate (because OneList fails to save on Android 10+). 29 déc. 2022 16:20:08 Loïc Teyssier ***@***.***>:

…

I do not myself use this feature anymore as it is not stable 😄 . I'd like to find a simple way to replace it with some kind of syncing. — Reply to this email directly, view it on GitHub[#41 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAIRFXRGQHHE3SYE3ED3UDLWPWT2RANCNFSM45UYXNAQ]. You are receiving this because you were mentioned.[Image de pistage][https://github.com/notifications/beacon/AAIRFXRSFBIBPJDS2QIYLSTWPWT2RA5CNFSM45UYXNA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOKGAPGXA.gif]

[lolo-io]

I just tested the feature on my Android 12 Pixel 5 and it works perfectly, event after restart. The "import" functionality does'nt work though.
SimpleStorage looks promising I will give it a try

[lrq3000]

Awesome! I'm glad the proof of concept worked for you too :D I did not implement an import functionality, did you mean the "Share all lists"? Or you mean i did not modify the import function that was already implemented? If the latter, yes i didn't think about it, but it should be easy to do (it's much easier to select and open a file using SimpleStorage than a whole folder). 29 déc. 2022 16:41:28 Loïc Teyssier ***@***.***>:

…

I just tested the feature on my Android 12 Pixel 5 and it works perfectly, event after restart. The "import" functionality does'nt work though. SimpleStorage looks promising I will give it a try — Reply to this email directly, view it on GitHub[#41 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAIRFXT74NB7D7Y3QIJSKXDWPWWKPANCNFSM45UYXNAQ]. You are receiving this because you were mentioned.[Image de pistage][https://github.com/notifications/beacon/AAIRFXR3JVPTUFVXIEJG4RLWPWWKPA5CNFSM45UYXNA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOKGASLXQ.gif]

[lolo-io]

The original app worked, I will try the POC later :).
The import functionality is available when creating a list -> more options.
Saving lists on an external storage makes no point if they can't be imported afterwards.

[lolo-io]

May I ask you what device / android version you use and how the app is currently behaving for you ?

[lrq3000]

I'm using a Huawei P40 Pro device running Android 10, but on the emulator I run a Pixel 2 on Android 11. 29 déc. 2022 16:50:04 Loïc Teyssier ***@***.***>:

…

May I ask you what device / android version you use ? — Reply to this email directly, view it on GitHub[#41 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAIRFXSHSE5YKR5KMJOQ3QLWPWXKXANCNFSM45UYXNAQ]. You are receiving this because you were mentioned.[Image de pistage][https://github.com/notifications/beacon/AAIRFXW4TXLVK3VTG643VDLWPWXKXA5CNFSM45UYXNA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOKGATTQI.gif]

[lrq3000]

Update: i worked a bit more on trying to fix the issue, here are my observations:

• The app must be reworked to store the URI as-is (the root or folder variables that is fed to the callback after selecting a folder and being granted permission), otherwise if we convert to a String then all is lost, we lose the ability to modify the content of the folder.
• The only workaround I found is to create a OneList subfolder in the Download folder. The Download folder is considered a public storage space, so no permission is necessary to save in it. I added an option to save in the Download folder. To make it work, I had to change the file extension from .1list to a recognized document extension .1list.json.

Right now, saving lists in Download/OneList works. But I can't get my head around how getListAsync() is supposed to work. listsIds[listId] is always returning empty strings. I would need something like list.getNewPath(list.title) instead.

@lolo-io can you please help me? Do you have any suggestion how I can do that? (ie, fetch the list filename?). I'm really having a hard time understanding how this is supposed to work, as the saving and loading list functions behave differently and expect different list representations.

[lrq3000]

Ah, in fact my latest changes make the app crash if we try to create any list except the default "My list" for some reason. It used to work with other lists but I don't know why it doesn't now.

I'll drop the towel here. Kotlin development is too messy for me.

[lrq3000]

I'm too perseverant for my own good. I tried once more, and I could succeed in implementing saving and loading in the Download/OneList folder, because the Download folder requires no permission, it's considered a shared public folder. The APK is available here:

https://github.com/lrq3000/OneList/releases/tag/downloadstorage

Just make sure to enable the "Download/OneList" storage option in the parameters BEFORE creating any new list, because only new lists will be stored in the selected storage option. That's how OneList works, you can theoretically store various lists in different places. But currently, only "Download/OneList" works.

Note however that importing lists does not work. Why? Because even for the Download folder, once you lose permission on files (eg, by erasing OneList's app storage), OneList will be denied permission to open any file it did not create or created in the past before losing permissions.

I think the only way around is to use SimpleStorage to get a ScopedStorage permission, and then store and use the URI directly, without ever converting to a String, because once you convert to a String, you lost permission for some reason, you can't convert back to a working URI.

I implemented SimpleStorage functions to pick files and even folders (so the "Select folder" option should work once the app is rewritten to only use URIs).

To make the importList function work, the blocking issue is to convert the onPathChosen() function to store URIs instead of Strings. I just don't know enough about Kotlin to understand what are function types (it took me hours to find what this was even called, luckily this long document helped). It would be great to fix importList, it's a low hanging fruit, and it would allow to finally get a fully functioning app (although storage would be limited to "Download/OneList", but that's enough to use SyncThing or other apps to sync and backup lists).

I may or may not give a try, but I'm seriously reaching the limits of my abilities in Kotlin and of my patience with its hideous java-like typing and function callback magic system.

[lrq3000]

Relevant issue: anggrayudi/SimpleStorage#103

As the SimpleStorage's author mentions, my current implementation uses MediaStore/MediaStoreCompat instead of DocumentFile/DocumentFileCompat to write/read, and indeed as he writes, MediaStore can only have a temporary access, if we want to get access in the future too we need to use a DocumentFile. The issue is that DocumentFile literally never work! I did not find a single way to get access to any files using DocumentFile or DocumentFileCompat.

So I guess that's the end of the line. I'll ask SimpleStorage's author, but I doubt this is fixable.

There is an alternative library ModernStorage made by Google, currently in alpha, maybe we will have better luck in the future with this library, but for the moment it's way too complex for me to even just try.

[lrq3000]

Well, SimpleStorage's author was very very helpful! Thanks to his inputs, I could fix the original custom storage picker. All features now work, including importing. I also added a new feature to copy all lists to the new storage, so that if you use an older version of the app and install the new one, you can save your lists even if originally they were only saved in the app's private storage.

I am implementing a few more features I want in OneList, then I'll clean up the commits and code to make a clean PR.

[lrq3000]

Done, for those who are interested, please see PR #44, I also provide there a link to a debug APK for the impatient ones.

[lrq3000]

BTW @lolo-io I would also be happy to become a maintainer. I can't promise I will be the most responsive or the most competent, but I think I can do well enough to merge currently outstanding PRs and debug major issues from time to time.

[lrq3000]

@lolo-io Any news in merging PR #44? I have been using my version daily for 2 months now, I have fixed some bugs along the way (several were already there before my PR), I think the PR is very stable and makes the app a lot more useable.

I would also like to reiterate that I would be happy to become a maintainer, it's a great app and a great idea, it would be sad if it would die out.

[lolo-io]

All the storage part have been rewritten and rethinked.
See my comment in this PR for more details : #44 (comment)