Request: Fix for blank/grey thumbnails after running SystemCleaner - "Thumbnail Files"

[ghost]

What should the feature do?

It's a known issue of Android where if you delete the .thumbnails folder in sdcard/DCIM, the system will not longer re-create thumbnails for existing media, in apps using it's default media scanner (the "Media Storage" APK).

When/How would you use it?

After running SystemCleaner - "Thumbnail Files"

How would you implement it?

There are a few ways to implement this, as the developer explained to me in an email (facing the same issue) and I formulated here a few choices, since I would really want to see this brought officially in SD Maid for users not aware of this Android problem:

1. Universal - Haven't tested!!! - Forcing a media rescan of Android;
2. For no-root - Warn the user that after running "Thumbnail Files" the system may fail to re-create the thumbnails for existing media and guide him to delete the app data for the "Media Storage" app (which, also include a disclaimer for this, will also delete some custom playlists and custom options, nothing too important)
3. For root - I pin-pointed the exact file which Android uses to index every single thumbnail file in sdcard/DCIM/.thumbnails, and which does not get updated after cleaning that folder, causing no re-creation of existing files' thumbnails. Deleting this very file solves the issue without affecting anything else in Android's media scanner. It requires a reboot afterwards:
   /data/data/com.android.providers.media/databases/external.db

Comments

///

SD Maid

Version: 4.14.18 (41418)
Unlocker: 4.3.8 (40308) (true)

Device

Root: RootContext(rootState=Root(state=ROOTED)
Fingerprint: /// OnePlus 6 on OOS Open Beta 19 - which is a near-AOSP variant of Android 9

[d4rken]

Some thoughts/brainstorming:

Universal - Haven't tested!!! - Forcing a media rescan of Android;

The downside here would be that we don't know which files were represented by the deleted thumbnail files. Forcing a rescan on the whole storage could lead to previously not added media items being added to the databases, e.g. be shown in the gallery.

Theoretically it should update itself, at the latest after a reboot, but it doesn't always do that and I have not yet found out under which conditions it does and doesn't update.

For no-root - Warn the user that after running "Thumbnail Files" the system may fail to re-create the thumbnails for existing media and guide him to delete the app data for the "Media Storage" app (which, also include a disclaimer for this, will also delete some custom playlists and custom options, nothing too important)

So far the idea with the least downsides, though we all know that not many read disclaimers ;).

For root - I pin-pointed the exact file which Android uses to index every single thumbnail file in sdcard/DCIM/.thumbnails, and which does not get updated after cleaning that folder, causing no re-creation of existing files' thumbnails. Deleting this very file solves the issue without affecting anything else in Android's media scanner. It requires a reboot afterwards:
/data/data/com.android.providers.media/databases/external.db

I was under the impression that this would be equal to clearing the data of the media storage app 🤔 .

[ghost]

This will be interesting

The downside here would be that we don't know which files were represented by the deleted thumbnail files. Forcing a rescan on the whole storage could lead to previously not added media items being added to the databases, e.g. be shown in the gallery.

Hmm sometimes Android has a more watchful eye for newly added files, so there isn't much of a concern of new files popping up in the gallery. Android will sense new files one way or another in the strangest of ways. It's existing files where it doesn't "care".

Theoretically it should update itself, at the latest after a reboot, but it doesn't always do that and I have not yet found out under which conditions it does and doesn't update.

For updating itself after a reboot, it's only for new files. I don't know however if this problem we're talking about, happens to all people or not. For me it happened on every phone and on every ROM I had ever since Android 5. There also exist some discussions online about this problem, where nobody pin-pointed the problem, like this recent one: https://www.reddit.com/r/GooglePixel/comments/avs44p/thumbnails_folder_growing_large_can_i_delete_it/ehlov9s

This problem also doesn't happen in apps not using the Android media scanner. For instance: Google Photos, Solid Explorer, OnePlus Gallery, always re-create thumbnails because they have their own indexing engine. However: Instagram, Facebook and most other social media apps, the "Documents" built-in app for selecting files in Android, etc. rely on the media scanner and are affected by this.

So far the idea with the least downsides, though we all know that not many read disclaimers ;).

Yeah everybody clicks ok ok next ;) At least the downsides aren't huge if you decide to implement an accessibility service to automatically click that button for no-root users.

I was under the impression that this would be equal to clearing the data of the media storage app thinking.

In that folder there is another file: internal.db. Inside of that file there are some pointers to other media files like system sounds and others. It seems that deleting that file will lose the setting for custom ringtone and notification sound among others. So yeah, there is a small difference where completely deleting the data of Media Storage will be slightly more invasive. Nope, tested just now, custom ringtone is still there. But can still be something else being lost in internal.db.

I will continue my research on this. If absolutely nothing happenes in all the cases I can test, then wiping data completely of Media Storage will be the best option.

[ghost]

Today I had some time for more tests. I'm trying to see if just deleting /data/data/com.android.providers.media/databases/external.db has the same effect as wiping the entire data for Media Storage.

It seems that I lose the custom ringtone/notification selection in either case.

And in either case a reboot it's needed for indexing media properly; otherwise it will appear as missing/not show up.

No playlists were lost in either case.

Apart from internal.db (which I will attach internal.db.zip, it's interesting, it appears useless but who knows...), There are two more files in the other directory of the Media Storage data folder:
One is com.android.providers.media_preferences.xml Despite my best efforts of doing media related stuff, this file remains unchanged and just has dummy xml code. So it can just be a placeholder for some niche thing. Creating playlists and stuff doesn't change it and neither do they get lost when messing around with the other app data.
The other one's MediaScanBuild.xml which just contains the device's fingerprint, and has got nothing else to do with our issue, already tried removing that alone and nothing was fixed.

I can say with 99% certainty that we just can delete the entire data of Media Storage. People will just lose the custom ringtone/notification and need a reboot.

Would you want to build a file modification tool into SD Maid? I can search for the exact thing in /data/data/com.android.providers.media/databases/external.db that's preventing the app from "seeing" that ".thumbnails" has been deleted.

Also I am not able to test whether a force rescan solves this issue easily. Can you, or anyone seeing this, force a media rescan after deleting ".thumbnails"?

Thank you.

[d4rken]

People will just lose the custom ringtone/notification and need a reboot.

Hm for something that happens as part of regular intervals, requiring and potentially doing an automatic reboot is a HUGE impact on user convenience. "Just ringtones" will be enough for my inbox to fill with angry emails ;).

Would you want to build a file modification tool into SD Maid? I can search for the exact thing in /data/data/com.android.providers.media/databases/external.db that's preventing the app from "seeing" that ".thumbnails" has been deleted.

Depends on how much work it is. My concern would be that this will likely not be a "one solution fits all" thing, but rather something that requires different modifications depending on app version, Android version and probably also device OEM... 😲

Some more random thoughts:

Would restarting/killing the media provider app, instead of rebooting also fix the issue?

Does reinserting the sdcard have an effect? That's something we could potentially fake.