Bug with avatar blobs - shows wrong avatars on desktop

[Simon-Laux]

@hpk42 and others reported that desktop would show the wrong avatar sometimes.

This seems to be a combination bug of strange blob handling from core combined with caching in desktop/electron.

My theory:
core seems to save avatars to avatar.jpg multiple times somehow.
So desktop loads it, then caches it. And then a new contact gets the same path and instead of reloading (like the other clients would, I assume) it loads it from the browser cache because it has the same name/path.

I don't have the reproduction steps, and I'm guessing from what I saw here, so take it with a grain of salt.

Furthermore my blobs folder is full of the same avatar blobs and as expected they are removed by housekeeping.

What can we do?
ensure core does not replace the avatar.jpg file, like always force a suffix or maybe even add the contact id to the filename to prevent conflicts

[iequidoo]

Maybe we should merge #5495? It can help here and i see no blockers preventing merging it. Currently we already have blobs with random suffixes if several files with the same name are sent. The PR only makes these suffixes always present

[WofWca]

Does Chromium really cache avatars though? I open the dev tools and it doesn't seem to ever use "disk cache" or "memory cache". The only cache-related header that is served with avatars is Last-Modified:, but that alone doesn't seem to affect caching. I tried modifying an avatar in the blob storage while keeping its modified date the same, and still it loads the modified copy, so no caching.

But I'm testing on Windows, maybe it's different on other OSes. And I've personally never noticed the caching bug at all.

[link2xt]

Maybe we should merge #5495? It can help here and i see no blockers preventing merging it.

I think the reason it is not merged is because then UIs need to use new APIs to get filenames.

Minimal fix for this issue would be to generate random filenames only for avatars instead of avatar.jpg and similar names. This avatar blob names are not user-visible, so better if they are completely random, like current timestamp plus address or something like this.

[hpk42]

On Sat, Jul 13, 2024 at 00:20 -0700, WofWca wrote: Does Chromium really cache avatars though? I open the dev tools and it doesn't seem to ever use "disk cache" or "memory cache". The only cache-related header that is served with avatars is `Last-Modified:`, but that alone doesn't seem to affect caching. I tried modifying an avatar in the blob storage while keeping its modified date the same, and still it loads the modified copy, so no caching.

i guess this is what happens: - core hands out the avatar via a filename to Desktop but there is no guarantee that the content is immutable forever. When a new avatar comes in later (even hours later), the former filename is removed and a new one is created. But now the old "avatar.jpg" name can be reused when a new avatar comes in. - Deskstop caches this filename (not the content) for a long enough time so at some point it read the wrong avatar from the removed-then-recreated filename. Android/iOS are a) less long-running than desktop b) probably query CFFI about a contact's filename and thus always are up to date for what is the avatar filename for a contact.

[hpk42]

On Sat, Jul 13, 2024 at 06:33 -0700, holger krekel wrote: On Sat, Jul 13, 2024 at 00:20 -0700, WofWca wrote: > Does Chromium really cache avatars though? I open the dev tools and it doesn't seem to ever use "disk cache" or "memory cache". The only cache-related header that is served with avatars is `Last-Modified:`, but that alone doesn't seem to affect caching. I tried modifying an avatar in the blob storage while keeping its modified date the same, and still it loads the modified copy, so no caching. i guess this is what happens: - core hands out the avatar via a filename to Desktop but there is no guarantee that the content is immutable forever. When a new avatar comes in later (even hours later), the former filename is removed and a new one is created. But now the old "avatar.jpg" name can be reused when a new avatar comes in. - Deskstop caches this filename (not the content) for a long enough time so at some point it read the wrong avatar from the removed-then-recreated filename. Android/iOS are a) less long-running than desktop b) probably query CFFI about a contact's filename and thus always are up to date for what is the avatar filename for a contact.

Note that core may evolve to give the guarantee that a filename will not be re-used but UIs still needs to deal with "file is gone" as an edge case and refetch the filename for a contact and try to get that then.

[Simon-Laux]

android and iOS have no network/file caching.
I have it disabled it sometimes, I guess thats why I didn't see that bug before on my dev machine:

But I recently saw it on another machine, here you can clearly see that electron/chromium must have some caching:

the AI generated woman image is shown for avatar.jpg where the tree image should be shown as it is in the file system.

[Simon-Laux]

Note that core may evolve to give the guarantee that a filename will not be re-used
but UIs still needs to deal with "file is gone" as an edge case
and refetch the filename for a contact and try to get that then.

I don't think thats the case, this is clearly the normal browser caching.
Core should avoid changing the file behind a name. (if we are completely sure core doesn't do this then we can look whether there is a bug in electron itself)

[WofWca]

@Simon-Laux could you tell me the order of operations that you performed? Did you reload Delta Chat page after changing the avatar file?

[Simon-Laux]

I don't know what exactly happened, besides the bot changing its avatar often (every time I restarted it), delta chat was not restarted in between. Sadly I have no reliable reproduction steps for this issue yet.

[WofWca]

A funny thing I noticed is that if you continuously switch between two group chats, sometimes it reloads loads the members' avatars (according to the "Network" tab) when you open the chat, and sometimes it doesn't, so it kind of looks like some Chromium-internal caching, perhaps related to GC?

[Simon-Laux]

Why do you all keep suspecting javascript code when it's clearly chrome web resource caching that has nothing to do with JS or GC?

[WofWca]

That is what I said. Chromium-internal. Try what I did.

[WofWca]

Perhaps this hack would work If we don't want to spend much time on this:

https://stackoverflow.com/questions/1431512/is-there-a-way-to-force-browsers-to-refresh-download-images/1431520#1431520

(i.e. appending a random query string to avatar URL).