Images do not render thumbnails in timeline if missing some data

[williamkray]

Describe the bug

I use a Giphy maubot plugin which fetches an image from Giphy.com and uploads the image to the room. In element, these files generate thumbnails just fine and are viewable in the timeline.

In Cinny, these images look like an unrecognized file upload with no thumbnail.

If I manually download the file and re-upload it to Cinny, it renders fine and includes additional image data in the message source. Cinny appears to be depending on this additional data in order to render the thumbnail.

Screenshots and message source included here.

here you see that the giphybot has uploaded the file, but it does not render a thumbnail in the chat. the message source is as follows:

{
  "type": "m.room.message",
  "sender": "@giphy:mssj.me",
  "content": {
    "msgtype": "m.image",
    "body": "something.gif",
    "url": "mxc://mssj.me/13eabfbb9c56b9d34b50bd4707c8d07a1f1aad5e",
    "info": {
      "mimetype": "image/gif",
      "h": "357",
      "w": "360"
    }
  },
  "origin_server_ts": 1660075708630,
  "unsigned": {
    "age": 84
  },
  "event_id": "$Li__PcQ_potscI6Dky-uBw3rJxssPls1xhbZsL07qGc",
  "room_id": "!wgqLEFKEaItGQxtRTp:mssj.me"
}

Uploading the same image results in an animated thumbnail:

with the following message source:

{
  "type": "m.room.message",
  "sender": "@william:mssj.me",
  "content": {
    "info": {
      "mimetype": "image/gif",
      "size": 3831901,
      "w": 360,
      "h": 357,
      "xyz.amorgan.blurhash": "UKB5$~DGugaPx[yqtktmpJbctSkVt%a7Roof"
    },
    "msgtype": "m.image",
    "body": "something.gif",
    "url": "mxc://mssj.me/0c79a3957be5085059b8f6c0e5c41fef3b0fe3f6"
  },
  "origin_server_ts": 1660076008594,
  "unsigned": {
    "age": 65,
    "transaction_id": "m1660076007636.102"
  },
  "event_id": "$YjadYSKv_eBdz00hn1Lpy2Hq3dlJfsVy224gA8RT7Kw",
  "room_id": "!wgqLEFKEaItGQxtRTp:mssj.me"
}

this same exchange is rendered correctly in element-web:

Reproduction

1. use maubot plugins for Giphy or auto-responder, configure them to upload images as reactions
2. trigger the maubot instances to send an image

or, alternatively, use a client that sends similar message source to the above posted message, or send a custom event that matches the same structure.

Expected behavior

the image should be uploaded and sent, and a thumbnail should render (this behavior is expressed in element-web/desktop)

Platform and versions

1. OS: Arch Linux
2. Browser: Firefox 103.0.1
3. Cinny Version: 2.1.1
4. Matrix homeserver: jobmachine.org and mssj.me

Additional context

No response

[williamkray]

i've tried adding the size field to the message source as uploaded by the giphybot, and it does not seem to impact the thumbnail rendering, so it must be something else that is missing from the message data.

[kfiven]

"h": "357",
"w": "360"

should be integers per spec https://spec.matrix.org/v1.3/client-server-api/#mimage but bot is sending them as string. @williamkray

[williamkray]

wow i'm a dumdum. fixing the bot to send the data as an int did indeed resolve the issue. while the behavior is different than the looser handling of element, since you pointed out the spec i can't realistically ask to keep this open.

[williamkray]

actually, i'd like to re-open this... the thumbnails are not rendered if there is no dimensions included either, and the spec does not say that the dimensions or filesize are required parameters. so this is still actually valid in my opinion.

for example, this message source is totally valid according to the spec:

{
  "content": {
    "msgtype": "m.image",
    "body": "to_be_fair.gif",
    "url": "mxc://jobmachine.org/lnHTATixzDaSKlbqVbycoImb",
    "info": {
      "mimetype": "image/gif"
    }
  },
  "origin_server_ts": 1660150043142,
  "sender": "@cog:jobmachine.org",
  "type": "m.room.message",
  "unsigned": {},
  "event_id": "$trtBQHtzYas6Gl0JDZH75VqKKKamlsdKp_wueDAoQgM",
  "room_id": "!asUvXyQSkofLJzSEqe:mssj.me"
}

but does not render a thumbnail image in the chat.

sorry for flip-flopping all over the place here.