bug: Cannot fetch Lemmy posts with links and/or images

[ShittyKopper]

💡 Summary

Fetching a Lemmy post with a link or an image does not work. Text-only posts do work fine.

🥰 Expected Behavior

For the post(s) to be fetched just fine

🤬 Actual Behavior

Log for https://sopuli.xyz/post/5429583:

Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [remote ap]        Creating the Note: https://sopuli.xyz/post/5429583
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [remote ap]        Creating the Person: https://sopuli.xyz/u/sunnie
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [remote ap]        Creating the Image: https://sopuli.xyz/pictrs/image/e98ecc3f-381d-48f8-8161-f5efe7cc399f.jpeg
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [download]        Downloading https://sopuli.xyz/pictrs/image/e98ecc3f-381d-48f8-8161-f5efe7cc399f.jpeg to /tmp/tmp-124-Y3Yr0s5cQsKb ...
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [metadata]        Fetching metadata of sopuli.xyz ...
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [metadata]        Fetching nodeinfo of sopuli.xyz ...
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [metadata]        Fetching HTML of sopuli.xyz ...
Nov 03 13:25:48 armserver sharkey[1734516]: DONE *        [download]        Download finished: https://sopuli.xyz/pictrs/image/e98ecc3f-381d-48f8-8161-f5efe7cc399f.jpeg
Nov 03 13:25:48 armserver sharkey[1734516]: DONE *        [metadata]        Successfuly fetched nodeinfo of sopuli.xyz
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [drive register]        {"size":144637,"md5":"1de80dfa99d6c3f712b981ac1b93f087","type":{"mime":"image/jpeg","ext":"jpg"},"width":500,"height":500,"blurhash":"eORfLexu??xux[_LWCRko>
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [drive register]        web image not created (original satisfies webpublic)
Nov 03 13:25:48 armserver sharkey[1734516]: INFO *        [drive register]        thumbnail stored: thumbnail-78ccd443-b5c9-4fce-b71a-77496631adf1
Nov 03 13:25:48 armserver sharkey[1734516]: DONE *        [drive register]        drive file has been created 9lm7qckpni1l06g3
Nov 03 13:25:48 armserver sharkey[1734516]: DONE *        [drive downloader]        Got: 9lm7qckpni1l06g3
Nov 03 13:25:48 armserver sharkey[1734516]: ERR  *        [api]        Internal error occurred in ap/show: invalid image: url not provided

Nov 03 13:25:48 armserver sharkey[1734516]: Error: invalid image: url not provided
Nov 03 13:25:48 armserver sharkey[1734516]:     at ApImageService.createImage (file:///sharkey/packages/backend/built/core/activitypub/models/ApImageService.js:52:19)
Nov 03 13:25:48 armserver sharkey[1734516]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Nov 03 13:25:48 armserver sharkey[1734516]:     at async ApImageService.resolveImage (file:///sharkey/packages/backend/built/core/activitypub/models/ApImageService.js:94:16)
Nov 03 13:25:48 armserver sharkey[1734516]:     at async Promise.all (index 0)
Nov 03 13:25:48 armserver sharkey[1734516]:     at async ApNoteService.createNote (file:///sharkey/packages/backend/built/core/activitypub/models/ApNoteService.js:164:23)
Nov 03 13:25:48 armserver sharkey[1734516]:     at async _$class.fetchAny (file:///sharkey/packages/backend/built/server/api/endpoints/ap/show.js:150:141)
Nov 03 13:25:48 armserver sharkey[1734516]:     at async file:///sharkey/packages/backend/built/server/api/endpoints/ap/show.js:111:28
Nov 03 13:25:48 armserver sharkey[1734516]:     at async ApiCallService.call (file:///sharkey/packages/backend/built/server/api/ApiCallService.js:316:16) 66bd4fc4-4d48-4cf7-84ae-72ebcc48b977

Log for https://programming.dev/post/5191385:

Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [remote ap]        Creating the Note: https://programming.dev/post/5191385
Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [remote ap]        Creating the Person: https://programming.dev/u/snaggen
Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [remote ap]        Creating the Image: https://programming.dev/pictrs/image/7fc56aab-5618-4dfb-abb1-0a35bd0d79df.jpeg
Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [download]        Downloading https://programming.dev/pictrs/image/7fc56aab-5618-4dfb-abb1-0a35bd0d79df.jpeg to /tmp/tmp-124-WFeVqWI2udH9 ...
Nov 03 13:37:51 armserver sharkey[1734516]: DONE *        [download]        Download finished: https://programming.dev/pictrs/image/7fc56aab-5618-4dfb-abb1-0a35bd0d79df.jpeg
Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [drive register]        {"size":160754,"md5":"d02d824871507715feb61f81e56c058f","type":{"mime":"image/jpeg","ext":"jpg"},"width":512,"height":512,"blurhash":"e-G+Xbs9whnhRQ~9i_rrj>
Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [drive register]        web image not created (original satisfies webpublic)
Nov 03 13:37:51 armserver sharkey[1734516]: INFO *        [drive register]        thumbnail stored: thumbnail-98bbfea5-e22f-4d59-b4a4-bbe324216302
Nov 03 13:37:51 armserver sharkey[1734516]: DONE *        [drive register]        drive file has been created 9lm85ur2ni1l06g7
Nov 03 13:37:51 armserver sharkey[1734516]: DONE *        [drive downloader]        Got: 9lm85ur2ni1l06g7
Nov 03 13:37:51 armserver sharkey[1734516]: ERR  *        [api]        Internal error occurred in ap/show: invalid image: url not provided

Nov 03 13:37:51 armserver sharkey[1734516]: Error: invalid image: url not provided
Nov 03 13:37:51 armserver sharkey[1734516]:     at ApImageService.createImage (file:///sharkey/packages/backend/built/core/activitypub/models/ApImageService.js:52:19)
Nov 03 13:37:51 armserver sharkey[1734516]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Nov 03 13:37:51 armserver sharkey[1734516]:     at async ApImageService.resolveImage (file:///sharkey/packages/backend/built/core/activitypub/models/ApImageService.js:94:16)
Nov 03 13:37:51 armserver sharkey[1734516]:     at async Promise.all (index 0)
Nov 03 13:37:51 armserver sharkey[1734516]:     at async ApNoteService.createNote (file:///sharkey/packages/backend/built/core/activitypub/models/ApNoteService.js:164:23)
Nov 03 13:37:51 armserver sharkey[1734516]:     at async _$class.fetchAny (file:///sharkey/packages/backend/built/server/api/endpoints/ap/show.js:150:141)
Nov 03 13:37:51 armserver sharkey[1734516]:     at async file:///sharkey/packages/backend/built/server/api/endpoints/ap/show.js:111:28
Nov 03 13:37:51 armserver sharkey[1734516]:     at async ApiCallService.call (file:///sharkey/packages/backend/built/server/api/ApiCallService.js:316:16) 0b010a8e-7d04-42ed-92ed-3cc61ff59a5a

📝 Steps to Reproduce

1. Try fetching https://sopuli.xyz/post/5429583 and https://programming.dev/post/5191385
2. Observe unlabeled error in the frontend

💻 Frontend Environment

No response

🛰 Backend Environment (for server admin)

* Installation Method or Hosting Service: Podman (custom built)
* Misskey: stable (commit 6390c829abbd38b9b813103c8d3edaa907e6162f) (with [extra changes that shouldn't affect any backend code](https://github.com/ShittyKopper/Sharkey/commits/kopper))
* Node: v20.5.1
* PostgreSQL: 16.0 (Debian 16.0-1.pgdg120+1) (docker.io/library/postgres:16-bookworm (image id 96f08c06113e))
* Redis: 7.2.2 (docker.io/library/redis:7-bookworm (image id 637ceb59b7a0))
* OS and Architecture: Debian GNU/Linux 12 (bookworm) aarch64

[Mar0xy]

It's hard to know what Lemmy passes through as a note and I would also recommend opening this issue upstream instead of here as I think this should be fixed on a wider scale than just this fork.

[ShittyKopper]

just to track things properly, i think i found the real issue. both lemmy and misskey are at fault here

lemmy bug: LemmyNet/lemmy#2611 (comment)
upstream misskey bug: misskey-dev/misskey#12243