-
Notifications
You must be signed in to change notification settings - Fork 377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MSC2529: Proposal to use existing events as captions for images #2529
Conversation
|
||
* Not sure how this relates to the broader questions discussed in MSC1849 | ||
* This is catering to a narrow use-case requirement. There may be a more general solution available | ||
* Would MSC1767 (extensible events) obsolete this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extensible events would definitely include captions, but IMO basic features like captions shouldn't be blocked on MSCs that aren't going to happen any time soon
}, | ||
``` | ||
|
||
If a client recognises the `rel_type`, they can render the caption with the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What should a client do if someone tries to caption a message that isn't m.image
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe they should be allowed to, but at render time, there is no change, since the rendering client will ignore the rel_type if the target isn't an image. Not sure.
@@ -0,0 +1,44 @@ | |||
# Use existing m.room.message/m.text events as captions for images |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, splitting image and caption into separate events will make bridging very complicated. When to send the image? How long until the caption is sent, then?
it'll still be practically impossible to bridge an image+caption away from matrix, the caption would basically have to be rendered as a separate message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would that be very bad? For example, bridging to Discord would result in two messages next to each other if they cannot be combined by debouncing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, as messages can be sent in between the image and the caption the entire caption context is lost.
Soru personally considers this a blocker, TBH.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Easier bridging (and handling captions in general) is one of the main benefits of #2530 btw
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 to embed the caption in the image event content
"msgtype": "m.text", | ||
"m.relates_to": { | ||
"event_id": "$(some image event)", | ||
"rel_type": "m.caption" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iirc this is a further metadata leak in encrypted rooms.
"msgtype": "m.text", | ||
"m.relates_to": { | ||
"event_id": "$(some image event)", | ||
"rel_type": "m.caption" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure this is the right type under MSC1849: there was fairly long discussion at some point about how we shouldn't need to define new relationship types very often, if at all.
@benparsons What do you think about merging this MSC with my MSC2881: Message Attachments? MSC2881 have similar implementation like yours, but also allow to group several images into one gallery with one text. |
Also I believe that term "Image caption" in separate event is not so suitable for chatrooms, because most of images in chats are screenshots, memes, gifs, random photos from internet, etc, that need not "caption", but the text comment or description from sender. The "Caption" term is good for long articles, which have the long text and inline image between text (aligned to left / right), that demonstrate some significant thing, related to some phrase in this long text. In that situation image really need a caption, that describes "What is this?". But in chat rooms users don't compose long text articles, there are usually short one-two-lined text. So the images, posted between those text lines, don't need the caption, they need the description or comment from sender. And real caption, that can be needed for chatrooms, is short So maybe better to rename "Caption" to "Description" in your MSC? |
A "caption" is a "text comment or description from the sender". |
This is true, but for chatrooms in most cases, it is the image that is the comment to the text message, and not vice versa (not text is comment to image, which can be named "image caption"), here are examples:
In those cases there are too hard to name the text as "Caption to image", because the text is main thing, the image is addition to it. Also, when chat apps displays those texts as regular caption to image (text with decreased size below the image, inside image border, or even shown only on hover), especially when text have many lines - this looks terrible! So, naming text field for picture as "Description" or "Text" instead of "Image caption" will prevent problems with implementing bad UI for display and compose interfaces in Matrix clients. And if we need to keep together with this - also the real "Capture" field for images (for real capture to image, not comment + describing image), better is to have it inline to |
Team member @KitsuneRal has proposed to close this. The next step is review by the rest of the tagged people: Once at least 75% of reviewers approve (and there are no outstanding concerns), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for information about what commands tagged team members can give me. |
|
||
## Background | ||
|
||
There is a demand to be able to apply a text caption to an image, as is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are already several MSC about adding text to images (this one MSC2529, MSC2530, MSC2881, MSC2674, maybe more? please point to them), but we can split their ideas to two types:
-
Classic behavior: media + caption: this is about adding text description, that comments the media. It suit well for articles, where we already have a long text and media (image, video, file) is just an item of the whole story, and it needs a short text description. In that cases it can be named "media caption" or "media description".
-
Chat behavior: text + media(s): This is a common usage of media in chat style conversations, where we have short text messages and a single media item (or series of several medias), that adds some information to that short text. So actually the text is the main entity, and the media is just an addition to it. The examples can be:
- Here is my photos from last weekend [image1, image2]
- My payment is rejected, here is the file with the rejection message: [file.pdf]
- I've got the "Access denied" error, screenshot is attached. How can I solve this problem?
Also very often we have not a single media, but several ones (two or more photos, several following screenshots, etc) and the "media + caption" behavior doesn't suite for such cases at all!
So I want to discuss the whole idea that the classic media caption implementation does not suit well for Matrix ecosystem (and other chat systems too), and that the text + media(s) behavior suits much better for chat style conversations. I agree that sometimes people really want to post a media and add caption to it (for people with disabilities, or to comment what is displayed on the image), but in most cases the behavior is totally backwards!
Please share your own thoughts about this idea!
And maybe discuss this in some better place, could anyone suggest it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems the better place to discuss it is here element-hq/roadmap#13
🔔 This is now entering its final comment period, as per the review above. 🔔 |
The final comment period, with a disposition to close, as per the review above, is now complete. |
Rendered
FCP Checkboxes