Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement sticker packs #2950

Open
rocurley opened this issue Jan 14, 2017 · 52 comments
Open

Implement sticker packs #2950

rocurley opened this issue Jan 14, 2017 · 52 comments
Labels

Comments

@rocurley
Copy link

@rocurley rocurley commented Jan 14, 2017

I'd like to add support for sticker packs to Riot. Since this would involve writing a lot of code, I wrote this little document up so we could discuss it before I've spent a long time on it. That said, while I'd love to have help, I'm happy to implement this myself.

Why sticker packs?

Most of my friends use Telegram as their chat client of choice. This isn't ideal for several reasons, and I'd like to get them to switch. It may seem silly, but I'm pretty sure the biggest lock-in factor there is stickers. Telegram's sticker system is really slick, and using anything without it or something similar feels like a real downgrade. Relatedly, I've framed most of this in comparison to Telegram's sticker system.

What makes sticker packs good?

I'd claim that the most important features are:

  • User creatable: This is nice on several levels. It saves work on the dev side, obviously. But more than that, this lets people make stickers for their specific interests.

  • Easily sendable: Stickers won't get used if it's not convenient to do so.

  • Easily copyable: Almost all the stickers I use, I have because I saw one I liked, and I added it to my collection. Telegram makes this process trivial.

  • Grouped: Part of the reason stickers spread so effectively is that if I see a sticker I like and add it to my collection, I get a bunch of other stickers along with it that I will likely enjoy.

The ideal outcome is an environment where stickers can be easily
created, and easily spread between people when they chat with each
other.

What are features we could do without?

I'm not super confident here, but:

  • Updating packs. When the creator of a sticker pack adds new stickers to it, that change could be propagated out to users who have the pack, or not. I'm inclined towards not supporting that, mostly on implementation difficulty grounds. One could imagine having sticker packs label themselves as replacements for other ones, and make it easy to replace the old one if the user adds the new sticker pack, but that probably doesn’t need to be in the initial version.

Implementation

I’m proposing a specific implementation here as a proof of concept, but I’d be happy to see it improved. Mostly I’d like to get some discussion about this so I don’t spend a long time implementing this and then have it rejected.

The design goals with the implementation were:

  • Work entirely within within matrix: no external servers.

  • Avoid changes to the matrix protocol.

  • Avoid requiring the homeserver to know about stickers.

  • Avoid nonstandard messages.

This design achieves the first three. To allow subscribing to sticker packs, the fourth is compromised by adding a custom field to the image message when stickers are sent.

A sticker pack is an archive containing one or more images (stickers), and a metadata file. The metadata would associate each sticker with zero or more emoji, and contain the name of the sticker pack. It would also contain whatever information I forgot was important. Since it's likely we won't know ahead of time what information is important, we can version the spec and save the version in the metadata file. It’s unclear what file formats should be supported: that probably depends on how inline images shakes out (although this can be done with the standard image sharing until then).

To publish a sticker pack, one would:

  • Create this archive (ideally assisted by the client, but it could start out as a manual process).

  • Upload the sticker pack to your homeserver. The resulting MCX URI enables other users to subscribe (a misleading term, since it's impossible to update a sticker pack) to the sticker pack.

To subscribe to a sticker pack, the client would:

  • Download the pack using the pack's URI.

  • Save the URI in the client config. This makes subscribing to a sticker pack a a per-user, not per-device affair.

When using the pack, the client would:

  • Check to make sure all the sticker packs in the client-config exist locally. This includes deleting sticker packs that have been unsubscribed from.

  • Allow the user to easily browse and send stickers in the sticker packs.

  • When sending a sticker, include in the message the sticker pack URI. This allows other users to preview and subscribe to other sticker packs.

  • (This can be added late) If the user starts a message with an emoji, show all stickers indexed by that emoji.

Stages

I think the natural way to break this down into sub-tasks, each of which
can be implemented separately, is:

  1. Implement sending (manually created) sticker packs from local storage. This is somewhat similar to Gregoor’s proposal for issue 2648. I’d be happy to coordinate with them on this.

  2. (This doesn’t block anything else, it just has to come after 1) Implement a UI for creating sticker packs.

  3. Synchronize a user’s sticker packs via the client-config.

  4. When using a sticker from a pack, attach the link to the sticker pack.

  5. Make a UI for subscribing to sticker packs you’ve seen.

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 15, 2017

Some details that tessgadwa and I hashed out over chat (it's all at the link above, so you can just read it there if you prefer).
Stickers should be 512x(<=512) vice versa (one dimension 512, one <=512) pngs, or vector graphics (svgs?).
Stickers can be associated with emoji. This implicitly requires the ability to recognize emoji. it seems simplest to piggyback off of the Emoji One conversion process (where if you get an emoji and nothing else in a message, it turns it into a bigger emoji).
We also discussed making some number of default sticker packs, which tessgadwa is interested in doing (in addition to UI work for steps 2 and 5). Stickerpacks of about 12 stickers, ideally including ones that convey: happy, bored, sad, angry, curious, celebratory, congratulatory, amused. (This was pretty off the cuff though).

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 18, 2017

Some more discussion here. We're pretty much set for stage 1 as described above.

It was suggested that we find an existing sticker pack format and just use it. The only such thing I could find was apple's here. I'm sort of disinclined from using it, because it supports a bunch of features we probably don't want (animated stickers, for example), as well as not supporting some features it should (emoji). I think a simple json format along the lines of:

{
    "name" : "Goats with sunglasses"
    "author" : "Nunya Business"
    "stickerpack spec version" : 1.0
    "stickers" [
        {
            "file" : "evilGoat.svg"
            "emoji" : "😈🔥"
        },
        {
            "file" : "similingGoat.png"
            "emoji" : "😊"
        },
        {
            "file" : "nondescriptGoat.png"
            "emoji" : ""
        }
    ]
}

would be a lot easier.

We discussed where to store packs, with the tentative plan to use mxc for now. Other options include IPFS, some website, or torrent hashes. We probably want to have an external website for searching for sticker packs as well at some point.

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Jan 18, 2017

This works for me.

@ara4n
Copy link
Member

@ara4n ara4n commented Jan 18, 2017

how are the emoji fields meant to be used here?

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 18, 2017

If the user starts a message with an emoji, show all stickers indexed by that emoji. Since stickers are used more or less in the same context as emoji, showing a relevant sticker when you type an emoji can be pretty useful

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Jan 18, 2017

I was also noticing that when I used the built-on Riot emojii, I only get 8 emojii showing up. Is this standard?

@tessgadwa tessgadwa mentioned this issue Jan 19, 2017
16 of 71 tasks complete
@tessgadwa
Copy link

@tessgadwa tessgadwa commented Jan 19, 2017

Potential artist to contact suggested: http://www.deviantart.com/art/Phil-The-Owl-629856759

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Jan 19, 2017

Good discussion about in Riot main room today about image uploading vs sticker packs.

Goal is to allow users to access a common set of sticker packs direct from the app without having to upload their own each time from the hard drive.

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Jan 19, 2017

Another big goal: Create a way for artists to upload/share stickers under a CC license for inclusion in Riot.

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 20, 2017

Currently thinking we should just base64 encode the images and stick them in the json file directly. It's a bit inefficient (4/3), but that way we don't have to add a dependency on some tar library or store every sticker separately on the server.

@ara4n
Copy link
Member

@ara4n ara4n commented Jan 20, 2017

@rocurley the disadvantage of doing that is that you don't end up with a canonical URL for the individual images, which will encourage clients to go and take the base64 out of the JSON and keep uploading them in duplicate to their HSes. Ideally the HS should deduplicate the content, but currently synapse doesn't sadly. Whereas if you reference the actual bits of artwork as separate mxc:// URLs, they will automatically get deduplicated. And clients won't have to mess around re-uploading when they send 'em?

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 20, 2017

@ara4n That's definitely a benefit of storing them separately. My main concern there is individual stickers getting deleted. If you just have one file, that's less likely to happen. I still don't have a great model for how likely files are to be deleted though.

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 20, 2017

Also worth figuring out: How do we cache stickerpacks locally? I was planning on fetching them from the homeserver at startup every time for now, but I don't love that option. I looked into localstorage, but 5Mb isn't very much.

@ara4n
Copy link
Member

@ara4n ara4n commented Jan 20, 2017

@rocurley
Copy link
Author

@rocurley rocurley commented Jan 20, 2017

Alright, sgtm.

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Jan 21, 2017

@ara4n
Copy link
Member

@ara4n ara4n commented Jan 21, 2017

one thought that could help here: if we switch to naming mxc urls based on content hashes then deduplication becomes trivial, as does fixing up and recognising content if the original content does get somehow deleted... see also matrix-org/matrix-doc#446 and matrix-org/matrix-doc#539

@ara4n
Copy link
Member

@ara4n ara4n commented Feb 19, 2017

@rocurley did this get anywhere?

@ara4n ara4n added feature p2 labels Feb 19, 2017
@rocurley
Copy link
Author

@rocurley rocurley commented Feb 19, 2017

Sorry, haven't had much time on this for the past couple weeks. As it stands, you can send stickers if you've created and added them manually (with artisinal hand crafted http). Working on the UI for making sticker packs now.

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Feb 19, 2017

Cool. Let me know when the UI is ready.

@rocurley
Copy link
Author

@rocurley rocurley commented Mar 2, 2017

@ara4n so far we have code to send stickers from packs. The mechanisms for adding packs (making your own or adding packs you've seen) are still a work in progress. I'd like to get some code review before then, before we build too much on something that gets heavily modified. The issue is that it's not really mergeable at the moment: We'd be adding a sticker button that essentially nobody can use. It won't really be usable until we have the sticker pack creator working, or a set of sticker packs that are included by default (although that would be much easier to create if we had the sticker pack creator working). Idk what's the best solution here: I could make a mock PR?

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Mar 2, 2017

@rocurley where in the repo would one upload sample stickers to be used in testing?

@rocurley
Copy link
Author

@rocurley rocurley commented Mar 2, 2017

@tessgadwa Honnestly probably the easiest thing would be to send them over riot (to generate the mxc urls). That would make it pretty easy to put together a pack.

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Mar 3, 2017

Hey @rocurley I sent you some raw materials. Probably not what you're looking for but if you need specific image files there are plenty more under CC license (no attribution) on Google Images.

@aviraldg
Copy link
Collaborator

@aviraldg aviraldg commented Mar 7, 2017

@tessgadwa Autocomplete is configured to only show the 8 most relevant ones, but there are more.

@tessgadwa
Copy link

@tessgadwa tessgadwa commented Mar 7, 2017

@aviraldg how are we determining relevance?

@lukebarnard1
Copy link
Contributor

@lukebarnard1 lukebarnard1 commented Jul 10, 2017

@rocurley, how close are we getting mxc URLs to stickers to Riot? The new message composer would need modifying such that users can search for stickers, but we should be landing some changes that allow for sending arbitrary <img src="mxc://..."/> in messages soonish.

@lboklin
Copy link

@lboklin lboklin commented Oct 20, 2017

Any progress on this?

@lukebarnard1
Copy link
Contributor

@lukebarnard1 lukebarnard1 commented Oct 20, 2017

@lboklin the Riot devs have been fairly busy with Communities over the last few weeks and so haven't had the bandwidth, but we'll be getting to it Soon.

@rocurley
Copy link
Author

@rocurley rocurley commented Oct 24, 2017

I kind of dropped this a while back, after I wasn't able to figure out how to get code review on what I had thus far (I don't want to throw stones here: I'm sure everyone's busy, and I'm not super familiar with the processes here). If we can get code review of the work so far, I'd like to keep working on this. I can't guarantee fast process (a lot of my time is spent job-searching right now), but I'm still interested in this feature.

@ara4n
Copy link
Member

@ara4n ara4n commented Nov 14, 2017

hm, i may be going insane, but I don't think i've ever seen any code for this? Is there a PR or a branch somewhere I should know about?

@rocurley
Copy link
Author

@rocurley rocurley commented Nov 14, 2017

https://github.com/rocurley/riot-web/tree/develop
https://github.com/rocurley/riot-web/tree/develop

It's been a long time and TBH I have no idea how I tried to communicate these to you.

@iav
Copy link

@iav iav commented Nov 20, 2017

@rocurley you have to press a button "new pull request" on top of main page of your fork on github.

@arpsyapathy
Copy link

@arpsyapathy arpsyapathy commented Nov 28, 2017

Hi! Is there any test Riot version with the opportunity to exchange stickers? Or still not implemented?

@lukebarnard1
Copy link
Contributor

@lukebarnard1 lukebarnard1 commented Nov 28, 2017

@psyapathy It hasn't been implemented yet, with the main devs working on stabilising widgets and implementing communities.

@arpsyapathy
Copy link

@arpsyapathy arpsyapathy commented Nov 28, 2017

@lukebarnard1 Sad... And then there is the implementation of the Emoji picker on click? To click on the button and get a full list of Emoji?

@lukebarnard1
Copy link
Contributor

@lukebarnard1 lukebarnard1 commented Nov 28, 2017

@psyapathy there's an issue for that somewhere. This isn't the place for discussion though, better off poking us in #riot

@t3chguy t3chguy mentioned this issue Nov 28, 2017
@ara4n
Copy link
Member

@ara4n ara4n commented Dec 14, 2017

So the original PR seems stuck, but we're hoping to get to this as part of Widgets in the relatively near future. Going to keep this bug open for historical value, and to act as a tracker for the upcoming stickerpack widget.

@ara4n ara4n added the meta label Dec 14, 2017
@Qwertie-
Copy link

@Qwertie- Qwertie- commented Apr 10, 2018

I noticed there was some sticker picker integration in riot recently. Are there new updates to this issue?

@lboklin
Copy link

@lboklin lboklin commented Apr 11, 2018

The integration seems to have been implemented in develop, but no actual sticker button appears when it's added.

@t3chguy
Copy link
Collaborator

@t3chguy t3chguy commented Apr 11, 2018

Did you enable it in User Settings > Labs?

@lukebarnard1
Copy link
Contributor

@lukebarnard1 lukebarnard1 commented Apr 11, 2018

Stickers have landed as a "Labs" feature on the develop branch.

@lboklin you're better off asking in #riot:matrix.org or opening a new issue. But yes, as @t3chguy says, make sure you've enabled the feature flag in user settings.

@lboklin
Copy link

@lboklin lboklin commented Apr 11, 2018

I had not! It works after enabling it there.

@lampholder
Copy link
Member

@lampholder lampholder commented May 30, 2018

Okay, so we now have stickers in Riot/Matrix :)

What's the delta (from a user's perspective) between our impl and this original discussion?

  • User creatable: not trivially. Users can create stickers and send them to us to host, or host them on their own personal Dimension instance, but that's not something everybody is going to be capable of/interested in doing/willing to do
  • Easily sendable: ✔️
  • Easily copyable: not really. When you see a sticker somebody else has posted, there's not an easy link back to adding that stickerpack to your own collection
  • Grouped: ✔️
  • Updating packs: ✔️
    We get this 'for free' with the chosen impl (for adding new stickers, not modifying existing stickers in place)
@Qwertie-
Copy link

@Qwertie- Qwertie- commented May 30, 2018

Is there any plan for user creatable packs? The way telegram does it is through a bot you message but I imagine creating a webUI or a custom page in riot for it might be easier to use.

@turt2live
Copy link
Member

@turt2live turt2live commented May 30, 2018

User-created (custom) sticker packs are in the proposal stage at the moment: matrix-org/matrix-doc#1256

@martindale
Copy link

@martindale martindale commented Mar 25, 2019

I don't think this should be included in Riot by default, but rather made available as a plugin.

@schiessle
Copy link

@schiessle schiessle commented Jul 30, 2019

It would be great if Matrix stickers would be compatible to Telegram stickers so that I can easily download a Telegram sticker pack and add it to Matrix.

@turt2live
Copy link
Member

@turt2live turt2live commented Jul 30, 2019

@schiessle keeping in mind that I'm the developer of the 3rd party project, Dimension (https://github.com/turt2live/matrix-dimension) allows you to import packs from Telegram for use in Riot/Matrix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet