Document widget API / specs

[dragetd]

Basically put https://matrix.org/blog/2017/08/23/introducing-matrix-widgets/ into a documentation document that we can point other project maintainers to and tell them: 'Please implement support for this, so we can use your Project inside Matrix' :-)

[richvdh]

We need to get them off the im.vector namespace first.

cc @rxl881

[turt2live]

Namespacing the widget types would also be incredibly helpful to ensure that widgets are interoperable between different clients and integration managers. Currently widgets, like Jitsi, require certain parameters to be specified in the URL so that riot-android and riot-ios can pick it up. Realistically, this shouldn't be hardcoded into the URL and instead should be specified in the data attribute, or possibly a new property. This helps in two ways: it makes parsing the widget easier on everyone and it allows clients to more easily decide if they are going to do custom processing on a widget or just throw up an iframe (as is the case with the Riot mobile apps and Jitsi).

Having the data standardized also helps integration managers work together instead of against each other. Dimension started off trying to support Scalar's widgets in a way where Scalar also could support Dimension's, but that fell apart quickly and ended up being a case where Dimension namespaced it's widgets to avoid breaking Scalar. While trying to remedy this, I got frustrated at the lack of spec, hence this long post.

The last known spec for widgets is here: https://docs.google.com/document/d/1TiWNDcEOULeRYQpkJHQDjgIW32ohIJSi5MKv9oRdzCo/edit

The only deviation from this that I'm aware of is a property named title in data may be specified to set a secondary title on the widget (such as with the TradingView widget currently on develop). Integration managers are expected to be able to serve up an alternative title where possible when this isn't supplied in the widget.

The widget types, and quirks, I'm aware of out in the wild are:

• customwidget - implemented by Scalar and Dimension
• etherpad - implemented by Scalar and Dimension
• googledocs - implemented by Scalar and Dimension
• googlecalendar - implemented by Scalar and Dimension
• jitsi - implemented by Scalar and Dimension
  • The url must have confId and isAudioConf specified as query paramaters.
• youtube - implemented by Scalar and Dimension (Dimension also accepts Dailymotion and Vimeo)
• grafana - implemented by Scalar only (for now)
• spotify - implemented by Scalar only (for now)
• tradingview - implemented by Scalar only (for now)
• twitch - implemented by Dimension only

Dimension goes a step further and prefixes types with dimension- although this feels counterproductive (and makes the widgets incompatible with scalar), so it's phasing this out.

Assuming each specced type is prefixed with m., here's what their specs could look like.

m.jitsi
data should contain:

• domain string - the domain for the jitsi server. eg: "jitsi.riot.im"
• conferenceId string - the conference ID to join
• isAudioOnly boolean - true for audio only, used by riot mobile apps

Wrappers would be responsible for also asking for the display name, avatar url, and user ID as part of their templated URL.

m.tradingview
data should contain:

• exchange string - The exchange to pull data from. eg: "COINBASE"
• symbol string - The symbol to pull data for. eg: "BTCUSD"
• dateRange string - The date range. eg: "1y" (copy the rest from the TradingView docs)

m.spotify
data should contain:

• uri string - The spotify URI. eg: "spotify:artist:378dH6EszOLFShpRzAQkVM" for Lindsey Stirling (artist).
  • This could also be the spotify share URL: "https://open.spotify.com/artist/378dH6EszOLFShpRzAQkVM"

scalar currently implements this, although under the spotify type instead of m.spotify

m.video (replaces youtube), m.googledocs, m.googlecalendar, and everything else
data should contain:

• url string - The URL being wrapped in the actual url. Optional if the url isn't being wrapped.
  • The motivation for this is, again, so clients may opt to do their own processing of the widget, ignoring the scalar/dimension wrapper for the widget.

Of course, comments are welcome on all of this. As stated at the top, this comes out of frustration for a lack of spec while working in the trenches of widgets.

[rxl881]

@turt2live - Firstly, many thanks for this write-up. It all sounds pretty sensible to me. However, we might need to split some of the suggestions out in to their own issues so that we can discuss some of the implementation specifics.

Apologies for lack of progress on the spec. As I'm sure you can understand, we're a little constricted at the moment due to lack of resources (something that should become much less of an issue once the current funding round is concluded, hopefully in the near future).

Right now I'm having to balance the resources of the Scalar team (just me at the moment) between bug fixes, feature requests, operational issues and trying to figure out how to progress widgets (and integrations generally) to make them successful and useful in the long-term. I think (and hope you'll agree) that we're making some really good progress with all of the recent new features, and I'm certainly starting to get a much clearer picture of how widgets and the integrations ecosystem will work, and what sort of features we're likely to want to support. Now that the shape of widgets is beginning to emerge from "the block of marble", there is definitely quite a bit of work to do to bring the spec. and widget APIs up to date. I'm really keen to work with the community on this (especially while the team is so small), so suggestions / proposals / updates for the spec. are really helpful.

As you've suggested, it's becoming clear that moving a lot of the widget config. data from the URL into the 'data' object is a good idea and most/all of the newer integrations that I have been adding (TradingView, Spotify, Google calendar etc.) are starting to do this already. However, there are a couple of items (like widget URL template interpolation that still need to be addressed on mobile - element-hq/riot-meta#125) for this to be fully adopted.

I have a couple of important feature items that I need to look at next, but once those are out of the way I will be coming back to look at this item - standardising the existing integrations and updating the spec. This is likely to be very similar to (if not exactly) what you have described.

In summary, I'm very excited about the direction that widgets and integrations in general are taking. This is all bleeding edge stuff and I am very keen to work with you and the rest of the community working on alternative integration managers to get the new features in to the spec. and make sure that we're all working together.

As ever please feel free to reach out at any time to ask any questions about any items that might not have made it in to the spec. yet. :)

[turt2live]

This ultimately became https://github.com/matrix-org/matrix-doc/issues/1236 - closing.