New Social Media Card extension

[2bndy5]

I'm exploring how to implement a theme-specific extension similar to upstream's builtin social plugin.

‼️ Coincidentally, there recently was a major overhaul of that plugin for the insiders edition. It now involves customizable "layouts" used to generate the social card image. I like the changes and want to include something similar in this new sphinx-immaterial extension, but I'm currently brainstorming an adequate approach (might end up with a pythonic approach in conf.py).

🚧 Locally, I have the basic features from upstream implemented, but it is still very fluid (optimizing code and execution overhead, adding thorough customizability, etc).

This will also rely on google fonts caching, so it will unfortunately add to the desired solution in #222

Alternate approaches

Currently there's nothing stopping doc authors from using other sphinx extensions (ie sphinxext-opengraph) or implementing it manually (via metadata roles/directives). However, these approaches lack in some way or another:

• sphinxext-opengraph isn't extensible enough and doesn't include the metadata required by twitter.
• manual implementation is best done with an added HTML template (almost identical to what was documented upstream), but can be laboriously done on a per-page basis using the metadata fields or the newer docutils meta directive. Either way, the card's image has to be created manually as well, making this approach even more laborious.

New dependencies

To generate the card image, I'll be using pillow.

cairosvg

Upstream uses pillow, but they also require a cairosvg to optimize SVGs and convert them to something pillow can use. My current focus is not on optimizing all images used in the generated docs.

The upstream docs state:

Cairo Graphics is a graphics library and dependency of Pillow

But I didn't find this to be true. Pillow works fine by itself for my needs here. Since cairosvg requires a multitude of non-python dependencies, I'm trying to avoid offloading that platform-specific setup (a perceivable nightmare without a docker container) to end-users and CI workflows. Unfortunately, this means that we can't support SVG images in card generation (unless pillow adds the necessary support). Remember that our html_logo config is not bound to only using SVGs (which is the behavior of an example in upstream's theme.icon.logo).

matplotlib

I looked at matplotlib as it is used in sphinxext-opengraph, but the matplotlib's docs recommend using pillow to open images. So, I moved away from the matplotlib idea entirely.

[jbms]

What sort of information do you intend to include in the card? Just the title?

It looks like the upstream card examples show the page's icon also being included in the card --- page icons is another feature we've discussed but haven't implemented. But that would presumably also require svg support.

In general it seems like svg support may be important.

An alternative approach to generating the cards could be to generate an actual screenshot of the page, e.g. using Selenium or other browser automation. But that may be rather slow.

[2bndy5]

Well, the basic features in upstream use the the site name (Sphinx' project confval), page title, and site description. Locally, I've added both title and description to the supported metadata fields. The card config will be used to fill in the rest, including a default description when the meta data is not specified.

As for the icon, it is true that upstream uses a page icon embedded in the background for the new default layout (an insider feature). But the public edition only uses the Sphinx equivalent to html_logo.

In the social_cards config, I plan to offer a way to use an alternate logo. But, we may need to find a better way to support SVG->PNG conversion if/when I implement the card layout approach (as used by upstream insiders edition).

My rough draft is currently generating this:

But notice I haven't solved the kerning/spacing of the differently sized texts, hence the debugging grey box shown.

[2bndy5]

page icons is another feature we've discussed but haven't implemented

I've made some headway here on the nav-roles branch. But I haven't implemented the icon metadata field yet. In fact, that is how I stumbled into this adventure.

[2bndy5]

I have had some past experience using ImageMagick to convert images to whatever I want. I looked at what python bindings are available and I found WandPy which requires ImageMagick be installed separately. Since ImageMagick is cross-platform software (and rather easy to install), I think this would be a feasible solution (more so than cairosvg).

Alternatively, we could simply subprocess.call() ImageMagick's convert CLI program directly since that's all we'd really need. The good news is that ImageMagick comes installed in the github runner images, so we don't need any additional CI steps.

[jbms]

Is the goal to just implement the general concept of a social card, or to closely follow mkdocs-material?

I don't have a lot of experience with social cards, but it seems to me that by the far the most flexible option would be to generate an HTML document for the card, and then convert that to an image. That would allow styling with CSS, and rendering anything that can be rendered into the documentation itself; otherwise you are basically implementing your own document layout system. The downside to HTML is that it will require a browser to render and is likely slower than other methods.

[2bndy5]

Pillow does have a screenshot function, but I doubt that idea would work without a desktop env (like in a CI workflow). I think screenshots of a web page with this theme would look rather unintelligible in a social media preview. I know it sounds like reinventing the wheel, but using pillow really isn't that hard. So, a basic template system (with a limited set of options) is still feasible.

I also don't have a lot of experience creating social media cards. I usually just paste an image and rely on the social media app to show the title of the site. This is really why I'm following the lead from upstream.

[2bndy5]

We could also maybe use inkscape's CLI to deal with SVG->PNG conversion.

Another rather far fetched idea is to write our own SVG plugin for PIL. This seems like a lot more work just to open and convert SVG images.

EDIT: I see someone took the same initiative already: https://github.com/gribbg/pillow_svg

[jbms]

Selenium (which provides a Python API for controlling browsers) includes a screenshot function that captures just the browser viewport, not the surrounding browser UI. Selenium can also be used with a headless browser, like headless chrome, to generate screenshots without actually displaying the browser.

There is also this command line tool specifically designed for the purpose:
https://github.com/wkhtmltopdf/wkhtmltopdf

I agree that merely capturing a screenshot of the existing generated documentation page would likely not produce a very satisfactory result for the social card. Instead I was imagining generating a special HTML page specifically for the social card, and capturing an image of that using a specific viewport size.

[2bndy5]

I think the real problem with that capturing idea is that it is too extensive. For example, JS manipulation of the DOM (like mermaid.js) may be problematic if added to the template, especially if the user expects to capture animation which doesn't seem to be a thing for social media previews. I just feel like it implements more than we need.

I think what squidfunc did is preferable because it limits the rendition's scope to that expected of a social media preview card (which is some images/color/text), rather than a full-blown HTML screengrab.

I haven't mentioned this idea yet, but I think it would be useful to have a directive that can generate a social media card and add it to the page as an image. This would make it easier to document the layered approach (instead of adding a bunch of static images to the docs' src like what was done upstream).

[jbms]

Agreed that animation probably doesn't make sense. But including math expressions, code highlighting, bold or italic text, and possibly a static mermaid diagram or graphviz diagram, seems potentially useful to me. If the card is based on html then the content can be specified with rST syntax and the existing sphinx html translator can be used. Otherwise each such feature would require a separate implementation.

[2bndy5]

There is also this command line tool specifically designed for the purpose:
https://github.com/wkhtmltopdf/wkhtmltopdf

The linked repo is archived now. Is there an actively maintained alternative? I'm not familiar with the software.

I agree that the snapshot approach would be slower since we'd have to wait until the browser page is fully loaded (after mathjax or mermaid is finished) to capture the rendered result. This can add up if using the apidoc exts because each member uses a different page.

I also found a bug in which the py-modindex.html and genindex.html have "None" for the page title.

[jbms]

Oh, I found wkhtmltopdf from a stackoverflow post, but I haven't actually ever used it, and I see that it is no longer maintained and the authors recommend not to use it: https://wkhtmltopdf.org/status.html#recommendations

I think there are potentially a few ways to speed up capturing screenshots:

• Create one big webpage containing the social cards of many or all pages (all of fixed size) then capture one screenshot and split it up afterwards.
• Use parallelism --- can use more than one browser instance in parallel.

These two methods could also be combined.

Anyway, to be clear, I'm not saying that the HTML approach is the only option, it is just what seems like the best option to me.

[2bndy5]

I understand the benefit of having those options, but I'm just not personally invested in them. Bold/italic fonts can be rendered in pillow with ease. Usually when I see highlighted code or math expressions in social cards, they are from a static image (typically part of a logo). I've never seen a graph used in one.

These social cards don't get a lot of screen real-estate when loaded in social media apps, so I just figured a simpler design was the way to go ("less is more" ideology).

[2bndy5]

It would probably be a good idea to get feedback from someone that actually uses social media (I don't). 🤣

[2bndy5]

I've started pursuing my idea as a separate extension that should work for any (HTML-based) sphinx theme. If this theme is installed, then my new ext will still allow using some of the things specific to this theme (like the bundled icons). But I'm getting fonts from fontsource and caching them independently from this theme's configured location of cached fonts.

This issue can be closed since there are no concrete plans (that I'm aware of) to implement the browser snapshot approach. If that idea is attempted, then this can be re-opened (or a new separate issue can be used).

[2bndy5]

Just found this python library that might do what's needed for the browser snapshot idea: playwright-python