Emoji support

[nathany]

Is there any way to have Emoji show up in a PDF?

None of these show up correctly for me.

😕

😕

😕

From http://apps.timwhitlock.info/unicode/inspect/hex/1F615

Preferably any UTF-8 character would work as is without the entity encoding stuff.

[mojavelinux]

Ideally we would integrate the emoji extension, though it would need to output the unicode character, not the entity.

https://github.com/asciidoctor/asciidoctor-extensions-lab/blob/master/lib/emoji-inline-macro.rb

Asciidoctor PDF currently does not support arbitrary named or numbered entities. The text formatter has to have explicit support for converting these into unicode characters or else it doesn't get passed through.

The reason I haven't done it so far is because I'm hesitant about the proliferation of HTML constructs (entities) in AsciiDoc because it makes a certain assumption about the output format. Granted, it is possible to recognize these and convert them for the output (as we do already with the core named entities), but it at least gave me reason to pause and think about what's happening here. And maybe they already are a portable way of communicating high-order characters. But I'm still undecided.

[mojavelinux]

Btw, this would be a role of the text formatter: https://github.com/asciidoctor/asciidoctor-pdf/blob/master/lib/asciidoctor-pdf/formatted_text.rb

[nathany]

I don't actually want html entities. I'd much prefer the actual UTF-8 character going from source to PDF. Consider, for example, emoji characters in source code.

Maybe it's just a font thing?

[mojavelinux]

In that case, you can just put the character directly into your document. But yes, the font must support it. As I mention in the theming guide, it's "bring your own font". If the font doesn't have a glyph, you get a blank in the PDF.

See https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc#fonts

I intentionally subset the bundled fonts to keep the gem size reasonable. You are free to link in your own fonts that are full versions of the fonts...and I almost always recommend that you do once you get beyond just playing with Asciidoctor PDF.

[nathany]

Based on that information, shouldn't "M+ 1p Fallback" be responsible for including Emoji?

[mojavelinux]

Yes, but it may not have those characters. M+ 1p isn't a true fallback font, it just better than some of the regular ones and fits the metrics. A true fallback font is something like VL Gothic.

[nathany]

I'm not familiar with VL Gothic, but I tried Apple Color Emoji.ttf as a fallback. It still came out with blank spaces.

font:
  catalog:
    #...
    Apple Color Emoji:
      normal: Apple Color Emoji.ttf
      bold: Apple Color Emoji.ttf
      italic: Apple Color Emoji.ttf
      bold_italic: Apple Color Emoji.ttf
  fallbacks:
    - Apple Color Emoji

[mojavelinux]

I play around a bit and see if I can get you some more details.

[nathany]

Dan, any suggestions? Thanks.

[mojavelinux]

I haven't had a chance to look at it yet. Had a busy weekend. I'll try to get to it tonight.

[nathany]

Thanks. Much appreciated.

[mojavelinux]

Okay, I got it working using the OpenSans Emoji font with the following sample document:

When you're feeling a bit 😕, go take a walk.

You get get OpenSans Emoji from https://github.com/MorbZ/OpenSansEmoji

It's important to understand that configuring the theme is still a bit awkward, so you can't miss any steps or else it won't work.

The first thing you need to do is make a folder that has all the fonts you are going to use. Once you start customizing the fonts, you need to supply all the fonts (as it stands now). I recommend setting up a folder and copying the Noto Serif and M+ fonts there so that you can start with the defaults and add on.

Step 1: Put all the fonts in a fonts folder:

fonts/mplus1mn-bold-ascii.ttf
fonts/mplus1mn-bold_italic-ascii.ttf
fonts/mplus1mn-italic-ascii.ttf
fonts/mplus1mn-regular-ascii-conums.ttf
fonts/mplus1p-regular-fallback.ttf
fonts/notoserif-bold_italic-subset.ttf
fonts/notoserif-bold-subset.ttf
fonts/notoserif-italic-subset.ttf
fonts/notoserif-regular-subset.ttf

Step 2: Add the OpenSansEmoji.ttf font to the fonts folder:

fonts/OpenSansEmoji.ttf

Step 3: Include the emoji font in your theme alongside your normal fonts:

font:
  catalog:
    Noto Serif:
      normal: notoserif-regular-subset.ttf
      bold: notoserif-bold-subset.ttf
      italic: notoserif-italic-subset.ttf
      bold_italic: notoserif-bold_italic-subset.ttf
    M+ 1mn:
      normal: mplus1mn-regular-ascii-conums.ttf
      bold: mplus1mn-bold-ascii.ttf
      italic: mplus1mn-italic-ascii.ttf
      bold_italic: mplus1mn-bold_italic-ascii.ttf
    M+ 1p Fallback:
      normal: mplus1p-regular-fallback.ttf
      bold: mplus1p-regular-fallback.ttf
      italic: mplus1p-regular-fallback.ttf
      bold_italic: mplus1p-regular-fallback.ttf
    Emoji:
      normal: OpenSansEmoji.ttf
      bold: OpenSansEmoji.ttf
      italic: OpenSansEmoji.ttf
      bold_italic: OpenSansEmoji.ttf
  fallbacks:
    - M+ 1p Fallback
    - Emoji
base:
  font_family: Noto Serif
...

Step 4: Invoke asciidoctor-pdf, specifying both the pdf-style and pdf-fontsdir attributes:

asciidoctor-pdf -a pdf-style=./custom-theme.yml -a pdf-fontsdir=./fonts sample.adoc

If necessary, use absolute paths for both the pdf-style and the pdf-fontsdir.

If you get the message “The following text could not be fully converted to the Windows-1252 character set”, then that means you didn't assign a base_font_family to a TTF font (and hence the fallbacks won't be used).

If you still have trouble, it might be best to push up a sample repository on GitHub that we can experiment with.

[mojavelinux]

I don't recommend using Apple's color emoji because I can almost guarantee that the TTF library in Prawn is not advanced enough to process color correctly. I could be wrong, but that's my guess.

[mojavelinux]

You might also have success with Symbola as a fallback font. In that case, you might be able to drop M+ 1p. http://www.fontspace.com/unicode-fonts-for-ancient-scripts/symbola

[mojavelinux]

More open source emoji fonts are listed here: http://graphicdesign.stackexchange.com/questions/31625/are-there-any-free-emoji-fonts

[nathany]

Thanks. I'll give them a try in the morning.

[mojavelinux]

I found out that EmojiSymbols works, but you first have to convert the WOFF to TTF using fontforge. EmojiSymbols has a rather restricted license, though, so OpenSansEmoji is probably the best choice.

[mojavelinux]

I realize we still need to make it easier to deal with fonts in the custom theme. That's something slated for beta1.

[nathany]

Yah, I just copied the standard fonts into my theme since it doesn't look at both folders. #80

OpenSansEmoji works, and honestly colour Emoji wouldn't work in print anyway.

Thanks for helping me track down a working font.

[mojavelinux]

Excellent news. Victory! ✌️

[amejiarosario]

@mojavelinux Thanks for your insights. I found out that Noto Emoji also works, also tried with Noto Emoji Color but this one didn't work.

undefined method `key?' for nil:NilClass
  Use --trace for backtrace
make: *** [pdf] Error 1

[mojavelinux]

You are correct that color emoji don't work. It seems to be a limitation of ttfunk (the TTF library).