User Guide PDF should support special characters

[marcphilipp]

Special characters (e.g., emoji from custom display names and glyphs used with unicode mode for the ConsoleLauncher) are currently broken in the generated PDF.

See discussion in asciidoctor/asciidoctor-pdf#323.

See #770 (comment).

Deliverables

• All special characters are displayed correctly in the PDF version of User Guide

[mojavelinux]

FYI, Asciidoctor PDF 1.5.x bundles an emoji font you can declare in your theme. It's also included in the default-with-fallback-font theme. See https://github.com/asciidoctor/asciidoctor-pdf/blob/v1.5.x/data/themes/default-with-fallback-font-theme.yml

[marcphilipp]

@mojavelinux Thanks for the pointer! I've tried using that (see 8cc8306) from the Gradle plugin but I'm running into a failure: https://ge.junit.org/s/7scjo3n42afqc/console-log?task=:documentation:asciidoctorPdf

Any ideas?

[mojavelinux]

That's not the result I got (using Java 11), so I'm not sure why you would be getting that failure. Here's what I did.

1. Upgrade to Asciidoctor PDF 1.5.3 (from 1.5.0.rc2, which is quite old)

   asciidoctorj {
       modules {
           diagram.use()
           pdf.version("1.5.3")
       }
   }
   

2. Set the pdf-theme in the document header of the AsciiDoc source file to use the default-with-fallback-font theme

   :pdf-theme: default-with-fallback-font
   

Here's the output you'll see:

Jun 10, 2020 1:33:57 AM uri:classloader:/gems/asciidoctor-pdf-1.5.3/lib/asciidoctor/pdf/converter.rb convert_listing_or_literal
WARNING: Could not locate the character `╯' in the following fonts: M+ 1mn, M+ 1p Fallback, Noto Emoji
Jun 10, 2020 1:34:11 AM uri:classloader:/gems/asciidoctor-2.0.10/lib/asciidoctor/abstract_block.rb content
WARNING: Could not locate the character `✔' in the following fonts: M+ 1mn, M+ 1p Fallback, Noto Emoji
Jun 10, 2020 1:34:26 AM uri:classloader:/gems/asciidoctor-2.0.10/lib/asciidoctor/document.rb content
WARNING: Could not locate the character `↷' in the following fonts: M+ 1mn, M+ 1p Fallback, Noto Emoji

Unfortunately, this indicates that these glyphs aren't included in the bundled fallback font (M+ 1p) or emoji font (Noto Emoji).

Therefore, it's necessary to create a custom theme file (junit-pdf-theme.yml) and point it at font that provides these characters, like Symbola.

extends: default
font:
  catalog:
    merge: true
    Symbola: /usr/share/fonts/gdouros-symbola/Symbola.ttf
  fallbacks: [Symbola]

Then add this line to the document header of the AsciiDoc source file:

:pdf-theme: {basedir}/junit-pdf-theme.yml

When I do that, I only end up with one warning:

Jun 10, 2020 1:48:38 AM uri:classloader:/gems/asciidoctor-pdf-1.5.3/lib/asciidoctor/pdf/converter.rb convert_listing_or_literal
WARNING: Could not locate the character `）' in the following fonts: M+ 1mn, Symbola

That's the from the table flip kaomoji, which uses a very rare character in a standard font (since it's from a Japanese charset, U+FF09).

[marcphilipp]

Thanks, I'll try that! We can probably get rid of the table flip kaomoji but might need a special fallback font like Symbola nevertheless.

@mojavelinux Is there a way to point to a font file using a relative path, e.g. in the Git repo, to keep the build portable?

[mojavelinux]

Is there a way to point to a font file using a relative path, e.g. in the Git repo, to keep the build portable?

Absolutely. See https://github.com/asciidoctor/asciidoctor-pdf/blob/v1.5.x/docs/theming-guide.adoc#custom-fonts and https://github.com/asciidoctor/asciidoctor-pdf/blob/v1.5.x/docs/theming-guide.adoc#applying-your-theme.