Pandoc failure when building in GitHub CI

[mgeisler]

In #1951, the PDF generation failed with:

2024-04-03 17:00:00 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
  Warning:  Could not convert image /tmp/tex2pdf.-b8d581420691af04/9b1e09f6511f387768da72145790446bd7e7cf63.vs: Cannot load file
    Jpeg Invalid marker used
    PNG Invalid PNG file, signature broken
    Bitmap Invalid Bitmap magic identifier
    GIF Invalid Gif signature : error 
    HDR Invalid radiance file signature
    Tiff Invalid endian tag value
    TGA not enough bytes
  Error producing PDF.
  ! LaTeX Error: Unknown graphics extension: .vs.
  
  See the LaTeX manual or LaTeX Companion for explanation.
  Type  H <return>  for immediate help.
   ...                                              
                                                    
  l.[16](https://github.com/google/comprehensive-rust/actions/runs/8488669494/job/23404502766?pr=1951#step:8:17)8 ...1e09f6511f387768da72145790446bd7e7cf63.vs}}
  
  pandoc exited unsuccessfull

It is the same error message in the run for German, Danish.

For Bengali, the error instead says

2024-04-03 17:00:09 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
  [2024-04-03T17:00:25Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'https://img.shields.io/github/actions/workflow/status/google/comprehensive-rust/build.yml?style=flat-square' in chapter 'Comprehensive Rust 🦀 এ আপনাকে স্বাগতম': Unable to load remote image 'https://img.shields.io/github/actions/workflow/status/google/comprehensive-rust/build.yml?style=flat-square': https://img.shields.io/github/actions/workflow/status/google/comprehensive-rust/build.yml?style=flat-square: status code 522
  Warning:  Could not convert image /tmp/tex2pdf.-e893803b8b925751/9b1e09f6511f387768da72145790446bd7e7cf63.yml: Cannot load file
    Jpeg Invalid marker used
    PNG Invalid PNG file, signature broken
    Bitmap Invalid Bitmap magic identifier
    GIF Invalid Gif signature : error 
    HDR Invalid radiance file signature
    Tiff Invalid endian tag value
    TGA not enough bytes
  Error producing PDF.
  ! LaTeX Error: Unknown graphics extension: .yml.
  
  See the LaTeX manual or LaTeX Companion for explanation.
  Type  H <return>  for immediate help.
   ...                                              
                                                    
  l.[16](https://github.com/google/comprehensive-rust/actions/runs/8488669494/job/23404501913?pr=1951#step:8:17)7 ...e09f6511f387768da72145790446bd7e7cf63.yml}}
  
  pandoc exited unsuccessfully

Note that it's a .yml file here, but .vs files above.

[mgeisler]

Hi @max-heller, before I start looking more into this, do you recognize this error off the top of your head? 🙂

Cc @henrif75 and @djmitche who looked at other PDF errors in #1911.

[max-heller]

Hi @max-heller, before I start looking more into this, do you recognize this error off the top of your head? 🙂

These have to do with Pandoc attempting to embed remote images in the PDF.

[2024-04-03T17:00:25Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'https://img.shields.io/github/actions/workflow/status/google/comprehensive-rust/build.yml?style=flat-square' in chapter 'Comprehensive Rust 🦀 এ আপনাকে স্বাগতম': Unable to load remote image 'https://img.shields.io/github/actions/workflow/status/google/comprehensive-rust/build.yml?style=flat-square': https://img.shields.io/github/actions/workflow/status/google/comprehensive-rust/build.yml?style=flat-square: status code 522
Warning:  Could not convert image /tmp/tex2pdf.-e893803b8b925751/9b1e09f6511f387768da72145790446bd7e7cf63.yml: Cannot load file

This error relates to this workaround mdbook-pandoc implements for shields.io badges. The 522 status code suggests a spurious error downloading the image.

Warning:  Could not convert image /tmp/tex2pdf.-b8d581420691af04/9b1e09f6511f387768da72145790446bd7e7cf63.vs: Cannot load file

I'm not sure about this one and can't reproduce it locally. I don't see any images with .vs extensions in the source. Let's try running with verbosity = "INFO" as shown in #1911 (comment)

[mgeisler]

Thanks @max-heller for the explanation!

Your analysis seems spot on: the error went away on the third run of the build.

It would be nice if we could make our build resilient to this somehow... Perhaps I should just remove the shields.io images from the source when generating the book. I don't think they fit very well into a PDF 🙂

[max-heller]

It would be nice if we could make our build resilient to this somehow

Agreed, although I'm not sure of a good general solution. Maybe mdbook-pandoc could strip out images it can't load and emit a loud warning?

[mgeisler]

Maybe mdbook-pandoc could strip out images it can't load and emit a loud warning?

Yeah, for this particular case that would work well. But it seems like the wrong place to add this kind of logic — I feel it should be the caller of mdbook-pandoc that implements the retry logic.

I think we can close this issue for now: it's a transient error and now we know that it should go away after a while. If it pops up again, then we can take another loop at implementing some kind of retries (or more likely: strip out the unreliable images).