Installing on Ubuntu

[ldesousa]

Hi there,

I just followed the install instructions on Ubuntu 20.04. Unfortunately it did not work, either something happened during install (although no obvious error message is shown), or pandoc fails to find the filter. Could some environmental variable be missing?

The install log is below. Thank you for your help.

$ sudo aptitude install haskell-stack
The following NEW packages will be installed:
  haskell-stack 
0 packages upgraded, 1 newly installed, 0 to remove and 9 not upgraded.
Need to get 7 084 kB of archives. After unpacking 51,7 MB will be used.
Get: 1 http://fr.archive.ubuntu.com/ubuntu focal/universe amd64 haskell-stack amd64 1.9.3.1-1 [7 084 kB]
Fetched 7 084 kB in 2s (4 180 kB/s)        
Selecting previously unselected package haskell-stack.
(Reading database ... 507691 files and directories currently installed.)
Preparing to unpack .../haskell-stack_1.9.3.1-1_amd64.deb ...
Unpacking haskell-stack (1.9.3.1-1) ...
Setting up haskell-stack (1.9.3.1-1) ...
                                         
$ cd git

$ git clone https://github.com/lierdakil/pandoc-crossref.git
Cloning into 'pandoc-crossref'...
remote: Enumerating objects: 6281, done.
remote: Counting objects: 100% (753/753), done.
remote: Compressing objects: 100% (422/422), done.
remote: Total 6281 (delta 323), reused 600 (delta 199), pack-reused 5528
Receiving objects: 100% (6281/6281), 1.80 MiB | 4.50 MiB/s, done.
Resolving deltas: 100% (3313/3313), done.

$ cd pandoc-crossref

$ git checkout v0.3.10.0
Note: switching to 'v0.3.10.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at cd4d134 Update docs

$ stack install
Downloading lts-17.5 build plan ...
RedownloadInvalidResponse Request {
  host                 = "raw.githubusercontent.com"
  port                 = 443
  secure               = True
  requestHeaders       = []
  path                 = "/fpco/lts-haskell/master//lts-17.5.yaml"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 "/home/duque004/.stack/build-plan/lts-17.5.yaml" (Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}, responseVersion = HTTP/1.1, responseHeaders = [("Connection","keep-alive"),("Content-Length","14"),("Content-Security-Policy","default-src 'none'; style-src 'unsafe-inline'; sandbox"),("Strict-Transport-Security","max-age=31536000"),("X-Content-Type-Options","nosniff"),("X-Frame-Options","deny"),("X-XSS-Protection","1; mode=block"),("Content-Type","text/plain; charset=utf-8"),("X-GitHub-Request-Id","2CCC:CE61:6BAF9:73405:60914026"),("Accept-Ranges","bytes"),("Date","Tue, 04 May 2021 12:37:58 GMT"),("Via","1.1 varnish"),("X-Served-By","cache-ams21054-AMS"),("X-Cache","MISS"),("X-Cache-Hits","0"),("X-Timer","S1620131878.010057,VS0,VE158"),("Vary","Authorization,Accept-Encoding"),("Access-Control-Allow-Origin","*"),("X-Fastly-Request-ID","1ed53b5801abf988a594bbdcafb27a9a44ac1513"),("Expires","Tue, 04 May 2021 12:42:58 GMT"),("Source-Age","0")], responseBody = (), responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose})

$ cd ../the.document/

$ pandoc main.tex --filter pandoc-crossref -o main.docx
Error running filter pandoc-crossref:
Could not find executable pandoc-crossref

[lierdakil]

RedownloadInvalidResponse Request {
  host                 = "raw.githubusercontent.com"
  port                 = 443
  secure               = True
  requestHeaders       = []
  path                 = "/fpco/lts-haskell/master//lts-17.5.yaml"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 "/home/duque004/.stack/build-plan/lts-17.5.yaml" (Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}, responseVersion = HTTP/1.1, responseHeaders = [("Connection","keep-alive"),("Content-Length","14"),("Content-Security-Policy","default-src 'none'; style-src 'unsafe-inline'; sandbox"),("Strict-Transport-Security","max-age=31536000"),("X-Content-Type-Options","nosniff"),("X-Frame-Options","deny"),("X-XSS-Protection","1; mode=block"),("Content-Type","text/plain; charset=utf-8"),("X-GitHub-Request-Id","2CCC:CE61:6BAF9:73405:60914026"),("Accept-Ranges","bytes"),("Date","Tue, 04 May 2021 12:37:58 GMT"),("Via","1.1 varnish"),("X-Served-By","cache-ams21054-AMS"),("X-Cache","MISS"),("X-Cache-Hits","0"),("X-Timer","S1620131878.010057,VS0,VE158"),("Vary","Authorization,Accept-Encoding"),("Access-Control-Allow-Origin","*"),("X-Fastly-Request-ID","1ed53b5801abf988a594bbdcafb27a9a44ac1513"),("Expires","Tue, 04 May 2021 12:42:58 GMT"),("Source-Age","0")], responseBody = (), responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose})

this is actually an error message.

Your stack version is severely outdated. The current version is 2.5.1 (with 2.7 being around the corner). Your repository has version 1.9.3.1. It's been released near the end of 2018, so... yeah, a while ago.

[lierdakil]

I suggest you upgrade stack. You might also try tweaking stack.yaml, replacing resolver: lts-17.5 with

resolver:
  url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/17/5.yaml

This might work on older stack release. Let me know how it goes.

[lierdakil]

But also, not to seem pushy, but prebuilt binaries are an option: https://github.com/lierdakil/pandoc-crossref/releases/tag/v0.3.10.0a ; way, way faster than building everything. The from-scratch build takes about 40-45 minutes on CI, that's why I'm bothering with the prebuilts at all, building is a pita.

[ldesousa]

Hi @lierdakil thank you for your comments.

I managed to upgrade stack, as explained its own install instructions:

$ stack upgrade --binary-only

After that the compilation succeeded, taking about half an hour to complete. I then gave another go at the filter:

$ pandoc main.tex --filter pandoc-crossref -o main.docx
WARNING: pandoc-crossref was compiled with pandoc 2.12 but is being run through 2.5. This is not supported. Strange things may (and likely will) happen silently.
pandoc-crossref: Error in $: Incompatible API versions: encoded with [1,17,5,4] but attempted to decode with [1,22].
CallStack (from HasCallStack):
  error, called at src/Text/Pandoc/JSON.hs:112:48 in pandoc-types-1.22-DEMfXzmoq28KGFEtrrRzE:Text.Pandoc.JSON
Error running filter pandoc-crossref:
Filter returned error status 1

The filter got compiled with a version of pandoc different from what I have in my system:

$ apt policy pandoc
pandoc:
  Installed: 2.5-3build2
  Candidate: 2.5-3build2
  Version table:
 *** 2.5-3build2 500
        500 http://fr.archive.ubuntu.com/ubuntu focal/universe amd64 Packages
        100 /var/lib/dpkg/status

So the question now is: how can I make sure the filter is compiled with the correct version of pandoc?

[lierdakil]

stack has actually built pandoc 2.12 along with pandoc-crossref, just didn't install it (although bear in mind that the latest pandoc version is actually 2.13, and the relevant stack config is in last tag, v0.3.10.0a). You can install it into ~/.local/bin via stack install pandoc from the pandoc-crossref project directory (this bit is important, otherwise stack will build whatever pandoc version it wants, which isn't necessarily the same you want).

If you really want to build against pandoc 2.5, I think pandoc-crossref v0.3.4.2 is the last version to support that (to be clear, it's 2 years old).

[ldesousa]

Thank you @lierdakil that was really fast. I started by tweaking stack.yaml but rapidly came to the conclusion I need to compile a different version of the filter, as you state.

I would like to stick with a packaged version of pandoc, but I am not ruling out upgrading from 2.5. I will report back further findings.

[ldesousa]

I managed to compile the filter putting together the information in this thread. The complete method is digested in this answer.

In the end I stuck with the packaged version of pandoc (2.5). The alternative implies relying on various binaries compiled by third parties.

[lierdakil]

The alternative implies relying on various binaries compiled by third parties.

Unless you call yourself a "third party" you can build everything yourself, including the toolchain. It's a huge PITA, but far from impossible. Why you'd choose to go that route is beyond me, but to each their own I guess.