Pandoc haskell filter

[samhedin]

Pandoc filters are primarily written in Haskell or Lua. The first version of the rustic-doc filter was written in Haskell. The first comments I received were that I should port the filter to lua, to make it easier for contributors as lua was seen as an easier language than Haskell. I listened to the advice and ported it to lua, but in hindsight, it was a mistake.
The lua filter is way messier and harder to interpret, and it has made working in the filter rather painful. This PR is restoring the old Haskell filter, and making some related changes along the way.

• Pandoc is now installed automatically. Recently some issues related to versioning were discovered (particularly on debian based distros), and by installing our own local version of pandoc we know everyone has a compatible version. Pandoc is tiny enough and entirely self-contained, so I'm just placing the binary next to the other required resources. This also ensures compatibility with the filter.
• To make distribution simple, I added the 38 MB Haskell binary to git. I previously used github releases to distribute it, and I'm fine with doing that or something else entirely - I know some wince at the thought of tracking binaries.
• Despite not being touched for 1.5 years, the Haskell filter generates better-looking org documents than the Lua filter ever did.

[psibi]

I guess we should fix the various things in the file...

[psibi]

To make distribution simple, I added the 38 MB Haskell binary to git. I previously used github releases to distribute it, and I'm fine with doing that or something else entirely - I know some wince at the thought of tracking binaries.

Is it a static binary ? I'm concerned if it will work on all distributions.

[samhedin]

To make distribution simple, I added the 38 MB Haskell binary to git. I previously used github releases to distribute it, and I'm fine with doing that or something else entirely - I know some wince at the thought of tracking binaries.

Is it a static binary ? I'm concerned if it will work on all distributions.

It should be, except for glibc, which I painfully realized last night before going to bed. It seems possible to statically link musl instead though, I will look into it as that part is essential.

[psibi]

It seems possible to statically link musl instead though, I will look into it as that part is essential.

One option is to use this project to build musl based executable: https://github.com/fpco/alpine-haskell-stack

But it seems it still doesn't support GHC 8.10.7 (since you are using Stackage LTS 18.23), but supports 8.10.6. But I believe adding support for 8.10.7 shouldn't be hard.

To make distribution simple, I added the 38 MB Haskell binary to git. I previously used github releases to distribute it, and I'm fine with doing that or something else entirely - I know some wince at the thought of tracking binaries.

Yeah, I think distributing it via github releases would be probably better. :-)

[samhedin]

It seems possible to statically link musl instead though, I will look into it as that part is essential.

One option is to use this project to build musl based executable: https://github.com/fpco/alpine-haskell-stack

But it seems it still doesn't support GHC 8.10.7 (since you are using Stackage LTS 18.23), but supports 8.10.6. But I believe adding support for 8.10.7 shouldn't be hard.

We can use different versions of GHC, so no worries there. I'll try tonight/in the coming days.

[samhedin]

As I was looking at this I found rust-lang/rust#76578
Seems like this started just a month or two after I began working on rustic-doc. Using JSON would be a much more robust solution.

[samhedin]

I gave it a try this week and kind of ran into issues, listed here: https://www.reddit.com/r/haskell/comments/siktmp/building_a_standalone_haskell_binary_with/
The most straightforward way now seems to be to do it properly and distribute it as a regular package, but I'm hesitant to introduce more steps to the installation process.

[psibi]

I haven't looked the error message in detail, but if you are looking for an example - here is a Dockerfile I create to create static binary of Haskell's tldr program: https://github.com/psibi/tldr-hs/blob/master/Dockerfile Do you face the same issue if you try the same method ?

[samhedin]

I haven't looked the error message in detail, but if you are looking for an example - here is a Dockerfile I create to create static binary of Haskell's tldr program: https://github.com/psibi/tldr-hs/blob/master/Dockerfile Do you face the same issue if you try the same method ?

That seems like a much more robust solution, thank you! How do you run it? When I clone the repo and write
docker build -t tldr . I get

Getting project config file from STACK_YAML environment
/app/tldr.cabal was modified manually. Ignoring /app/package.yaml in favor of the cabal file.
If you want to use the package.yaml file instead of the cabal file,
then please delete the cabal file.
No setup information found for ghc-8.10.7 on your platform.
This probably means a GHC bindist has not yet been added for OS key 'linux64-ncurses6'.
Supported versions: ghc-7.10.3, ghc-8.0.1, ghc-8.0.2, ghc-8.2.1, ghc-8.2.2
``

[psibi]

@samhedin You can see that the CI script builds the image: https://github.com/psibi/tldr-hs/blob/master/.azure/azure-linux-musl.yml#L16

And finally it produces a musl based binary here: https://github.com/psibi/tldr-hs/releases/tag/v0.9.2

The idea is that you build the source code inside the docker image. You can even do it in a much better than I have done by using a multi stage docker build and making it much faster.

[samhedin]

As you may have noticed, my energy and time for this come sporadically. The reason is that I don't use Rust anymore, and haven't for a very long time.
Closing this for now.