docgen as a preprocessor, fixes #193

[seanhess]

Switched docgen to be a preprocessor, and updated build-tools to depend on it. This works out of the box for cabal build and cabal haddock, but unfortunately requires manaully installing the preprocessor to your PATH for HLS to work. I can't think of a workaround.

I suppose we could easily add a line to the developer instructions in the README to remind people to run:

cabal install docs:docgen

[seanhess]

@sectore what needs to happen to get a local preprocessor working for nix? Note that cabal build works without any extra steps, but HLS fails unless you manually install it:

cabal install docgen:docgen

[sectore]

Just FYI: With latest commits in current PR, Nix users will have docgen on PATH by entering nix develop shell by default. Not sure if its worth to mention it in dev.md as this document is for non Nix users only.

[sectore]

Look at this ⬇️ Nix builds take only 1-3min now. Cachix does a nice job.

[sectore]

Please double check packcheck/ci-sdist ..., which is still failing (here). It expects docgen:docgen, but that's a package available locally only?

[seanhess]

I don't understand why sdist is failing. Packcheck is supposed to be reproducable when you run it locally, but it works absolutely fine on my laptop. I can't figure out why the sdist check would handle docgen differently

Just FYI: With latest commits in current PR, Nix users will have docgen on PATH by entering nix develop shell by default. Not sure if its worth to mention it in dev.md as this document is for non Nix users only.

That's great! I'll add a note that it's only for non-nix users. Actually, it probably makes sense to have completely separate nix and non-nix dev documents. I assume other things in dev.md don't apply? Certainly not brew install npm.

[seanhess]

Grrr, it doesn't look like this is going to work: composewell/packcheck#121 (comment)

[sectore]

Grrr, it doesn't look like this is going to work: composewell/packcheck#121 (comment)

Nice!

Actually, it probably makes sense to have completely separate nix and non-nix dev documents. I assume other things in dev.md don't apply? Certainly not brew install npm.

Yes and yes.

Please try to bring commits 13af33c and 4242287 back into this branch. It seems they are lost by a previous force push. They are parts of my previous commits I have done before here.

13af33c (HEAD -> docgen, origin/docgen) nix: `docgen` on PATH
4242287 fix(nix): preprocessor failed

[seanhess]

Please try to bring commits 13af33c and 4242287 back into this branch. It seems they are lost by a previous force push. They are parts of my previous commits I have done before here.

Oh, I'm sorry, I didn't realize you had pushed to this branch. I was confused how I got out of sync.

Nice!

Ha, no, it's bad news. I'm not sure the local pre-processor approach will work at all. Maybe the right thing to do is to publish the preprocessor to hackage or research how other people are accomplishing the same thing. Having type-checked examples in your haddock is incredibility useful, I can't be the only one who wants to do that.

[seanhess]

Our options are as follows: (See this reddit thread)

1. Switch to using doctest
2. Publish docgen to hackage

Doctest is more standard, and we don't have to publish anything. Our docgen produces better output, and doesn't break HLS editor integration to find errors in the examples.

I'm leaning towards publishing docgen to hackage, leaving it specific to hyperbole, and namespacing it as hyperbole-docgen. That's the least amount of work, and I really like having the examples typechecked in real haskell modules

[seanhess]

@sectore Could you help me repair the nix config? I decided we should publish hyperbole:docgen. Local builds can still depend on it via the cabal.project file, while sdist can rely on the published version.

Specifically I changed:

1. Removed docs/docgen.cabal (via package.yaml)
2. Added docgen executable to hyperbole.cabal
3. Renamed docs/Main.hs to docs/Docgen.hs
4. Removed ./docs from cabal.project, since it is now local

[seanhess]

@sectore See above. I don't want this branch to get too out of date before merging

[sectore]

@sectore Could you help me repair the nix config?

Do current changes work for you? packcheck/ci-sdist seems still to be failing at CI.

I have tried to fix Nix, but it's not that easy... Now we have a docgen executable defined in hyperbole.cabal pointing to another folder. Besides that a symlink approach of docs folder to be available in demo ... Nix goes another way than Cabal.

Before I continue to try to make Nix work, could you make packcheck/ci-sdist running "green" first, pls?

[seanhess]

Do current changes work for you? packcheck/ci-sdist seems still to be failing at CI.

That's expected. With the current approach, it can't be fixed until we cut a release. It needs a hyperbole:docgen published to hackage.

[sectore]

With the current approach, it can't be fixed until we cut a release. It needs a hyperbole:docgen published to hackage.

Why not publishing a simple package hyperbole-docgen to Hackage (instead of adding it hyperbole)?

No another hyperbole release needed then. packcheck/ci-sdist can then be tested easily at CI, too. Also things on Nix side will be simplified by just loading such a docgen package from Hackage.