macOS: serapi loads Coq .vo from compile time path

[MSoegtropIMC]

@ejgallego : I have an issue with serapi on MacOS: serapi wants to load Coq .vo files from a compile time path.

On Windows, when running serapi from a binary installation I see paths like:

(Feedback((doc_id 0)(span_id 0)(route 0)(contents(FileLoaded Coq.Init.Prelude"C:\\Coq\\lib\\coq\\theories\\Init\\Prelude.vo"))))

On Linux snap I see:

(Feedback((doc_id 0)(span_id 0)(route 0)(contents(FileLoaded Coq.Init.Prelude /snap/coq-prover/x1/coq-platform/lib/coq/theories/Init/Prelude.vo))))

On MacOS, running serapi from a folder under /Applications with with env | grep COQ empty, I get:

(Feedback((doc_id 0)(span_id 0)(route 0)(contents(FileLoaded Coq.Init.Prelude /Users/msoegtrop/.opam/CP.2023.11.0~8.18~2023.11/lib/coq/theories/Init/Prelude.vo))))

On Github runners, where this folder doesn't exists any more when the smoke test is run, I get this error:

sertop: internal error, uncaught exception:
        Synterp.UnmappedLibrary(0, _)

I wonder what the difference in sertop is between MacOS and Windows/Linux. I can understand that Windows has special handling, but why is MacOS and Linux different?

I now spent already > 4 weeks to get sertop running on all platforms (honestly sertop took more effort than all other packages in Coq Platform together) and need to get this running soon or disable serapi on MacOS.

[ejgallego]

Hi @MSoegtropIMC , I'm sorry it took you 4 months to get sertop running, I was not aware you were having problems, I guess most of the time is due to path length issues? Is there any other issue that I'm not aware of?

I think the difference in this case is that the Linux snap is likely setting COQLIB whereas in OSX you are using a different relocation method?

sertop determines the path for coqlib in an almost standard way:

• command line parameter --coqlib
• search for COQLIB env var
• if not use the path passed to configure

So I'd suggest you set COQLIB for the smoke test for now, IMHO that's an acceptable workaround for users too if the auto-detection is not working properly.

As I've mentioned some other times, I'd be great to refactor sertop and Coq's init code so they share all this logic. As of today this is not the case as sertop added its own init code back in the day when Coq's API didn't allow to be used.

Later, Coq API was refactored to support different methods, but it was done without coordination with the other toplevels. So as of today some coding needs to happen to unify this, but I'm afraid I don't have the resources to take care of that myself :(

[MSoegtropIMC]

Yes, most of the issues were path length issues on Windows, and this time it really required quite a bit of restructuring of the Coq Platform path structure to make it work (with 1 character margin left). If the paths get any longer - without fixing in either OCaml or dune - I need to disable it on Windows.

I am quite sure that COQLIB is not set in the Windows and Mac case, but indeed I can't exclude that it is set in the snap case via the alias mechanism - it is quite possible. So this explains the difference between MacOS and Linux snap, but it doesn't explain why it works on Windows. Is it possible that you have special handling on Windows?

[ejgallego]

We don't have anything specific for Windows, but maybe you update Coq configure in a different way?

That would explain the difference.

[MSoegtropIMC]

I am using opam to install coq and serapi, so nothing special I am aware of.

Also on all platforms we are talking about relocated installations, where the final path does not match the configure path.

For coqc on Windows this works by an environment file which is written by the installer, but serapi does not use this (afaik). Also there is a mechanism in ocamlfind to find libraries after relocation (as discussed), but this does not influence finding .vo files.

[ejgallego]

Next SerAPI release will be the last, so IMHO not worth fixing this.

SerAPI has been now replaced by all purposes by coq-lsp, which should integrate with the platform better.