Rethink or remove Linux's FHS support

[alaviss]

Right now FHS supports come from hardcoded snippets like these:

nimskull/compiler/options.nim

Lines 708 to 716 in 80108d5

	when defined(posix):
	if prefix == AbsoluteDir"/usr":
	conf.libpath = AbsoluteDir"/usr/lib/nim"
	elif prefix == AbsoluteDir"/usr/local":
	conf.libpath = AbsoluteDir"/usr/local/lib/nim"
	else:
	conf.libpath = prefix / RelativeDir"lib"
	else:
	conf.libpath = prefix / RelativeDir"lib"

nimskull/compiler/nimconf.nim

Lines 237 to 239 in 989adc6

	when defined(unix):
	if not fileExists(result): result = p / RelativeDir"etc/nim" / filename
	if not fileExists(result): result = AbsoluteDir"/etc/nim" / filename

This is error prone, and is not supported by some format strings like $nim where programs out of tree have learnt to utilize.

To reduce the maintenance burden we have to decide on one layout to support and do it well. So I guess either we rework the current layout into something that can be put in a /usr folder, or we just drop the idea altogether and mandate the exact layout as in a binary archive (which is the source layout).

[saem]

Is the bigger problem instead the fact that we don't decide it once in a single place?

[alaviss]

wdym?

[alaviss]

The first thing to do is to figure out what should be expected to be available in a compiler installation and what should not be. Right now we pack these in binary releases:

• Configuration
• Standard library
• Icons for Windows
• Documentation (source RST)
• Binaries
  • nim-gdb
  • nim
  • nim_dbg (compiler debug version)
  • nimgrep
  • nimpretty
  • nimsuggest
  • testament
• Sources
  • The full testsuite
  • The compiler source
  • nimpretty
  • nimsuggest
  • testament
  • The tools folder
  • koch

I think we should cut this to:

• Configuration
• Standard library
• Documentation (source RST and/or rendered HTML)
• Binaries (and their dependencies ie. tools/nim-gdb.py for nim-gdb)

As a start for organizing the structure.

[alaviss]

A feature to consider removing is the $nim format string, which points to the installation prefix of Nim, or more commonly as it is used, the root of the compiler source tree. This format string is used only in nimfix build configuration.

The reason to remove this format string is to prevent third-party developers from depending on the installation layout of the compiler, enabling us some more flexibility in shuffling these files around.

[alaviss]

Right now this format string is often used to get the compiler source matching the exact version of the compiler.

We will have to support this via other means if we want to remove this implementation detail.

[juancarlospaco]

Go install Python in Windows, installer asks user:

• Install Documentation ?.
• Install Unittests ?.
• Install for all users ?.
• Make Backups ?.

etc etc you get the idea...

[CyberTailor]

Don't remove FHS support if you ever want Nimskull in *nix package repositories.

[alaviss]

I think FHS can only be supported if:

• The layout is used and tested by developers
• The layout can be used on Windows without relying on a distribution like MSYS2

Right now since builds are in-tree, the only layout actually being tested is the source layout. This layout can be packaged into a FHS-style layout rather easily. There are only two things you'd have to do:

1. Symlink all binaries in bin/ to /usr/bin
2. Symlink config/ folder to /etc/nimskull

Now it doesn't matter where you place the rest of the files. They can be in /usr/lib/nimskull or /opt/nimskull depending on how you interpret FHS.

This is why I proposed to remove FHS support altogether as it's easy to make a faux FHS out of the current layout. And that the "support" is not even that great with docgen not working at all.

Now suppose we really want FHS, then we can do it like Rust:

• For development, the compiler is automatically installed into a fake root layout. This way the layout is always tested regardless of the source layout.
• Make the prefix the discovery root, ie:
  • bin/../lib/nimskull is where stdlib is
  • bin/../share/doc is where docs are
  • bin/../etc/nimskull for configuration (or /etc if bin/../etc is not found)

This would solve the two problems I raised, though adds a few:

• The build system needs to support it. This can be done with enough effort, only koch and niminst would have to adapt for the most part.
• It'd disrupt the current stdlib development flow (edit, compile, repeat becomes edit, build new compiler, compile, repeat). Mostly a learning curve for developers.

The benefits as I see it right now are questionable at best, especially when the FHS can be recreated without much effort.

[CyberTailor]

The layout is used and tested by developers

Maintainers of distro repositories will send the patches if necessary. So it can be tested downstream.

[alaviss]

I find downstream testing to be unreliable, and if downstream messed up, users will come here with issues we can't reproduce, like https://bugs.archlinux.org/task/67092?project=5&string=nim (this was submitted by a Nim dev after repeated complains from Arch users about cross-compiling).

And 3rd party software has been known to rely on the compiler installation layout as that's what used by the toolchain installer (a necessity for a growing language as distro updates are slow). Two examples of this is nimph and nimlsp, both will not compile if you install compiler with the FHS-style layout as implemented currently.

And downstream maintainers don't just maintain this one package, they might maintain anywhere between 10 - 30 and won't be able to provide adequate QA.

If FHS were to be supported natively, it must be tested upstream and developers must use it by default, or else it's gonna be a broken mess. This already happened with docgen refactoring which introduced a dependency on the compiler prefix and doesn't work when packaged in FHS mode downstream, for which Debian's solution was to hardcode their path and of course was not upstreamable.