NixOS Packaging

[Baughn]

Hi,

(This'll be a slightly longwinded feature request. Skip to the bottom if you prefer.)

I (very) recently packaged CKAN for NixOS. Well, practically that's still in progress; see NixOS/nixpkgs#15278.

That works. It autodetects the Steam-homed KSP installation, installs mods correctly, and so on. Don't know how I'd live without it. :)

However, like many players I'd like to maintain multiple KSP installs. Unlike most players, I can't do so without packaging KSP for NixOS, to some level or another. Steam uses a chroot that makes the distro looks like Ubuntu, but running KSP outside of Steam is otherwise not going to work at all; /lib and /usr don't even exist, let alone contain the libraries it expects.

So I need to fix that, and while I'm at it I'm going to add a kspWithMods package constructor. That will make it much easier for me to handle disparate installations, plus by the nature of things, the KSP directory is supposed to be read-only... save, I suppose for savegames. Time will tell how possible that is.

I don't actually need anything from you, emphasis on 'need'. I can make the package build script run ckan install without that being a problem. Unfortunately, doing so will make every rebuild re-download every mod, which is not exactly ideal; package builds always happen in a clean workspace, so CKAN's native caching cannot work.

Therefore, I'd like to request three features:

• A ckan download command that just downloads some mod, not including its dependencies, and drops it in CWD.
• A flag for ckan show which outputs the dependencies in machine-readable format. JSON is fine. Alternately, I guess I can try to parse the manifests myself. I'd offer to send a PR, but I've never touched C# before. :x
• And an option for ckan install which looks for files downloaded to CWD before re-downloading. Well, basically a flag to select the cache directory would be fine.

The basic idea being that, if a user requests four separate mods, I can construct packages for each of those four mods such that the Nix store will cache them, before installing them all in to KSP install being built.

[Baughn]

As an aside, make test fails because package builds happen as a sandbox user, with $HOME set to /var/empty. As I said, I'm not familiar with C#... do you know what I'd need to do to make it use a tempdir instead?

Setting HOME in the script does nothing; presumably it looks up the homedir some other way.

[politas]

Would #1699 solve the first and/or third feature requests?

[Baughn]

It would, if I can set the location without using the GUI. It can't simply be a location relative to home, as home is /var/empty and unwriteable. A flag would be ideal.

(CKAN appears to have no flags whatsoever at the moment, but I'm imagining something like "--store-path=/foo/wherever". One potential snag is that I'd also need the ability to concatenate stores, which is trivial if they're just directories with tarballs, but difficult if there's a registry.)

The reason for #2 is that Nix is somewhat opinionated. I'll be storing mods in "derivations", which are named by hash of the configuration and code which creates them, so it'd be very hard to avoid downloading some mods multiple times unless I can store them individually.

[politas]

There is a --cachedir <path> option on netkan, but I'm not sure if that has been carried over to CKAN.

[politas]

Could your script for creating a Steam+KSP install make a symlink from a world-writeable folder to SteamLibrary/steamapps/common/KerbalSpace Program/CKAN/downloads? That would share your downloads across KSP installs, and is basically how most people running mutiple installs are doing it now.

[Baughn]

Unfortunately, no. Nix operates at a system-wide level, so I can't access the homedir of any particular user while building packages, not even the user who requested the build.

[Baughn]

It might be best if I bypass CKAN for the initial build, read the manifests with custom code, and then only use the ckan binary itself to do the final install. I'd still need some way of telling it where to find the files it's installing, though.

[politas]

Can you take the KSP from Steam and built a package for it that doesn't involve Steam? There are no access controls on KSP

[politas]

I don't understand how any operating system can exist that doesn't allow a file location to be set up as world-writable.

[Baughn]

The location might not even exist. It's entirely legit for an admin to request KSP be installed system-wide, and common even if it's a single-user system, plus if the same setup is requested by two distinct users it will provide the same package without rebuilding. Can't have one user's installation depend on another's.

The workflow I'm envisioning looks like this:

• Download KSP, either through Steam or... well, there are quite a few ways. Ideally, I'd like this step to be automated (requiring users to supply their u/p), but realistically that's unlikely to happen.
• Inject the downloaded KSP base package into the Nix store: nix-store --add "./Kerbal Space Program"

KSP versions will be disambiguated by hash, as is protocol.

That's all the manual work that should be required. At this point, it's up to whatever configuration the user has supplied. Something like this:

environment.systemPackages = [
  chromium firefox kde etc etc
  ksp.kspWithMods {
    version = "1.1.2";
    launcher = "ksp-interstellar";
    includeRecommended = true;
    includeSuggested = false;
    mods = with ckan; [ KSPi RemoteTech DeadlyReentry etc etc etc ];
    exclude = with ckan; [ KAS ];
  }]

Which will create a ksp-interstellar script in /run/current-system/sw/bin and automatically keep it up to date with the latest compatible version of each mod. Naturally, you can add different KSPs with different major versions, launcher names and/or mods.

Behind the scenes, the build script will:

• Generate one derivation for each mod, with dependencies.
  (I'm not sure exactly where this should happen, user-side or through automated PRs. I'll think of something.)
• Recursively download and cache all needed mods in the Nix store. That's one directory per mod, by the way, and I can't control in advance where they end up. ("Somewhere in /nix/store/")
• Create a single directory containing symlinks to all of those mod downloads.
• Unpack KSP. Run CKAN to install all the requested mods. Hope it doesn't need to hit the internet, because depending on the user's sandbox setting it may not be able to. (Incidentally, it'll need to read its manifest from that directory as well.)
• Patchelf the KSP binaries so it can find all needed libraries.
• Write a launcher script ("ksp-interstellar") which does all necessary setup to make KSP work for any given user, perhaps including setup of save directories and such. Of course, most directories (GameData, etc.) will be symlinked to read-only paths under /nix/store.

Does that explain it a little?

This all needs to be submitted to https://github.com/NixOS/nixpkgs/ in the end, and I don't think they'll accept anything overly hacky. :)

[politas]

No, I don't think I'm getting the milieu you're working in.

[Baughn]

After a bit of thought, I think it's almost definitely better if I clone ckan (the binary) rather than attempt to shoehorn my approach into the current one. Thanks anyway!