Enable installation under a specified prefix

[pradyunsg]

There are requests for baking in support for https://gnu.org/prep/standards/html_node/DESTDIR.html in this library.

This is a basic feature in any kind of installer, and without it packagers cannot use this tool.

Originally posted by @FFY00 in https://github.com/pradyunsg/installer/issues/52#issuecomment-835831533

Yeah, use-sysconfig-but-chroot-first is plain necessary for this common use case.

Originally posted by @flying-sheep in https://github.com/pradyunsg/installer/issues/52#issuecomment-838092371

[pradyunsg]

I don't think destdir support needs to be baked into any Destination object, or need special casing in this library. The caller can do the prefixing themselves.

def prefix_with_destdir(path: str) -> str:
    return os.path.join(my_destdir, path.lstrip("/"))


schemes = {key: prefix_with_destdir(value) for key, value in sysconfig.get_paths()}
interpreter = sys.executable
my_destdir_destination = SchemeDictDestination(schemes, interpreter, "posix")

Originally posted by @pradyunsg in https://github.com/pradyunsg/installer/issues/52#issuecomment-841639249

[FFY00]

This is the incorrect fix, I don't think anything uses full paths from sysconfig right now, but they could in the future. The project would not be ready for such cases, in fact, the implementation would be incorrect. This really is a feature for the destination, I just want to place the file in a custom path during the copying/install process, everything else should be the same.

If I implement it in a separate library/CLI it would be via write_to_fs, not by changing the sysconfig paths. But that is not the issue, the issue is that it would be extremely helpful if this was supported by the CLI of this project.
If it isn't, this project no longer becomes viable for Linux distros. We have to write a separate project for that which will just make the bootstrapping process even more difficult. Given how easy it is to implement, and how common of the feature it is in any kind of installer, I don't understand why it can't be supported by this project. I am even willing to contribute all the code and tests, which isn't that much anyway, and commit to maintainership of it.

[pradyunsg]

There's 3 things here:

• the example isn't demonstrating "use sysconfig" but rather that you don't need dedicated custom code in SchemeDictionaryDestination to do the prefix based stuff.

• if you want to inherit and override, you're free to. I don't think that gets you anything, if all you're doing is putting a prefix to a path that you provided yourself.

• The question of "should this be special cased in the API" is separate from "should this be supported in the CLI". If there's a CLI, yea, that should support this. I don't think having it in the CLI requires having it special cased in the API.

[eli-schwartz]

From the other issue:

I'll rephrase my question, if you're installing under a different prefix, why do you NOT want the RECORD to reflect that prefix?

As an example, do object files containing the debug information not contain that prefix in filenames?

Excellent question! In fact, Arch Linux for example passes the GCC option -fdebug-prefix-map to ensure all build references to the source files are remapped to refer to files such as /usr/src/debug/hello-world-1.0/hello.c

/usr/src/debug is the root hierarchy for "debug source" packages, and when debug info is enabled, we actually package these files to help users step through the debugger with full references. The original file would be something like /home/eschwartz/git/pkgbuilds/hello-world/src/hello-world-1.0/hello.c

Not all languages are perfectly handled, e.g. nodejs package.json often contains inscrutable references such as:

"_where": "/home/eschwartz/git/pkgbuilds/src/rapydscript-ng/node_modules/@babel/core",

or

"_where": "/home/eschwartz/git/pkgbuilds/gulp/pkg/gulp/usr/lib/node_modules/gulp/node_modules/gulp-cli",

which is flagged as a warning for manual review because most commonly, references to the build tree, or the install tree without removing $DESTDIR, indicates the package is broken at runtime and will actually try to read/write files from that location (/home/eschwartz/git/pkgbuilds/some-program/src/) once installed on Jane User's computer.

In the nodejs case, there are actually package guidelines to sed out this "_where" reference: https://wiki.archlinux.org/title/Node.js_package_guidelines#Package_contains_reference_to_$srcdir/$pkgdir

Solving this sort of thing when the installer doesn't handle it is a hassle, and may end up with people ignoring a screenful of false positives and allow actual legitimate errors to slip by without being noticed.

Compiled C/C++ is generally pretty clean, golang has -trimpath, rust has --remap-path-prefix. Python has distutils/setuptools, which handled this via --root="$DESTDIR".

At least as a CLI experience, I'd say this is pretty important functionality. I don't know whether or not it should be the API's responsibility to provide a common library abstraction to other tools for this, but one could make the argument that if every installer wants to support this then having that abstraction avoids duplicating the work everywhere...

[pradyunsg]

This is implemented in #94, as part of the CLI (--destdir flag) and API (destdir argument).