Install app or library system-wide

[epagone]

I know it might be a bit early but I'd like to log this issue that is particularly important for me.

To use routinely fpm, I would need a functionality that works a bit like sudo make install to install the executable or library system-wide.

PS: thanks for your great work!

[certik]

@epagone thanks! Yes, we have to do it.

@everythingfunctional how can one do that using stack install? I read through stack install --help but didn't see an option to install into /usr/local/bin/.

[epagone]

I read through stack install --help but didn't see an option to install into /usr/local/bin/.

For the records, on Ubuntu 20.04, if I copy manually my library files (libsomething.a) in /usr/local/lib/ or /usr/lib, gfortran finds it with $ gfortran libtest.f90 -lsomething.

However, copying the module files to /usr/local/include does not work and the full path must be specified after the -I flag: $ gfortran libtest.f90 -lsomething -I/usr/local/include

PS: sorry for having repeatedly edited my message, but I have been experimenting a bit and I didn't want to leave confusing or contradictory messages around.

[everythingfunctional]

I think having an install functionality is definitely desired. I believe with stack, stack install is just for executables. I'd lean towards doing the same thing with FPM. I can easily see the desire to install an executable, but I'm not sure I fully understand the benefits of having FPM install libraries.

For installing libraries, I tend to lean on the side of not doing anything that goes around the system package manager, since there's a decent chance of breaking things. Also, FPM is compiling archives, not shared object libraries. And if you're using FPM, you don't need to have the library installed on your system in order for other packages to use it. That's whole point of FPM.

I'm not completely opposed to the idea, but I would need somebody to show me a use case that would constitute good practice. I'd rather FPM not try and also take on the role of system package manager. I think twice about anything that needs sudo to do it.

[certik]

Ah, I missed this point: this is about fpm installing the apps and libraries systemwide.

Why don't we start with what Cargo does via cargo install --root ... which however is for Cargo managed packages, and then we see what else we can do. I think installing apps that are statically linked should be no problem.

Finally, that's a good point that if fpm is to become a usable replacement for CMake (let's say), it must be able to install the executables and libraries to a user defined location.

(Most probably we'll have to be able to build both static and dynamic libraries with fpm down the road.)

[LKedward]

For installing libraries, I tend to lean on the side of not doing anything that goes around the system package manager, since there's a decent chance of breaking things.

The Linux file system has a well-defined segregation between files installed by package managers and those built by users; as @epagone mentioned user-built executables/libraries/headers go in /usr/local/bin /usr/local/lib (both shared and static) and /usr/local/include. This is a common target for makefiles for c libraries and avoids the need to specify include and library paths when compiling and linking.

However this approach can't be used for Fortran libraries because:

1. Unlike c, Fortran has no application binary interface which means that compiled libraries are specific to the compiler and compiler version that compiled it.

2. There is no agreed-upon location to place (and search for) the requisite .mod files, which are also binary-specific to the compiler and compiler version that generated them.

I agree with @everythingfunctional that one of the goals of fpm is to solve this problem for us; specifically, it handles module paths and linker library paths for us in a robust and repeatable manner (which is the main advantage of installing system-wide IMO).

I think install functionality for executables is a natural functionality, I would like:

• (sudo) fpm install for /usr/local/bin and;
• fpm install --local for $HOME/.local/bin

[epagone]

@everythingfunctional

I think having an install functionality is definitely desired. I believe with stack, stack install is just for executables. I'd lean towards doing the same thing with FPM. I can easily see the desire to install an executable, but I'm not sure I fully understand the benefits of having FPM install libraries.

For installing libraries, I tend to lean on the side of not doing anything that goes around the system package manager, since there's a decent chance of breaking things. Also, FPM is compiling archives, not shared object libraries. And if you're using FPM, you don't need to have the library installed on your system in order for other packages to use it. That's whole point of FPM.

Sorry, but I do not understand. Let's take this example: I have a string manipulation library that I build from source pulling a GitHub repo. I need this library in almost all of my many projects. I think it's obvious to expect from the package manager the ability to make the library available system-wide. (FWIW, I am not using FoBiS exactly because it lacks this functionality.) Am I doing something wrong in my development procedures?

@LKedward

However this approach can't be used for Fortran libraries because:

1. Unlike c, Fortran has no application binary interface which means that compiled libraries are specific to the compiler and compiler version that compiled it.

2. There is no agreed-upon location to place (and search for) the requisite `.mod` files, which are also binary-specific to the compiler and compiler version that generated them.

Ouch. Thanks for this explanation, but it's really a bad news! 😞

[everythingfunctional]

I need this library in almost all of my many projects.

If you're using FPM, then it has the responsibility to provide the appropriate version to compile and link with your project. It does not rely on whatever version happens to be installed on your system.

Say for example, the latest version of a library introduced a breaking change that your project relies on. You can continue using the old version for your old project, but use the latest version in your new project, on the same machine, and not have any issues.

[certik]

@epagone wrote:

I think it's obvious to expect from the package manager the ability to make the library available system-wide.

Yes, that's the goal. Fpm must have an option to install a library systemwide or to any location you want.

[milancurcic]

I also think it's important to allow both static and dynamic library builds down the road.

If I understand this correctly, it seems to me that the high-level debate here is whether fpm should work only within the context of a local, package-specific environment (@everythingfunctional position I think), vs. whether fpm could also install to places elsewhere on the system (regardless whether it's a root or user directory).

If you always worked in the package-specific environment, that means that you'd need to rebuild the same dependency for every project that uses it, rather than re-use an existing build. I don't know if Cargo for example can build system-wide, but its canonical behavior is to rebuild everything per-project. For example if you try a demo database or web-server project in Rust, you'll first be building all the 500 dependencies that it needs.

Besides the rationale others already provided, another advantage to this is that some libraries take some time to build (for example NetCDF). Rebuilding them for every new project would hurt user experience.

I think both approaches should be possible. Default should be like Cargo. How about this UI:

• For in-package build, fpm build suffices, no need for fpm install. The library is already installed in build/... in the package directory.
• For install elsewhere, fpm install installs to a sane default user location, perhaps /home/user/.fpm/{bin,lib,include}.
• For install to a specific place, fpm install --prefix=/path/to/install.

[certik]

I agree with @milancurcic's proposal at the end.

Let's start with rebuilding everything for each directory where you call fpm build. But down the road fpm can simply build somewhere in ~/.fpm/ and cache each build and reuse them where possible. Note also that this build directory (whether local to a project directory or in ~/.fpm/) I think might be a bit different from the fpm install command --- the build directory might have more information / metadata that the install command doesn't need to install.

Anyway, I think we can design this so that each of us will get the functionality that we want.

[LKedward]

If we do eventually place built libraries in a central location like ~/.fpm/ then I would suggest that this path include the compiler and compiler version for the same reasoning as #66 due to binary incompatibility between compilers. e.g. ~/.fpm/gfortran-9.1.0/bin etc.

[certik]

@LKedward yes, that's necessary. In fact it should be done by hashes, where a hash depends on:

• compiler vendor and version
• whether Debug or Release build
• hash of the sources for the package
• all dependencies' hashes
• how the given package was configured --- Cargo allows some optional capabilities and we might too in the future, say if you want to build a package with or without netcdf support
• ...