suggestion for a --bootstrap option on the fpm build subcommand

[urbanjost]

Sometimes you want a standalone single source file for an application. A pure Fortran project is particularly amenable to that. It can be needed for a variety of reasons from optimizing inlining to debugging to bootstrapping on another platform. A simple trick when you have a makefile to build a project is to use a script for compiling and have an option for the compiler to become a simple copy of the source file to stdout (usually after expanding INCLUDE files and preprocessor directives). Since fpm(1) is a source package manager
it could easily generate a file list at a minimum, and certainly a Makefile if nothing else. Just curious if that sounds worthwhile. If you do something like that with f-fpm you get a single Fortran file that bootstraps (at least at the moment) except for having to change "include_" to "inc_" to handle a a known issue. It was nice to be able to try f-fpm even as is on a machine without h-fpm and haskell. Just needed a Fortran compiler and a single file. So just changing the program to list the filenames instead of compiling them and a few shell commands gave

# output of files built from a full build
cat \
./src/fpm_environment.f90 \
./src/fpm_strings.f90 \
./src/fpm_filesystem.f90 \
./src/fpm_model.f90 \
./src/fpm_backend.f90 \
./src/fpm/error.f90 \
./src/fpm/git.f90 \
./build/dependencies/toml-f/src/tomlf/constants.f90 \
./build/dependencies/toml-f/src/tomlf/error.f90 \
./build/dependencies/toml-f/src/tomlf/datetime.f90 \
./build/dependencies/toml-f/src/tomlf/utils/verify.f90 \
./build/dependencies/toml-f/src/tomlf/utils/convert.f90 \
./build/dependencies/toml-f/src/tomlf/utils.f90 \
./build/dependencies/toml-f/src/tomlf/type/value.f90 \
./build/dependencies/toml-f/src/tomlf/structure/base.f90 \
./build/dependencies/toml-f/src/tomlf/structure/vector.f90 \
./build/dependencies/toml-f/src/tomlf/structure.f90 \
./build/dependencies/toml-f/src/tomlf/type/array.f90 \
./build/dependencies/toml-f/src/tomlf/type/keyval.f90 \
./build/dependencies/toml-f/src/tomlf/type/table.f90 \
./build/dependencies/toml-f/src/tomlf/type.f90 \
./build/dependencies/toml-f/src/tomlf/build/keyval.f90 \
./build/dependencies/toml-f/src/tomlf/build/array.f90 \
./build/dependencies/toml-f/src/tomlf/build/table.f90 \
./build/dependencies/toml-f/src/tomlf/build.f90 \
./build/dependencies/toml-f/src/tomlf/de/tokenizer.f90 \
./build/dependencies/toml-f/src/tomlf/de/character.f90 \
./build/dependencies/toml-f/src/tomlf/de.f90 \
./build/dependencies/toml-f/src/tomlf/ser.f90 \
./build/dependencies/toml-f/src/tomlf/version.f90 \
./build/dependencies/toml-f/src/tomlf.f90 \
./src/fpm/toml.f90 \
./src/fpm/manifest/dependency.f90 \
./src/fpm/manifest/executable.f90 \
./src/fpm_sources.f90 \
./build/dependencies/M_CLI2/src/M_CLI2.f90 \
./build/dependencies/fortran-intrinsic-manpages/src/M_intrinsics.f90 \
./src/fpm_command_line.f90 \
./src/fpm/manifest/build_config.f90 \
./src/fpm/manifest/library.f90 \
./src/fpm/manifest/test.f90 \
./src/fpm/versioning.f90 \
./src/fpm/manifest/package.f90 \
./src/fpm/manifest.f90 \
./src/fpm.f90 \
./src/fpm/cmd/new.f90 \
./build/dependencies/toml-f/src/tomlf/all.f90 \
./app/main.f90 \
> ffpm.f08
replace include_ inc_ -- ffpm.f08
mkdir /tmp/scratch
gfortran ffpm.f08 -J /tmp/scratch -o ffpm.exe
exit

This was run on a machine that did have h-fpm on it so the dependency files were available and then that single file was moved to a machine with just a fortran compiler and it built and ran; albeit the version I used does not get remote dependencies and so on yet.

I was thinking even it was not to be a general feature it might be useful for maintaining a bootstrappable fpm(1) command in the future.

[ivan-pi]

This is roughly related to #69 where it was suggested to have a backend which can emit a makefile.

In principle this could carry over directly to bootstrap fpm on a computer platform with restricted access.

[everythingfunctional]

My initial inclination for producing a "source code release" (or bootstrap version) was to put all of the necessary source files and a Makefile into a tarbal, but just combining them all into a single source file is supported by the Fortran language, and could thus be simpler. Not a bad idea. (Note that I think order likely still matters to many compilers.)

[certik]

I think this is almost a duplicate of #123. The only difference is that here everything is put into just one file, while #123 is to create a tarball.

Either way, I think the answer is yes, we want that.

[ivan-pi]

My initial inclination for producing a "source code release" (or bootstrap version) was to put all of the necessary source files and a Makefile into a tarbal, but just combining them all into a single source file is supported by the Fortran language, and could thus be simpler. Not a bad idea. (Note that I think order likely still matters to many compilers.)

I guess this can only be done for pure Fortran projects without mixed language dependencies. If C code is somehow used, then a tarball is needed.

[urbanjost]

The files were listed in their dependency order not just as a list so I do not know of a compiler that cannot build it with no other requirements -- CMake, make, ... . I have done that for a long time. I think this is similar enough to the others that I will close it. One of the more interesting things about the Fortran features that support a single-file build is that many compilers are much better at optimizing such a file, catching programming errors, and debugging and profiling the program. I have seen large programs reduce their wallclock times by 20% just by being compiled in a single file. There use to be larger gains possible but compilers have gotten better over the years(so it was often already optimizing even though compiled in seperate compilations). The compiler that produced the least gains when doing this has generally been the Cray compiler which had delayed inlining capabilities. Some compilers used to fail if the file was too large but I have not seen that in some time. With that exception the better the compiler is at optimizing the more gains this produced in the recent past. Have not tried this lately with codes > 300 000, lines but I probably will soon just to see what numbers it produces with a few different compilers. The biggest gains I have seen often involved inlining and I am not sure how well the compilers handle that now that modules are much more common.

[LKedward]

I believe the current --list option does not list file in a suitable build order, but I can easily incorporate this into the existing backend to do so.

---

I know of CFD codes where the production code is still compiled in a single compilation unit to maximise inter-procedural optimization. I think nowadays the potential gains are less. Most compilers also now have link-time optimization, which is essentially the same as single unit compilation but with some parsing/processing done beforehand. This is something I want to benchmark at some point, to see the difference between separate compilation, LTO and single-unit compilation. I seem to think that it has a big effect for operator overloading, but I can't find a reference atm. I still perform a single-unit compilation for intensive programs to check that some significant IPO isn't being left out somewhere due to program structure.