Add in-depth introduction for Fortran with make

[awvwgk]

I just gave make and Fortran another try and realized again why I dropped it from all my project in the first place. Nevertheless, the attempt of writing this up might still be helpful for somebody learning make.

The idea of the guide is the following, take a real world example from the Fortran package index with medium complexity (more than one source file, but not a huge project), which is not using make as build system and describe the process of porting it to make as insightful as possible.

This guide describes:

• creating simple functions
• creating maps in make
• foreach loops over lists
• using eval to generate make expressions from lists
• handling of dependencies
• partly generating the Makefile with external tools
• wildcard functionality to detect source files

Let me know what you think about this.

[arjenmarkus]

What is the level of sophistication you intend to reach? I have seen incredibly complicated Makefiles (mostly generated via tools like automake) that are not intended to be read or understood by mere mortals, but I have never encountered functions, maps, loops over lists or eval, at least not in Makefiles that were intended for mere mortals.
And I do not think it is necessary to make it so sophisticated ;). But I may be mistaken. I would say that describing the last three points in depth would suffice for most if not all Fortran projects, but you can easily prove me wrong. After all, an essential tool like make has been described in preciously few books at any length, or I have missed most of them.

[awvwgk]

The aim is to show a concise, scalable and still readable handwritten Makefile. The resulting Makefile will probably be sophisticated, but not complicated or bloated like an autogenerated Makefile. The part I wrote so far targets only the generation of dependencies in an automatic way, explaining the other features while showing use-cases for them.

[milancurcic]

I like this idea and think it will be useful. It will be useful for me for sure. I even own a book on GNU Make that I struggled learning anything useful from.

If the guide shows to be advanced, we can always give a heads-up note at the beginning of the guide, stating the intended level and who it's for.

[certik]

Here is an example of a manual Makefile that we use in stdlib:

https://github.com/fortran-lang/stdlib/blob/3733ae3624ec6852f9fe624d1f91fbd0423bdf01/src/Makefile.manual

and it has module dependencies, and yet is seems simpler than the one proposed in this PR (no awk, no addsuffix, no patsubst, ...). Wouldn't it make sense to recommend the simpler approach that we did in stdlib?

[awvwgk]

There is certainly more than one style of make possible, the question is which we want to advertise.

I find the substitution reference mechanism of make rather obscure, even if it avoids writing out patsubst or addsuffix functions explicitly. So technically the two Makefiles are setup quite similar, except for the stdlib Makefile is using recursively expanded variables while the guide is using normal variables instead.

[certik]

I see. The approach that I like the most is fpm, and let fpm generate any other build system (if so desired), so even though I think I prefer the stdlib's style of makefile, I am fine with your approach to makefiles also. We can put your style in, and later if people have time, they can submit the other approach also.

[awvwgk]

@certik The scope of this PR is to give an advanced introduction to make. It depends on the status of #156 and I'll adjust this PR to account for any review comments on make style there. I opened it as draft to see if there is any interest to have an advanced make guide in the minibooks at all and to discuss if the format I have chosen seems appropriate.

The approach that I like the most is fpm, and let fpm generate any other build system (if so desired)

I agree that make would be my least favourite choice as build system for Fortran, since there are plenty other build systems like CMake, meson, fpm, ... which are easier to use. But I don't think that letting fpm generate a Makefile removes the need of an introduction to make. An fpm generated Makefile would be on the same level as an autotools or CMake generated one and not supposed to be read or modified by the user (at least in my opinion). Also, I would prefer to discuss fpm related features at the fpm repository instead.

[awvwgk]

Let's give this PR another bump, since it has been stale for a while now.

This guide builds on the basic make introduction from #156, which is now part of the building programs book. It presents an intermediate to advanced overview over some powerful functionality and less known concepts of make. The idea is to show some interesting and unusual approaches how to actually use make to solve problems, going beyond the stuff you will usually find in a random Makefile.

The automation of the dependencies is one of the main issues faced when using make for Fortran projects, of course one can hard code them in the Makefile and avoid the topic, but this is in my opinion not a good answer on how to deal with module dependencies in make.

If this is not suited for the building programs book, I would suggest to start a new book on build systems and development tools as intermediate minibook instead.

[certik]

I just talked with @awvwgk over video regarding this issue and I am fine with merging it as is. While we all work hard on fpm, it will still take some time before it can be used with larger production codes, and even when it can be used, not everybody will be using it for historical or other reasons. And thus having an authoritative source of "best practices" for Make and CMake with Fortran would always be helpful, even if they might not be the first approach we would recommend to beginners down the road.

So I am for merging this, and see what the response is, and if we get feedback, we can iterate on it.

@milancurcic, are you ok to merge as is?

[milancurcic]

I will review it over the weekend and will have minor edit suggestions. But overall I think it's a great addition. Thank you for putting it together!

[arjenmarkus]

I agree - it is a straightforward receipe for creating makefiles in a generic way. And it definitely contains features that I was unaware of. Op vr 18 dec. 2020 om 20:33 schreef Milan Curcic <notifications@github.com>:

…

I will review it over the weekend and will have minor edit suggestions. But overall I think it's a great addition. Thank you for putting it together! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YRZAWLMJFVTHUMA67ETSVOVA3ANCNFSM4TA7XQFQ> .

[awvwgk]

Might be a good occasion to check if the #build_preview is working again.

Edit: Nope, it's still broken.

[milancurcic]

I left many minor editorial suggestions. They mostly address sentences that are too long or some redundant wording.

I'm not sure about the whole awk scripting for dependencies. As you write, it doesn't generally work, only for some simpler cases. Being unfamiliar with awk, reading this part was daunting to me. When I saw the script and read that it won't generally work but only for simpler cases, I was already decided that I'll be writing out my dependencies by hand. So it wasn't very useful for me but may be for others. I guess my only problem with this part is that the tutorial spends a significant fraction teaching how to use a non-Fortran low-level tool to do something that will work only sometimes. At the same time, I think it's clear from the text that "you don't have to do this if you don't want to", so the reader can decide for themselves.

Otherwise I think it's good.

[awvwgk]

The awk script is a weak spot, it only shortcoming is that it cannot deal with submodules, but otherwise it can handle most Fortran module and use statements, which are not using line breaks and/or whitespace hacks in fixed format. The difficulty is that you actually require a full Fortran parser to solve the dependency generation issue generally.

Hardcoding dependencies is a maintenance effort I found unacceptable for using make in Fortran projects, especially for projects I develop together with others. The only robust solution I found so far was over-engineering the Makefile generation or moving to another build system.

[awvwgk]

I added some comments on how awk is processing the Fortran source to extract the information we are looking for.

[ivan-pi]

If you have by chance the O'Reilly book Unix for FORTRAN Programmers written by Mike Loukides it contains a nicely written chapter on make. Since the book was first published in 1990 the content is limited to F77, where make might have been sufficient to resolve all dependencies. If interested I can send you an excerpt from the book.

[awvwgk]

@ivan-pi Thanks for the offer. My best source of knowledge on Unix tools was/is the built-in documentation so far. For example the info page of make is excellent (check info make or https://www.gnu.org/software/make/manual/make.html).

[awvwgk]

Thanks everybody for the comments and suggestions. With two approvals I'll go ahead and merge this PR.