Formulate long term vision for Fortran in terms of features

[certik]

I would like the community and the committee to formulate a long term vision what features we would like to work towards in Fortran and how they work together, and consequently, which features we do not want.

As an example, let's say that we would like to have some form of templates and exceptions. Then the two features should be formulated in a way so that it allows us to write all the things we want and that they work well together. Then there are a lot of features that should be implemented in a way to play well together, such as: #44, #45 and then there are features that are not needed, such as #58, because one can achieve the same thing with templates and strong concepts / interfaces.

As another example, let's say we do not want generic programming. In that case, we might consider something like #58.

The goal should be to design the language features well, so that we do not end up with lots of orthogonal features that do not play well together. Having such a vision will help us guide which proposals and features we should prioritize and also help us design all the features to work well together.

[certik]

@FortranFan, in #2 (comment) you mentioned:

Fortran is not a general-purpose language. Rather, it is a domain specific language for array oriented scientific computing.

That may be the current reality with Fortran, but almost everyone who have worked on its language design and continue to do so will greatly dislike being reduced as such and would very much want Fortran to be seen as a general-purpose language.

Can you expand on this a bit?

My view is that I want Fortran to be strong for array oriented scientific computing. If I want to write compilers, C++ is a better language (and so I chose C++ for LFortran). If I want to write web servers, then Python, Go, or other languages are much better also. But if I want to write numerical code, Fortran is a much better language than C++ (easier to write, read and for compilers to optimize), and I would like it to stay that way.

When you say "general-purpose", what domains do you want Fortran to be used in? And as part of this, what features would you like to see added?

[FortranFan]

@certik asked:

When you say "general-purpose", what domains do you want Fortran to be used in? And as part of this, what features would you like to see added?

I think the Wikipedia page on Fortran got it correct with its description, "Fortran is a general-purpose, compiled imperative programming language that is especially suited to numeric computation and scientific computing." And it summarizes Fortran as "multi-paradigm: structured, imperative (procedural, object-oriented), generic." Fortran is now also a parallel programming language - cheers!

Numeric computations and scientific computing are now highly multidisciplinary and they involve a lot of cross-fertilization and collaboration, as you know. And increasingly, there is the acceptance of "computational science" as a veritable third branch of scientific advancement that draws from and also feeds into progress in the traditional ones of theory and experiments. Given all this, Fortran needs to continually evolve with advances with science and computing and it needs to offer features that cover almost all the paradigms, as listed above.

Processing of massive amounts of data in whatever format, machine or human-readable, toward all manner of activities - whether it be climate modeling, DNA research, machine learning, AI/automation, visualization, etc. - is also just as important to scientific and numeric computing as floating-point computation of matrices. It's often counter-productive, I think, to stress one area, say array arithmetic of reals, too much above some other e.g., bit stream processing,

My vision is for Fortran to rise back as the lingua franca in scientific and technical computing. I think this requires Fortran to be an evolving and advancing language that keeps pace in terms of being "feature complete" toward the programming needs of scientists and technologists.

The way I see things is that Fortran currently has GAPS in a few areas of critical interest to computational scientists and technologists including in research, academia, and especially industry. Thus it is NOT "feature complete". The results of the survey by WG5 cover most of these gaps e.g., generics, exception handling, bit data processing, unsigned integers, etc. And I would add a few more in object-oriented aspects as well as good coding practices. Most of these are essentially "solved problems" in terms of computer engineering; robust and functional implementations are available in too many languages and there is little to no open-ended computer science research/investigation element in almost any of the current gaps e.g., Ada added support for generics during the early 1990s.

So my immediate mission will be to try to help "knock off" as many items on the gaps list as quickly as possible, so Fortran can get back to being competitive with other languages and compute platforms.

Modern Fortran is, unfortunately, like the curate's egg, good only in parts and that too only with a modern i.e., "p.c." interpretation of that phrase. When one tries to "use it in anger" toward any big, new effort or attempt a significant refactoring of an existing FORTRAN legacy code-base one ends up uttering stuff like in this cartoon: WG5 is either unaware or in denial about this basic fact. That has gotta change, otherwise Fortran will soon lose its raison d'etre.

[certik]

@FortranFan thank you. I think that's fine to expand Fortran's reach to be especially suited for "numeric computation and scientific computing". (I think your "general purpose" is the same as my "domain specific". We both want a language for scientific computing, as opposed to, say, for web servers.)

I just want to caution against thinking that generics or exceptions are a solved problem, I don't think they are. For example just half a year ago the committee was considering generics in a form that I would be against. It's extremely easy to get it wrong and end up with a system that has more negatives (such as long compilation time, bad error messages, unreadable code, etc.) than positives. At the same time, I am optimistic, and I think there is a way to do it right, but we have to work hard on that and I think having well designed templates in Fortran would be beneficial to its goal of being the language for scientific computing.

[FortranFan]

@certik wrote:

..
I just want to caution against thinking that generics or exceptions are a solved problem, I don't think they are. For example just half a year ago the committee was considering generics in a form that I would be against. It's extremely easy to get it wrong and end up with a system that has more negatives (such as long compilation time, bad error messages, unreadable code, etc.) than positives. ..

I agree.

When I wrote "solved problem", I only meant the computer science and engineering behind the approach and language design that might be needed to develop any particular feature.

What's extremely, extremely difficult, as can be seen on any issue thread on this site, is to first establish the use cases of sufficient interest, to gain any sense of consensus around them especially when 2 or more people particularly "long in the tooth" are involved, to then agree on requirements and to set priorities. Basically, problem formulation is itself very hard. But once it is done for any of the features including generics, a solution can be developed given the current state of compiler "technology". Now the direction Fortran was going previously with generics was driven by an attempt to limit the language to certain use cases and that was pushed by an influential few. A narrow mind can greatly hijack the conceptualization phase and bias the problem formulation.

[gronki]

What I consider a "Fortran spirit" (or: what makes it NOT C++):

• code fast, don't waste time on reinventing the wheel
• let compiler do the optimization
• encapsulate common operations and let vendors provide lightning fast implementations native for the hardware
• be explicit, save time on debugging
• make language rules force clean, self-explanatory code as your students will have to use it in 10 years

This is all in the context that it is mostly used by people who are paid for the results of their computations rather than the code itself. This is why I think tinkering with user-written primary components (strings, list) is very against the goal here and would like to see more common operations closed in intrinsic functions/subroutines.

[FortranFan]

A comment based on DRY/WET discussions in #1: I hope the vision for Fortran will always consider it as a general-purpose, multi-paradigm language even as it retains a strong focus on scientific and technical programming!

Re: multi-paradigm: to achieve it I think one has to practice it regularly even in small aspects and being more open-minded to facilities that might be in areas outside their own experience base or inclinations (e.g., being considerate to needs in generics even if one doesn't care for it because of being interested only in, say, Fortran 95 style code-bases with coarrays perhaps) and to generally have a wide, open tent attitude to allow its practitioners to follow different approaches that suit their needs, all within some reasonable boundaries of course.

I've lost count of the number of times I suggest some improvement in the realm of object-oriented (OO) programming and some other Fortranner will comment they don't particularly care for OO and they never use it. But now, OO is such a critical aspect of so many large and new code-bases coming up in the scientific and technical computing domains even (particularly in industry) and it's blended so much together with other paradigms including parallel programming that to overlook areas of improvement toward OO in Fortran is a major disservice.

Back to the smallish aspect: I think it is entirely reasonable for a group of practitioners to want to control "namespaces" and particularly their "pollution" (which are at the root of several issues like #1, #87, etc.) in one way, say via

use, namespace :: foo_m
..
call foo_m::bar(..  ! note :: is only an illustrative token
..

and another group to do so by

use foo_m, only : bar ..
..
call bar(..
..

and for Fortran to support both the options which legitimately fall in the category of good coding practices.

The usual arguments against this are language bloat and time-resource constraints for standard development. I personally do not think the first is applicable once a matter at hand has the passed the filter of good coding practice. The latter is something that can be managed better and better by striving continuously for improved collaboration and greater workflow efficiency with online platforms like this.

Fortran seriously needs to evolve faster and further to have any hope of being relevant in the future.

[certik]

Fortran's strengths are in writing prototype codes and pure numerical codes. Much better than C++ there (faster and easier to develop, faster compilation, faster runtime execution, ...). However, for large multiphysics codes, a lot of time is spent doing all the maintenance around the numerical core (reading input files, managing the many multiphysics pieces, turning features on and off, ....) and for that C++ does provide more building blocks that one can use, while in Fortran the hands are tied. And so even though C++ is worse for the numerical part, one you get a very large code, the argument for C++ is very compelling. (It becomes subjective, I personally prefer using Fortran for very large codes also, but people who know both and prefer C++ have a very solid reason for that, that cannot be dismissed as "they don't know what they are talking about").

And so that is the main motivation for adding dictionaries, lists into the language, better string support (#24), some generic programming, improvements in OO, etc., to help maintain large codes and give maintainers of those codes tools to choose from so that their hand is not so tied behind their back. For the performance critical parts of the codes, one should still use raw arrays and loops, just like today.

Fortran strength can also be in parallel programming (coarrays being in the language) and in principle on GPU also. But currently C++ has at least a way to run on GPUs (Kokkos, RAJA), while Fortran does not (yes, there is CUDA Fortran which some of our teams use, but it does not work in every compiler...) --- and that is the primary motivation why there is a strong push away from Fortran to C++.

[everythingfunctional]

I wrote up a blog post about what I think should be done to develop Fortran's "vision". I base my opinion's there heavily on some of the ideas the Rust team seems to be using as found in this video. (the stuff most relevant to my point starting around the 13 minute mark).

I'm curious what you guys would think Fortran's "core values" should be? What about secondary values? What shouldn't we care about?

[MarDiehl]

I hope that Fortran finds a better balance between maintaining backward compatibility and modernization. I know that backward compatibility is important to many users, but I have the impression that it stops the further evolution of the language. The 'code once, run/compile forever' idea that the Fortran community sticks to is IMHO obsolete. Unit testing is industry standard today and enables continuous refactoring and modernization.
I find it ridiculous that fixed form code is still used. Instead of forcing compiler developers to spend their time and efforts to ensure that such old code still works, the respective maintainers should to their job and modernize their code.

One striking example is LAPACK: It is the prime example for a very critical, and maintained software that is needed by virtually everyone doing linear algebra. It is written in Fortran, but one does not even get type checks during compilation from modules. Modules are a feature introduced in Fortran 30 years ago! As a result, using this Fortran library from python/numpy is more convenient that using it directly in Fortran. I'm convinced that pressure in the form of 'make modules mandatory' for F2008 would have triggered a re-write of LAPACK in truly modern Fortran and the situation would be much better now.

So my vision of Fortran is a fast, compiled language for scientific and engineering applications that values stability and innovation at the same time. To put numbers on this: A new revision every 2 years, a reference compiler that implements the new features after 2 more years and removal of deprecated features after 10 years.

[epagone]

@MarDiehl I entirely agree. Some time ago, as a stepping stone in this "modernisation" direction, I have proposed a set of defaults for fpm to encourage this transition but reactions were, uhm, mixed.