Fortran Intrinsics

[urbanjost]

Fortran Intrinsics descriptions have been discussed as an addition to fortran-lang.org. Herein is a skeleton for the descriptions as a series of MINIBOOK entries.

Preview available at https://fortran-lang.org/pr/347/

[certik]

Nice! This looks awesome. Did you write all that? I am asking to ensure we don't run into copyright issues.

I noticed sometimes the formatting needs improvements, e.g. here:

https://fortran-lang.org/pr/347/learn/intrinsics/ATAN2

the part -PI <= atan(x) <= PI. is inconsistently bold. But we can fix that after merging.

[urbanjost]

I started off writing my own for each intrinsic but at one point to have a starting point for each I filled it in with the pages from the Fortran Wiki, which are under the GFDL, which as I understand it allows for them to be modified and reproduced like this. Many have been changed so much "modified" is an understatement. Ones I know I wrote from scratch I marked at MIT. Many more are from scratch but I didn't have anything marking a lot of them as such so to be conservative I marked them as GFDL as they might have started from the Wiki versions. Since all descriptions of the intrinsics in some way depend on the standard definition of Fortran I have puzzled over exactly how anyone can produce Fortran documentation that describes Fortran without issues with the copyright of the standard. The intent here is that the community as a whole can write new ones for each GFPL-licensed minibook and generate an MIT-only based collection. Each of the vendors has produced a description of the intrinsics, often identical or nearly identical to the descriptions in the Fortran standard, which bolstered my interpretation, but I could be wrong. There is a section in the introduction that is also the second index item that I meant as explanation. Otherwise, it would have to be torn down to just an index and volunteers could be asked to write descriptions (from memory?). Perhaps a vendor has an easier time as they are describing their implementation (?). I started each one as plain text and used some simple scripts to initially make them markdown, and will go through and correct the results (have already been doing this) but learning some of the vagaries of this version of markdown partly by looking at the results (and already have been rewriting some of the results) but there are quite a few unexpected ones such as a / needing escaped and such that can be easily corrected; but already spent too much time on this if it cannot proceed so wanted to present what I have so far and see if we can all agree on going forward, or coming up with an alternate plan. Since you are working on LFortran I was thinking you might have some quick answers on exactly how you can document your results if they are based on the Fortran Standard. I thought the text in the Intro explained the pedigree, but perhaps not well enough.

[urbanjost]

For some reason the links in the text are not working, but worked on the fork I made I was trying this with.

[certik]

@urbanjost thanks for the feedback. I wish FortranWiki was licensed under MIT or some of the permissive Creative Commons licenses (I can't remember which one it is).

I think the best way forward is to simply note for each part where it is taken from. If you wrote it, then it would be MIT, which is the default, so you don't have to mark it. If it comes from FortranWiki, then credit them and mark GDFL (hopefully it doesn't mean we have to license the whole website under GDFL?). And stuff that came from the standard, let's reference that it came from the standard.

That should allow us to move forward.

I would love to reuse all this for LFortran eventually, so that the compiler can point to the definition, say for error messages, as well as bundle the documentation in some way so that things are available locally, so that it works without internet access. Perhaps we can integrate with VSCode somehow, I don't know yet.

Yes, it would be nice to figure out how to handle the GDFL parts as well as parts copied from the standard, so that we are legally on a solid ground. I don't have the answer right now, but I think the above approach should allow us to get moving, and we can figure out the answer as we go.

[urbanjost]

It was written as part of fpm, and later as the fpm-man plugin. The original intent was to make it available from the command line.
So that part might be easy. I can try converting back to text and then to Fortran statements and see how it looks. I just installed pandoc, which it says reads github markdown and can output ASCII text. If that works it would take a minor rework to make it automated to turn this into a fpm-man replacement.

The Fortran standard says you can reproduce parts of it by permission, with a possible fee. That might be the ultimate permission or the ultimate roadblock.

I used to use a nice error message utility called explain. Every code included a message routine that took a product name and an error code. If you hit an error an external library was called and given the two codes. It then produced the message using a message catalog. This allowed for easy multi-lingual capabilities as you could have a message catalog for any supported language, or even add local info to the messages, which were kept in ASCII. It had a terse and verbose mode. When called from programs the default was terse, when called manually the default was verbose. So if you got a message like
f90 #554 - improper something-or-other

you could then enter "explain f90 544" and get a full description of the error. You could also turn off warnings or change their severity level, so if you turned on all the warnings the compiler (or other product) generated, and then realized every line was telling you "f90 #300 FORTRAN Statements must be all uppercase" or "f90 #200 Variable names are limited to six characters" you could turn those off with an environment variable. Except for occasional problems with the environment variable that was used to give a list of directories to search for the catalogs it was great.

For security reasons some programs cleared the environment table on startup, which caused the programs to not find the catalogs, and once one of the developers accidently let his personal message catalog get sent to a customer and during a demo in front of the corporate executives of the customer an error produced a barage of profanity; which made quite a few people laugh but outraged some other rather important ones. Other than that it was great.

[certik]

Haha, great story. Yes, indeed we need translations and explanations for each error in LFortran, along the lines you wrote. Rust does something similar.

[awvwgk]

Thanks, great work. I have one stylistic remark, could we get away from the all caps keywords and headers?

[urbanjost]

Went to #|### to de-emphasize the headers, but the man-style is being used (which requires the all-cap headers) not only because it is a good style many are familiar with but because I am hoping these pages can be automatically rendered into flat text for use in providing data for a simple in-line tool that can be written in Fortran (ie. that the data can be used for fpm-man) and to create actual manpages. Experimenting with pandoc(1) for that and it looks very feasible. Perhaps getting triple use out of the descriptions via an automated process is asking for too much, as it limits you to the ASCII character set for the most part, making representing more complex mathematics and probably some other things too much; but hoping to get this in place not just for the intrinsics but for the stdlib procedures too, where part of the process of providing a routine in stdlib would be to create a description here as well. At least on my screen going to a smaller size helped, I think. Maybe should have stuck with HTML but the Markdown seemed like the write way to go, as it has a lower entry barrier so a lot more people would feel comfortable generating the documention, I think.

[urbanjost]

pandoc worked reasonably well considering I do not know what all the switches do yet.
If you change the name of the directory where the source markdown is on a Linux or Cygwin or WSL
environment and have pandoc installed it makes reasonable man-pages automatically, maybe (text needs a little tweeking, perhaps). I find that enticing, but a program like fman/fpm-man could provide similar functionality and not require following the man-page syntax nor the existence of man(1) for that matter.

#!/bin/bash
#@(#) use pandoc(1) to create man-pages.
header(){
cat <<EOF
." Text automatically generated by panman.sh
.TH "$SHORTNAME" "3" "$(date +'%B %d, %Y')" "" "" " "
." -----------------------------------------------------------------
." * set default formatting
." disable hyphenation
.nh
." disable justification (adjust text to left margin only)
.ad l
." -----------------------------------------------------------------
EOF
}
cd $HOME/github/FORK/fortran-lang.org/learn/intrinsics
mkdir -p /tmp/man/man3
for NAME in *.md
do
   SHORTNAME=$(basename $NAME .md)
   SHORTNAME=${SHORTNAME,,}
   echo "NAME: $NAME to $SHORTNAME"
   (
      header
      sed -n -e '\%^### NAME%,${ p }'  $NAME|
         pandoc -f gfm -t man --columns=72 
   )>/tmp/man/man3/$SHORTNAME.3fortran
   gzip --force /tmp/man/man3/$SHORTNAME.3fortran
done
cd /tmp/man
mandb -c .
env MANWIDTH=256 man --manpath=/tmp/man -k .|col -b
env MANWIDTH=80  man --manpath=/tmp/man --regex '.*'|col -b
exit

[awvwgk]

Ugh, our static side generator doesn't seem to scale well, 1m 11s for the current default build, with the new minibook we are at 3m 43s. We can still stand this build time, but that might not stay sustainable in the long run for more learning resources. Not relevant for this PR but something to keep in mind for the future.

[certik]

I have good experience with Hugo. Very fast builds.

[urbanjost]

#build preview

[urbanjost]

#build preview

The master files are the kramdown files, so anyone else that wants to contribute can just change the files. A particular style that is basically a man-page format is being followed, with a simple set of rules being followed to allow for simple parsing and to maintain compatibility with pandoc(1) for conversions. As an alternate way of contributing, particularly example programs, see
https://urbanjost.github.io/fortran-intrinsic-descriptions/; or we can move this to the Fortran Wiki, which has a much lower bar for contributing and less issues with potential copyright issues. I have not gotten the type of definitive answer from the ISO standard support yet, although there is a page saying that as long as you properly acknowledge the standard as a referenence that reasonable use is permitted. Anyone published a Fortran manual that has some feedback on how to navigate properly using the standard without infringing on it?

PS: I have yet to get a LaTex formula to work with kramdown on this page. Anyone have an example?

[awvwgk]

I have yet to get a LaTex formula to work with kramdown on this page. Anyone have an example?

I remember using MathJax with jekyll / kramdown in the past. There was an issue with TeX and markdown sharing a lot of control characters, something hacky was needed to get those working together like passthroughs or similar. I could look it up, but maybe there is a better solution than TeX? I recall @certik commented on the usage of SymPy for rendering formulas instead of TeX.

[milancurcic]

@awvwgk @henilp105 @certik

In light of the new GSoC project by @henilp105 and future transition to Sphinx+MyST parser, should we go ahead and merge this? I think so--we'd have a good intrinsics reference on the website (despite longer builds), and a good baseline for transition to Sphinx. The intrinsics reference files are in Markdown, which I think is a good format to use as the "source" for translation to html.

[urbanjost]

Have not quite sorted out Sphinx, but the current build converts them to HTML, and pandoc converts to reStructuredText, with something like

pandoc -f markdown_mmd -t rst -i MAXLOC.md  -o ../RST/MAXLOC.md

so (theoretically, have not tried it) they should convert to reStructuredText (maybe with a little massaging with sed(1) or something like it); which I believe is what Sphinx uses; if that is of any value. There was some discussion quite a while ago about migrating to something else, so the pages were written with that in mind. Hopefully, that will allow for them to be moved easily.

[henilp105]

@awvwgk @milancurcic
It would be easy to implement/migrate the intrinsics part to Sphinx as it is already written in markdown, We can easily parse it using Sphinx+MyST parser and they will be directly converted to html and Transitioning to Sphinx+Myst parser would also considerably decrease the Build times.

Thanks and Regards,
Henil Shalin Panchal

[milancurcic]

@certik can this PR be merged?

[certik]

I think so.

[milancurcic]

#build_preview

[github-actions]

This PR has been built with Jekyll and can be previewed at: https://fortran-lang.org/pr/347/

[milancurcic]

From a cursory look at the preview, nothing stands out as broken, so I'll go ahead and merge this.

Thanks a lot @urbanjost for this monumental effort and contribution, and thanks to all reviewers.

[milancurcic]

#delete_preview

[github-actions]

The preview build for this PR has now been deleted.