Add support for logging

[arporter]

PSyclone currently has no mechanism for issuing messages other than when an exception is raised. However, there are many cases where we might want to generate a warning to the user but allow execution of PSyclone to continue.

In this ticket we will add an interface to a logging mechanism that will allow us to generate warning/debug messages.

[MatthewHambley]

This is probably related to issue stfc/fparser#65.

[arporter]

Python provides the logging module which probably does what we want. (https://docs.python.org/3.8/library/logging.html) I'm experimenting with using it in my NEMO optimisation script and so far, so good.

[rupertford]

in #581 some simple logging was added (and then commented out). However, when this is tested as part of the full set of tests the log information is not output. When the test is run in isolation or just the psyir directory tests are run then the log information is output. I've created a failing test for this (again which is now commented out), however the test does not fail (so an error is reported) if the full test suite is not run. I'm guessing it is to do with fparser and the way the logging is manipulated there.

[arporter]

We need to decide what functionality we want from the logging system:

1. Do we log to file or to the console?
2. Do we want it to be configurable? (e.g. specify default level and the destination in the config file or via flags.)
3. If logging to file, do we create a new one or append to an existing one?

Personally, I think it should be possible to control the logging from the command line (e.g. to make it more or less verbose by changing the default logging level). Probably it would make sense to also allow the control of the destination of the logging output from the command line too.

What do @rupertford, @hiker, @TeranIvy and @sergisiso think?

[rupertford]

Yes, I think it should be configurable for level and where it is output. Perhaps it should be on the command line for overrides and in a config file for site-specific defaults.

[hiker]

I am all in favour for KISS re where to output: output to stderr (or stdout if preferred), if someone wants it to a file, the build system can easily redirect.

I'd recommend command line options only - I can't see a need to (say) never print warnings - this would just lead to missed errors. At least for user-based transformations there should be an option to disable a warning programmatically (e.g. the HPC expert decides the warning can be ignored and disable the warning in the transformation script), so that the output remains clean. I was thinking about this, but not even this is easy - e.g. you might know that a warning related to variable 'abc' is ok and can be ignored. But then the code might be changed, and we get the same warning for variable 'def' - if we ignore all warnings (or all warnings of this type), no one will noticed that something got broken :( I don't have an easy solution (without investing a lot more time then I am willing to do 😁 ) .

[sergisiso]

There is another option for providing messages that relate to transformations which is to write them as comments on the output code.

This has the advantage that we don't need to generate a new file (so we won't need to open the log and the output to see what happened), and also that we are not good at providing context to where the issue happens because nodes are often not connected to the fparser and even if they do we don't print the line numbers. And this is important when processing long files because the log is also long and is hard to keep track of what part of the code it is talking about.

I recently discussed this with some external users and they thought that something like:

! Psyclone: array2loop failed because we don't know the datatype of A
R(:,:) = A * 4
! Psyclone: loop not parallelised because there is a loop carried dependency between y and z
do i= 1, N

will be much clearer to them in comparison to what Nemo currently does which is writing logs and stderr.