Adding logfile as output of the exe

[markhegnauer]

Currently, the exe doesn't produce any log-file while or after running wflow. I think it's important to have this logfile, including at least:

• Wflow version
• Echo of important model settings
• Echo of parameter changes made via the TOML change parameter options
• Log information when a model is not running / chrashing
• Runtime information (start/end time)

[verseve]

It is possible to redirect stdout to a file, see also related closed issue #100 (comment)

[markhegnauer]

When I add this to my *.cmd file, I get an empty log.txt.

"..\wflow 0.2.0\bin\wflow_cli.exe" test.toml > log.txt

[verseve]

And what about?

..\wflow 0.2.0\bin\wflow_cli.exe" test.toml > log.txt 2>&1

[markhegnauer]

That indeed produces a log-file, see attached. However, in this way, you get no output at all during the simulation, also not in the CLI. Would be nice if both would be possible at the same time :)

As a next step, I think also more info in the logfile would be nice.

log.txt

[verseve]

This may work at the same time (console and file):

..\wflow 0.2.0\bin\wflow_cli.exe" test.toml 2>&1 | tee log.txt 

indeed logging can be improved (e.g. debug level).

[markhegnauer]

Ok, tested this in a simple *.bat script, that doesn't work (tee is not recognized).

However, running the command from Cmder does work. The log-information is in this case also printed in the file for each step, compared to only at the end when running from simple cmd. The printed log-info in Cmder is much pretier then in simple cmd.

So partly solves the issue, since I do still use batch-files quite a lot (as many others as well I guess).

[verseve]

If tee is not recognized you need to install cygwin or GNU utilities for Windows, then you can also use tee in a *.bat script.

[janverkade]

I support this request, notably for reimporting into Delft-FEWS. The log wouldn't have to be in the PI diagnostics format. A simple text file similar to what the Python logging module would produce, would be fine. Screenshot enclosed. Delft-FEWS would be able to identify the log level from the keywords INFO, DEBUG, WARN, etc.

[visr]

Some implementation ideas for when we get to this:

Right now we show some basic model info at the start, and then a progress bar (https://github.com/JunoLab/ProgressLogging.jl) that is logged to the terminal (https://github.com/c42f/TerminalLoggers.jl). Something like this is probably fine for the terminal such that you can see essentials but not get overloadeded with logging statements. So for a log file we'd probably want to send different information there, and in a different format, with timestamps. I guess we should then use https://github.com/oxinabox/LoggingExtras.jl to split the logging, and send the right info to the right place. That way we can use the same logging infrastructure for both.

[janverkade]

Could I gently ask where, on the list of priorities, this 'logging' is? We see an increasing number of wflow.jl models in gloffis - and a decreasing number of log entries - not neccessarily for the right reasons though :)

[visr]

I think besides bug fixes it's at the top of the list. Next release should include it, would be nice to have that in October.

[janverkade]

Gentle reminder... ;) Background: I debug wflow model runs a lot. As in, A LOT. I'm left in the dark a lot more than I'd like when having to do this without any logs.

[verseve]

@janverkade : it is in progress (finally) in the following branch: log.

[janverkade]

Klinkt bemoedigend. Ik ben niet heel vertrouwd met softwareontwikkeling: begrijp ik dat de functionaliteit er nu is? Is er een gecompileerde wflow-versie die ik kan downloaden en proberen? Dank!

[visr]

The log functionality is pretty much finished in #183. We are holding off on merging this into master for now to investigate an issue that this branch seems to make worse. That only seems to affect some runs that have issues, so you can already try it out if you want. I'll email you a download link.