Transform5: outputs: message file(s)

[kcoyle]

This seems obvious but there may be options to discuss.

[tombaker]

@kcoyle By "message file(s)", do you mean that a program processing DCTAP instances should produce an optional log file?

[kcoyle]

Whatever messages are produced by the program. I don't know that it is "optional" - presumably anyone running the program will want to know if there are messages or errors. With a large table, having those just go to the screen may not be sufficient. This is something we need to discuss as part of output, so I will add it to the agenda.

[tombaker]

On yesterday's call, we talked about "messages" in terms of "warnings" that could be emitted by the program (and optionally displayed or supressed by the command-line utility).

Nishad pointed out that in order for a DCTAP utility to fit into pipelines (where the output of the utility is the input to another utility), warnings need to be handled in such a way that they do not interfere with the pipelines. Two options we briefly discussed were: "logging" (for which there are good Python modules) and "warnings" (which are like exceptions but are not intended to stop processing).

In the 'dctap-python' model under development, the only fatal exception is the lack of a "propertyID" header in the first line of a file. As currently implemented, this causes the utility to exit entirely, with the message that the file being examined is not a CSV file following the DCTAP model.

[tombaker]

@kcoyle
The script now emits warnings if called with a command-line switch:

dctap inspect --warnings some.csv

The warnings go to standard error while the output goes to standard output. That way, the warnings will not pollute the pipeline (as per Nishad's preference). I have implemented (but not switched on) alternative logging behavior: not to screen, but to a time-stamped logfile. And another option to implement not just "warnings" but "info" messages - the sort of things one would see in a "verbose" or "debug" mode, listing everything the program is doing.

[kcoyle]

This seems to be complete

[kcoyle]

See dctap-python documentation for resolution.