Align semantics of input_validator_flags: foo

[RagnarGrootKoerkamp]

(Original issue by @thorehusfeldt converted to PR.)

After some headscratching, I have realised that BAPCtools disagrees with the problemtools toolchain about a fundamental issue, namely what input_validator_flags should mean. This currently means that BAPCtools cannot verify problems following the other tradition. I’ve tried to summarise my understand of the current situation in Kattis/problem-package-format#25 and Kattis/problem-package-format#26 , but for now I would just want to reestablish shared understanding of semantics and ensure basic functionality.

A simple 3-line fix is this, in bin.run L 115:

if config.args.input_validator_flags_are_args:
    return data.split()
else:
    data = {'name': data}

Then the variable input_validator_flags_are_args can be set, for instance in .bapctools.yaml for a whole contest, and we could verify using both toolchains. (More dramatically, one could add --input_validator_flags_are_args to bt validate.)

[RagnarGrootKoerkamp]

I'm going to look a bit into this but it's probably more productive to chat and discuss. I see that my faulty implementation caused some quite lengthy posts 😅 sorry :/

A bunch of remarks.

My first thought is that (again) this stuff never was used much apart from setting validator_flags in problem.yaml, and hence whatever happens to be the BAPCtools implementation should not be authoritive and is open to be fixed / aligned with the spec and/or problemtools.

There are 3 sources of flags:

• there is problem.yaml:validator[_flags?] (this is for the output validator, but omitted from the name)
• there is data/secret/.../testdata.yaml:input_validator[_flags?]
• there is data/secret/.../testdata.yaml:output_validator[_flags?]

There are multiple ways these could be combined:

• pass all flags
• pass only flags from the innermost testdata.yaml, and problem.yaml:validator_flags otherwise
• pass only flags from the innermost testdata.yaml, and append problem.yaml:validator_flags

And there are 3 validation modes (indeed, 1 more than the spec/problemtools do)

• input_format: syntax of the .in file
• output_format: syntax of the .ans file (only BAPCtools does this, because we can)
• output: output of a submission

I'll have to look into the spec more and read your other issues more closely. From the BAPCtools end I'd say that breaking changes are fine and we can fix the few (possibly only NIO) cases where things are not backwards compatible.

[RagnarGrootKoerkamp]

Probably the most practical way forward is to determine what problemtools does, and update the (kattis) spec accordingly, and then port that to BAPCtools.

[thorehusfeldt]

Thank you. What problemtools does for input_validator_flags is known, and it is correctly documented by the spec; the spec is just not readable without consulting the source code, see Kattis/problem-package-format#25 .