Verbose output of test builder isn't verbose enough

[jscholes]

The create-all-tests script accepts --verbose and --validate flags, intended to help in debugging when a test plan won't build for whatever reason. Unfortunately, output only seems to be logged when:

• A part of the process completes successfully (such as parsing a CSV file); and
• explicit errors are thrown.

The script does not, however, log actions it is about to take, nor which file/row/column is being processed. As such, when an unexpected problem occurs, such as that highlighted in #1244, the test plan author is left at a complete loss about where to start looking for the issue.

Sticking with that example, there is literally no output when running node scripts\create-all-tests --verbose --validate --testplan=hang-sample:

• There are no successful actions for the script to report because it hangs almost immediately.
• There is no error thrown, because the regexp parser is technically doing its job albeit in a way that causes the script to hang.

As a result, it took me over an hour to track down exactly where the failure was occuring. Consider that if the script had output something like the following instead, it would've been a lot more obvious:

>node scripts\create-all-tests --verbose --validate --testplan=hang-sample
Validating row 0 in data/tests.csv for test plan "hang-sample".
Processing tests.csv column: testId
Processing tests.csv column: title
Processing tests.csv column: presentationNumber
Processing tests.csv column: setupScript
Processing tests.csv column: instructions
Processing tests.csv column: assertions

... it would've been immediately obvious that the script froze right after trying to validate the assertions column on that specific row of that specific file.

CC @howard-e

[mcking65]

@ccanash

The seems like a pretty light-weight request with significant productivity benefits. I'm adding it to the ops and maintenance project as a P1.

[ccanash]

@mcking65 this is noted. I will add it to triage and move it to the aria-at-app repo.

[jscholes]

@ccanash This issue relates to scripts in the aria-at repo. The PR to fix it would presumably be submitted to that repo, so I don't think this issue belongs under the app.

[ccanash]

This task had been added to the 2024 project by mistake, I have re added it to the 2025 maintenance project and will added it to our queue.