Added support for several different verbosity levels in our integration tests

[bieryAtFnal]

The changes are intended for the fddaq-v5.7.0 release.

Description

In discussions of possible improvements to our regression/integration tests, it was suggested to provide several different levels of verbosity. As described in a presentation at the 31-Mar-2026 SWI&T meeting, we currently have only two options available: one with very limited output and one with the full output. The proposal is to move to a model in which 6 different verbosity levels are supported (where the sixth one enables drunc debug messages).

Suggested instructions for testing the changes are provided below. Please note that the new default verbosity level is 3, so there will be less console output unless the full amount is specifically requested.

In addition to the verbosity-level changes, there are changes included in this PR that are not directly related, but are beneficial. These include:

• a reusable function to perform basic checks on the results of integration tests
  • this common function includes the check on the return code of the run_dunerc fixture, as before, and there is also now a check for warning messages that may have been caught during the setup of the tests. These latter messages were being silently ignored previously.
• support for a user-supplied substring that will trigger the printout of the full drunc output when it is encountered (This is only useful when only part of the drunc output is being displayed, of course)
• test-file-specific changes that will be described in the appropriate repository-based PRs

Here are the command-line options that control the verbosity level and the triggering of the full drunc output:

• for dunedaq_integtest_bundle.sh:
  • --verbosity <level> : requested level of console messages, in range 1-6, where 1 is least, 6 is DRUNC debug
  • --trigger-full-rc-output <phrase that will trigger the full printout of run control messages> : the phrase can be a Python regex, which can be useful in handling colorized text
• for calling pytest directly:
  • --integtest-verbosity <level> - controls the volume of messages that are printed out by the integration test infrastructure
  • --dunerc-fullprint-watch-string <string> - A phrase that, if found in run control messages, will trigger the printout of all RC messages

These changes are correlated with ones in the following repositories:

• Changes to support integration test verbosity levels daqsystemtest#312
• First round of changes to integtests for verbosity levels. dfmodules#485
• Updated the iceberg_real_hsi_test.py to include many of the elements … hsilibs#83
• Updated the listrev_test.py to include the changes that are part of t… listrev#120
• Updated the simple_transform_test to include the changes that are par… snbmodules#30
• Updated the change_rate_test.py to include the changes that are part … trigger#430

Here are suggested steps for testing:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_260413_A9 ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b kbiery/integtest_verbosity_level_ideas
git clone https://github.com/DUNE-DAQ/dfmodules.git -b kbiery/integtest_verbosity_level_ideas
git clone https://github.com/DUNE-DAQ/hsilibs.git -b kbiery/integtest_verbosity_level_ideas
git clone https://github.com/DUNE-DAQ/listrev.git -b kbiery/integtest_verbosity_level_ideas
git clone https://github.com/DUNE-DAQ/snbmodules.git -b kbiery/integtest_verbosity_level_ideas
git clone https://github.com/DUNE-DAQ/trigger.git -b kbiery/integtest_verbosity_level_ideas
cd ..

cd pythoncode
git clone https://github.com/DUNE-DAQ/integrationtest.git -b kbiery/integtest_verbosity_level_ideas
cd ..

dbt-workarea-env
dbt-build -j 12
dbt-workarea-env

dunedaq_integtest_bundle.sh -k min

dunedaq_integtest_bundle.sh --help

Type of change

• New feature or enhancement (non-breaking change which adds functionality)

Testing checklist

• Full set of integration tests pass (dunedaq_integtest_bundle.sh)