Fix the --loglevel flag #114

hagertnl · 2023-05-12T13:09:32Z

Currently, the --loglevel flag does not affect verbosity on the command line/stdout. I would like to change that so that it does affect verbosity on the command line.

…est_harness_drivery.py

hagertnl · 2023-05-12T13:14:05Z

A quick note on the logging module and the log levels:
The logger_threshold_level filters all log messages before they ever reach the file or console handlers.
So if you set logger_threshold_level to WARNING, then the file handler will only receive messages >= WARNING.
Then the fh_threshold_level and ch_threshold_level decide what level of messages to send to the file and console, respectively.

Major changes:

Added a --loglevel flag to test_harness_driver.py
Changed the --loglevel functionality so that instead of --loglevel changing the logger_threshold_level, it only changes the console log level.
- the filehandler (log file) is hard-coded to INFO, and the logger_threshold_level is either set to NOTSET or DEBUG.

hagertnl · 2023-05-12T13:19:30Z

Tested, and I think that this PR will need to also include a re-assigning of log levels to messages. There are some messages that aren't assigned proper priority. For example, here is the loglevel set to CRITICAL:

> runtests.py -i rgt.input.unittests --mode start stop --loglevel CRITICAL
Using machine config: crusher.ini
Using machine config: crusher.ini
Overriding tasks in inputfile since CLI mode was provided
runmodecmd =  ['start', 'stop']
self.__harness_task:  [['start_tests', None, None]]
self.__harness_task:  [['start_tests', None, None], ['stop_tests', None, None]]
reading harness config /lustre/orion/proj-shared/stf016/hagertnl/Scratch/harness-dev/olcf-test-harness/configs/crusher.ini
RGT_PATH_TO_SSPACE is already set. Skipping.
Starting tasks for Application.Test: harness_unit_tests.test_successful_run: [['start_tests', None, None], ['stop_tests', None, None]]
Using machine config: crusher.ini
reading harness config /lustre/orion/proj-shared/stf016/hagertnl/Scratch/harness-dev/olcf-test-harness/configs/crusher.ini
RGT_MACHINE_NAME is already set. Skipping.
RGT_MACHINE_TYPE is already set. Skipping.
RGT_SCHEDULER_TYPE is already set. Skipping.
RGT_JOBLAUNCHER_TYPE is already set. Skipping.
RGT_CPUS_PER_NODE is already set. Skipping.
RGT_GPUS_PER_NODE is already set. Skipping.
RGT_SUBMIT_QUEUE is already set. Skipping.
RGT_SUBMIT_ARGS is already set. Skipping.
RGT_NCCS_TEST_HARNESS_MODULE is already set. Skipping.
RGT_TYPE_OF_REPOSITORY is already set. Skipping.
RGT_GIT_REPS_BRANCH is already set. Skipping.
RGT_GIT_DATA_TRANSFER_PROTOCOL is already set. Skipping.
RGT_GIT_MACHINE_NAME is already set. Skipping.
RGT_GIT_SERVER_APPLICATION_PARENT_DIR is already set. Skipping.
RGT_GIT_SSH_SERVER_URL is already set. Skipping.
RGT_GIT_HTTPS_SERVER_URL is already set. Skipping.
RGT_PATH_TO_SSPACE is already set. Skipping.
RGT_SYSTEM_LOG_TAG is already set. Skipping.
RGT_ACCT_ID is already set. Skipping.
Using launch id: crusher_test/hagertnl@2023-05-12T09:18:56.59
Generated test unique id: 1683897536.7776563
In test_harness_driver, creating logger with logger_name=__main__, fh_filepath=/autofs/nccs-svm1_proj/stf016/hagertnl/harness/unit_tests/harness_unit_tests/test_successful_run/Run_Archive/1683897536.7776563/LogFiles/application_logfile.txt
Creating machine crusher: Type = linux_x86_64 ; Scheduler = slurm ; Job launcher = srun
Creating scheduler
Creating jobLauncher
RGT Test Parameters - Builtin
=============================
job_name = test_successful_run
nodes = 1
total_processes = 1
batch_queue = batch
walltime = 1
executable_path = ./dummy
batch_filename = run.sh
build_cmd = ./build_success.sh
check_cmd = ./check.sh
report_cmd = ./report.sh
resubmit = 1
RGT Test Parameters - User
==========================
Path to Source: /autofs/nccs-svm1_proj/stf016/hagertnl/harness/unit_tests/harness_unit_tests/Source
Path to Build Dir: /lustre/orion/proj-shared/stf016/hagertnl/harness_sspace/crusher/04.27.23-16.26/harness_unit_tests/test_successful_run/1683897536.7776563/build_directory
WARNING: Ignoring invalid built-in parameter key launch_id
Submitting job from SLURM class using batchfilename run.sh
sbatch  -p batch  -A stf016 run.sh
SLURM jobID =  313792
build exit value = 0
submit exit value = 0
Skipped 0, launched 1.

…ogger calls

hagertnl · 2023-08-29T13:56:58Z

After the most recent updates, below is what we see when using CRITICAL log threshold. One note. The build exit value and submit exit value lines typically at the bottom of the output are no longer there, because I added them as INFO logging threshold, instead of print(). To my knowledge, there's not a logger threshold that is "critical info" - info that should be printed every time. Should I use the CRITICAL threshold for this?

> runtests.py -i rgt.input.unittests --mode start stop --loglevel CRITICAL
Using machine config: crusher.ini
Using machine config: crusher.ini
Overriding tasks in inputfile since CLI mode was provided
runmodecmd =  ['start', 'stop']
self.__harness_task:  [['start_tests', None, None]]
self.__harness_task:  [['start_tests', None, None], ['stop_tests', None, None]]
reading harness config /lustre/orion/proj-shared/stf016/hagertnl/Scratch/harness-dev/olcf-test-harness-fork/configs/crusher.ini
RGT_PATH_TO_SSPACE is already set. Skipping.
Starting tasks for Application.Test: harness_unit_tests.test_successful_run: [['start_tests', None, None], ['stop_tests', None, None]]
Using machine config: crusher.ini
reading harness config /lustre/orion/proj-shared/stf016/hagertnl/Scratch/harness-dev/olcf-test-harness-fork/configs/crusher.ini
RGT_MACHINE_NAME is already set. Skipping.
RGT_MACHINE_TYPE is already set. Skipping.
RGT_SCHEDULER_TYPE is already set. Skipping.
RGT_JOBLAUNCHER_TYPE is already set. Skipping.
RGT_CPUS_PER_NODE is already set. Skipping.
RGT_GPUS_PER_NODE is already set. Skipping.
RGT_SUBMIT_QUEUE is already set. Skipping.
RGT_SUBMIT_ARGS is already set. Skipping.
RGT_NCCS_TEST_HARNESS_MODULE is already set. Skipping.
RGT_TYPE_OF_REPOSITORY is already set. Skipping.
RGT_GIT_REPS_BRANCH is already set. Skipping.
RGT_GIT_DATA_TRANSFER_PROTOCOL is already set. Skipping.
RGT_GIT_MACHINE_NAME is already set. Skipping.
RGT_GIT_SERVER_APPLICATION_PARENT_DIR is already set. Skipping.
RGT_GIT_SSH_SERVER_URL is already set. Skipping.
RGT_GIT_HTTPS_SERVER_URL is already set. Skipping.
RGT_PATH_TO_SSPACE is already set. Skipping.
RGT_SYSTEM_LOG_TAG is already set. Skipping.
RGT_ACCT_ID is already set. Skipping.
Using launch id: crusher_test/hagertnl@2023-08-29T09:54:13.79
Generated test unique id: 1693317253.9446626
Creating machine crusher: Type = linux_x86_64 ; Scheduler = slurm ; Job launcher = srun
Creating scheduler
Creating jobLauncher
RGT Test Parameters - Builtin
=============================
job_name = test_successful_run
nodes = 1
total_processes = 1
batch_queue = batch
walltime = 1
executable_path = ./dummy
batch_filename = run.sh
build_cmd = ./build_success.sh
check_cmd = ./check.sh
report_cmd = ./report.sh
resubmit = 1
RGT Test Parameters - User
==========================
WARNING: Ignoring invalid built-in parameter key launch_id
Submitting job from SLURM class using batchfilename run.sh
sbatch  -p batch  -A stf016 run.sh
SLURM jobID =  366639
Launched 1 tests, skipped 0 tests.

hagertnl · 2023-08-29T13:57:34Z

Most of the output currently is from files without loggers, which I identified in #130

ddietz89

I think this looks good, other than the one question.

harness/libraries/repositories/single_app_git_repository.py

…eption being thrown

hagertnl added 2 commits May 12, 2023 09:01

Initial changes to improve loglevel functionality

4643e5c

Used NOTSET as logger_level, or the pre-defined module variable for t…

afcd1e6

…est_harness_drivery.py

Fixed some typos

4243691

This was linked to issues May 12, 2023

LogLevel flags hard-coded in several object classes #102

Closed

No error message on the console when missing job template #37

Closed

Add verbose option for output #17

Closed

ddietz89 mentioned this pull request Jun 22, 2023

LogLevel flags hard-coded in several object classes #102

Closed

hagertnl added 5 commits August 29, 2023 09:07

Removed extraneous print statements from base_machine.py

57dbb98

Upgraded latest link failures to warnings from info

1397720

Upgraded git repository errors to warnings from info

22dc536

Updated error messages and logging levels in apptest.py

083c1be

Removed print statements from test_harness_driver.py, replaced with l…

5af57d2

…ogger calls

hagertnl mentioned this pull request Aug 29, 2023

Several classes in harness code do not have access to a logger #130

Open

Updated logging and printing

863bd94

hagertnl marked this pull request as ready for review August 29, 2023 14:41

hagertnl mentioned this pull request Aug 29, 2023

Updating doc for v3 #108

Merged

ddietz89 requested changes Aug 31, 2023

View reviewed changes

harness/libraries/repositories/single_app_git_repository.py Outdated Show resolved Hide resolved

Upgraded from warning to critical logging for logging prior to an exc…

f6462b7

…eption being thrown

ddietz89 approved these changes Dec 21, 2023

View reviewed changes

ddietz89 merged commit 9acbe49 into devel Dec 21, 2023

hagertnl deleted the nick-fix-log-level branch May 9, 2024 11:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix the --loglevel flag #114

Fix the --loglevel flag #114

hagertnl commented May 12, 2023

hagertnl commented May 12, 2023

hagertnl commented May 12, 2023

hagertnl commented Aug 29, 2023

hagertnl commented Aug 29, 2023

ddietz89 left a comment

Fix the --loglevel flag #114

Fix the --loglevel flag #114

Conversation

hagertnl commented May 12, 2023

hagertnl commented May 12, 2023

hagertnl commented May 12, 2023

hagertnl commented Aug 29, 2023

hagertnl commented Aug 29, 2023

ddietz89 left a comment

Choose a reason for hiding this comment