Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples FAIL_REGULAR_EXPRESSION regex for warning triggers incorrectly with DEPTHAI_LEVEL=warn #380

Closed
diablodale opened this issue Feb 15, 2022 · 0 comments

Comments

@diablodale
Copy link
Contributor

More than 2/3 of tests+examples fail with current develop when logging is set to WARN level.
Why? To see warnings from depthai-core, the envvar (or api) has to be set to the warning level or worse. For example

set DEPTHAI_LEVEL=warn

The example testing cmake/harness recognizes as a test failure using regex "\\[warning\\];\\[error\\];\\[critical\\]"
But setting the env var itself causes a warning.

[2022-02-15 05:01:40.171] [warning] Environment 'DEPTHAI_LEVEL' set to 'warn'

But you can't see warnings unless you set it to warn (or worse).
But setting it to warn causes a warning...which causes the test to be marked failed.
This is a circular issue 😵 caused by commit bd11775

Setup

Repro

  1. cmake config, build for debug with all tests and examples
  2. set your env var DEPTHAI_LEVEL=warn
  3. Run ctest for all tests and examples

Result

Mass failure

[ctest] 27% tests passed, 59 tests failed out of 81
[ctest] 
[ctest] Total Test time (real) = 982.53 sec
[ctest] 
[ctest] The following tests FAILED:
[ctest] 	 23 - bootloader_version (Failed)
[ctest] 	 24 - rgb_camera_control (Failed)
[ctest] 	 25 - rgb_preview (Failed)
[ctest] 	 26 - rgb_video (Failed)
[ctest] 	 27 - edge_detector (Failed)
[ctest] 	 28 - feature_detector (Failed)
[ctest] 	 29 - feature_tracker (Failed)
[ctest] 	 30 - device_queue_event (Failed)
[ctest] 	 31 - opencv_support (Failed)
[ctest] 	 32 - queue_add_callback (Failed)
[ctest] 	 33 - image_manip (Failed)
[ctest] 	 34 - image_manip_rotate (Failed)
[ctest] 	 35 - image_manip_tiling (Failed)
[ctest] 	 36 - rgb_rotate_warp (Failed)
[ctest] 	 37 - imu_gyroscope_accelerometer (Failed)
[ctest] 	 38 - imu_rotation_vector (Failed)
[ctest] 	 39 - mono_depth_mobilenetssd (Failed)
[ctest] 	 40 - rgb_encoding_mono_mobilenet (Failed)
[ctest] 	 41 - rgb_encoding_mono_mobilenet_depth (Failed)
[ctest] 	 42 - rgb_encoding_mobilenet (Failed)
[ctest] 	 43 - camera_mobilenet_sync (Failed)
[ctest] 	 44 - rgb_mobilenet (Failed)
[ctest] 	 45 - rgb_mobilenet_4k (Failed)
[ctest] 	 46 - mono_mobilenet (Failed)
[ctest] 	 47 - video_mobilenet (Failed)
[ctest] 	 48 - webcam_mobilenet (Failed)
[ctest] 	 49 - mono_camera_control (Failed)
[ctest] 	 50 - mono_preview (Failed)
[ctest] 	 51 - mono_full_resolution_saver (Failed)
[ctest] 	 52 - concatenate (Failed)
[ctest] 	 53 - normalize (Failed)
[ctest] 	 54 - object_tracker (Failed)
[ctest] 	 55 - spatial_object_tracker (Failed)
[ctest] 	 56 - object_tracker_video (Failed)
[ctest] 	 57 - script_camera_control (Failed)
[ctest] 	 58 - script_forward_frames (Failed)
[ctest] 	 59 - script_nndata_example (Failed)
[ctest] 	 60 - script_json_communication (Failed)
[ctest] 	 61 - spatial_location_calculator (Failed)
[ctest] 	 62 - spatial_mobilenet_mono (Failed)
[ctest] 	 63 - spatial_mobilenet (Failed)
[ctest] 	 64 - spatial_tiny_yolo_v3 (Failed)
[ctest] 	 65 - spatial_tiny_yolo_v4 (Failed)
[ctest] 	 66 - depth_crop_control (Failed)
[ctest] 	 67 - depth_preview (Failed)
[ctest] 	 68 - depth_post_processing (Failed)
[ctest] 	 69 - rgb_depth_aligned (Failed)
[ctest] 	 70 - stereo_depth_video (Failed)
[ctest] 	 71 - system_information (Failed)
[ctest] 	 72 - disparity_encoding (Failed)
[ctest] 	 73 - rgb_encoding (Failed)
[ctest] 	 74 - rgb_full_resolution_saver (Failed)
[ctest] 	 75 - rgb_mono_encoding (Failed)
[ctest] 	 76 - encoding_max_limit (Failed)
[ctest] 	 77 - mjpeg_encoding (Failed)
[ctest] 	 78 - tiny_yolo_v4_device_side_decoding (Failed)
[ctest] 	 79 - tiny_yolo_v3_device_side_decoding (Failed)
[ctest] 	 80 - apriltag (Failed)
[ctest] 	 81 - apriltag_rgb (Failed)
[ctest] Errors while running CTest
[ctest] CTest finished with return code 8

Expected

All pass.

Notes

I see little value in this code for users of depthai-core. In addition, unexpected display of the values of environment variables worries me.

// Warn if env variable set
if(!value.empty()) {
spdlog::warn("Environment '{}' set to '{}'", var, value);
}

I see some value for the developers of depthai-core itself. But highly recommend this logging be hidden away at the TRACE level of logging. This is something needed to be seen one or twice during debugging, not on every run.

diablodale added a commit to diablodale/depthai-core that referenced this issue Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant