Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE] Various fixes/improvements for unit test #41032

Merged
merged 2 commits into from Mar 13, 2023

Conversation

smuzaffar
Copy link
Contributor

Updated unit tests so that they can be run from their own separate directory. Mostly

  • convert <bin ...> to <test....>
  • avoid creating test output in cmssw/tmp
  • added flag USE_UNITTEST_DIR="1" to run each tests in separate dir. Once all the tests can run it their own tmp path then we can enable USE_UNITTEST_DIR="1" at project level and do cleanup

@smuzaffar
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41032/34563

  • This PR adds an extra 24KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for master.

It involves the following packages:

  • FWCore/Services (core)
  • FWCore/SharedMemory (core)
  • IOPool/Common (core)
  • IOPool/Input (core)
  • IOPool/Output (core)
  • IOPool/SecondaryInput (core)
  • IOPool/Streamer (core)

@makortel, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @missirol, @wddgit, @fwyzard this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsRun ${SCRAM_TEST_PATH}/NewStreamOutAlt_cfg.py compAlgo=${TEST_COMPRESSION_ALGO} > outAlt 2>&1 || die "cmsRun NewStreamOut_cfg.py compAlgo=${TEST_COMPRESSION_ALGO}" $?
cmsRun ${SCRAM_TEST_PATH}/NewStreamOutExt_cfg.py compAlgo=${TEST_COMPRESSION_ALGO} > outExt 2>&1 || die "cmsRun NewStreamOut_cfg.py compAlgo=${TEST_COMPRESSION_ALGO}" $?
cmsRun ${SCRAM_TEST_PATH}/NewStreamOutExt2_cfg.py compAlgo=${TEST_COMPRESSION_ALGO} > outExt 2>&1 || die "cmsRun NewStreamOutExt2_cfg.py compAlgo=${TEST_COMPRESSION_ALGO}" $?
cmsRun --parameter-set ${SCRAM_TEST_PATH}/NewStreamIn_cfg.py > in 2>&1 || die "cmsRun NewStreamIn_cfg.py" $?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --parameter-set could go away

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11b0e0/31209/summary.html
COMMIT: 1866501
CMSSW: CMSSW_13_1_X_2023-03-10-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41032/31209/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test NewStreamerZSTD had ERRORS
---> test NewStreamerUNCOMPRESSED had ERRORS
---> test NewStreamerZLIB had ERRORS
---> test NewStreamerLZMA had ERRORS
and more ...

Comparison Summary

Summary:

  • You potentially removed 16 lines from the logs
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3530381
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3530359
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 213 log files, 164 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@smuzaffar
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41032/34576

  • This PR adds an extra 28KB to repository

@cmsbuild
Copy link
Contributor

Pull request #41032 was updated. @makortel, @smuzaffar, @Dr15Jones can you please check and sign again.

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11b0e0/31223/summary.html
COMMIT: ebf53ee
CMSSW: CMSSW_13_1_X_2023-03-10-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41032/31223/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 1 lines from the logs
  • Reco comparison results: 12 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3550756
  • DQMHistoTests: Total failures: 10
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3550724
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 213 log files, 164 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

Comment on lines +47 to 51
#cmsRun -j ${LOCAL_TMP_DIR}/TestFastMergeFJRx_second.xml ${LOCAL_TEST_DIR}/FastMergeTest_x_second_cfg.py || die 'Failure using FastMergeTest_x_second_cfg.py' $?
##need to filter items in job report which always change
#egrep -v "<GUID>|<PFN>|^$" $LOCAL_TEST_DIR/proper_fjrx_second_output > $LOCAL_TMP_DIR/proper_fjrx_second_output_filtered
#egrep -v "<GUID>|<PFN>|^$" $LOCAL_TMP_DIR/TestFastMergeFJRx_second.xml > $LOCAL_TMP_DIR/TestFastMergeFJRx_second_filtered.xml
#diff $LOCAL_TMP_DIR/proper_fjrx_second_output_filtered $LOCAL_TMP_DIR/TestFastMergeFJRx_second_filtered.xml || die 'output framework job report is wrong for proper_fjrx_second_output_filtered' $?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this commented out test should be enabled (possibly in a later PR), or removed. I see it was added (as commented out) in 5f1487b.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dr15Jones what do you suggest here?
@makortel , changes for this can go in a separate PR

@makortel
Copy link
Contributor

+core

The tests show another incarnation of #39803

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 99638a6 into cms-sw:master Mar 13, 2023
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants