New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Core] Unit tests fixes/improvements #40988
Conversation
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40988/34498
|
A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for master. It involves the following packages:
@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
please test |
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6e2427/31137/summary.html Comparison SummarySummary:
|
+1 Thanks @smuzaffar! |
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2) |
+1 |
@@ -3,10 +3,11 @@ | |||
# Pass in name and status | |||
function die { echo $1: status $2 ; exit $2; } | |||
|
|||
LOCAL_TEST_DIR="${CMSSW_BASE}/src/FWCore/Framework/test" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that scram knows it is executing a test, couldn't we pass a special environment variable to the tests to let them know about where they come from? As seen here, that is a very useful thing for the test. It also avoids having to know exactly which CMSSW variable are available both in a local work area and in the IBs. It would also help if a test is moved from one directory to a different directory for maintenance reasons as one would not have to remember to update the internals of the test as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Dr15Jones , scram does set the following env
export SCRAM_TEST_NAME="<name-of-test>"
export SCRAM_TEST_PATH="$(CMSSW_BASE)/src/<Package/Name>/test"
export SCRAM_TEST_PACKAGE="<Package/Name>"
export SCRAM_TEST_PRE_TESTS="<test-dependencies-on-other-tests>"
so one can make use of these but the only problem is that these are set when you run tests via scram i.e. scram build runtests
or scram b unittests
. So test will fail if you try to run them directly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the previously written tests, they would also fail if not run via scram as the arguments needed for the test were embedded in the BuildFile.xml files. Therefore I do not see it as a problem to use those environment variables in the tests themselves.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not really, previously the TestHelper was setting various env e.g LOCAL_TMP_DIR, LOCAL_TEST_DIR
so when you run the unit tests executable e.g. TestFWCoreFrameworkDeleteEarly
then it works. Of course if you run the shell script directly then yes it would have failed.
But I have no issues if tests can make use of these SCRAM_TEST_*
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TestHelper
generates LOCAL_TEST_DIR
based on the 3rd argument it is passed and that argument is embedded in the BuildFile.xml file. So executing just TestFWCoreFrameworkDeleteEarly
should fail with an 'invalid number of arguments' error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah yes, you are right
It is a trade-off. One way is easier to maintain if the test is moved to a different directory and also it is a little easier to write the shell script (less characters to type). The other way makes it easier to run the test without scram. I don't have a preference. There is one thing that would be nice though. We could add a little to the scram documentation. If we pick Chris's choice, the documentation could give a recipe to run the test without scram (which environment variables need to be set to what and an easy recipe to copy/paste/edit that will do that). Also we should mention "scram b runtests_TestName" in the scram documentation. The biggest reason to run a test standalone is to avoid running all the tests in the directory. For FWCore/Integration, there are many. Although it is faster to run standalone even if scram only runs one test. |
Updated unit tests so that they can be run from their own separate directory. Mostly
<bin ...>
to<test....>
CMSSW_BASE/src/...
instead of./src/
USE_UNITTEST_DIR="1"
to run each tests in separate dir. Once all the tests can run it their own tmp path then we can enableUSE_UNITTEST_DIR="1"
at project level and cleanupFWCore/Framework/test/BuildFile.xml