Allow throwing of edm::Exceptions from python #40315

Dr15Jones · 2022-12-14T18:36:45Z

PR description:

Only a selected set of edm::errors codes are allowed.

PR validation:

Code compiles and a simple script shows it works.

cmsbuild · 2022-12-14T18:43:32Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40315/33395

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/ParameterSet/python/Config.py modified in PR(s): [DO-NOT merge] Revert "Enforce Modifier consistency between Process instances" #37996

cmsbuild · 2022-12-14T18:43:56Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/Framework (core)
FWCore/ParameterSet (core)
FWCore/PythonParameterSet (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@makortel, @missirol, @wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2022-12-14T18:46:06Z

please test

makortel

How about adding a unit test?

makortel · 2022-12-14T19:19:14Z

FWCore/PythonParameterSet/src/PyBind11Module.h

@@ -39,6 +40,8 @@ PYBIND11_MODULE(libFWCorePythonParameterSet, m) {
        std::rethrow_exception(p);
    } catch (const cms::Exception &e) {
      PyErr_SetString(PyExc_RuntimeError, e.what());
+    } catch (const pybind11::error_already_set &e) {
+      std::cout << "pybind " << e.what() << std::endl;


Is this printout still useful?

makortel · 2022-12-14T19:19:44Z

FWCore/PythonParameterSet/src/PyBind11Wrapper.cc

+        auto errorNameEnd = error.find('}');
+        if (errorNameStart != std::string::npos and errorNameEnd != std::string::npos) {
+          auto errorName = error.substr(errorNameStart + 1, errorNameEnd - errorNameStart - 1);
+          std::cout << "EDMException '" << errorName << "'" << std::endl;


Can this printout be removed?

makortel · 2022-12-14T19:19:50Z

FWCore/PythonParameterSet/src/PyBind11Wrapper.cc

+          auto errorName = error.substr(errorNameStart + 1, errorNameEnd - errorNameStart - 1);
+          std::cout << "EDMException '" << errorName << "'" << std::endl;
+          if ("Configuration" == errorName) {
+            std::cout << "throwing edm::Exception" << std::endl;


Can this printout be removed?

makortel · 2022-12-14T19:28:51Z

FWCore/PythonParameterSet/src/PyBind11Wrapper.cc

+        if (errorNameStart != std::string::npos and errorNameEnd != std::string::npos) {
+          auto errorName = error.substr(errorNameStart + 1, errorNameEnd - errorNameStart - 1);
+          std::cout << "EDMException '" << errorName << "'" << std::endl;
+          if ("Configuration" == errorName) {


Would it be feasible to generalize the name-to-error code translation by exploiting the information in

cmssw/FWCore/Utilities/src/EDMException.cc

Lines 9 to 10 in 52c0dac

static const std::map<ErrorCodes, std::string> codeMap = {FILLENTRY(CommandLineProcessing),

FILLENTRY(ConfigFileNotFound),

?

On the other hand, the number of error codes to throw from the python code is likely small (I can think of one to add, but not really more)

I'd prefer to only allow a very select few edm.errors to be used.

makortel · 2022-12-14T19:37:17Z

FWCore/ParameterSet/python/Config.py

+        Configuration = "{Configuration}"
+
+class EDMException(Exception):
+    pass


It is not crystal clear to me how these would be used. Based on the implementation I'd guess

raise EDMException(edm.error.Configuration + "Real exception message")

How about something along the following interface?

raise EDMException(edm.error.Configuration, "Real exception message")

makortel · 2022-12-14T19:39:00Z

FWCore/Framework/bin/cmsRun.cpp

@@ -240,6 +240,8 @@ int main(int argc, char* argv[]) {
      try {
        std::unique_ptr<edm::ParameterSet> parameterSet = edm::readConfig(fileName, argc, argv);
        processDesc.reset(new edm::ProcessDesc(std::move(parameterSet)));
+      } catch (edm::Exception const&) {
+        throw;


I wonder if adding the calling context to the exception would be helpful to interpret the exception message?

The call actually does have a lot of context. E.g.

----- Begin Fatal Exception 14-Dec-2022 14:21:44 CST----------------------- An exception of category 'Configuration' occurred while [0] Processing the python configuration file named test_pyhon_error.py Exception Message: test message At: test_pyhon_error.py(11): <module> ----- End Fatal Exception -------------------------------------------------

for my trivial test case where the configuration file only has the raise call.

Only a selected set of edm::errors codes are allowed.

cmsbuild · 2022-12-14T20:33:53Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40315/33396

This PR adds an extra 56KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/ParameterSet/python/Config.py modified in PR(s): [DO-NOT merge] Revert "Enforce Modifier consistency between Process instances" #37996

cmsbuild · 2022-12-14T20:34:14Z

Pull request #40315 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

Dr15Jones · 2022-12-16T14:46:53Z

please test

cmsbuild · 2022-12-16T21:06:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-baccdf/29659/summary.html
COMMIT: 6e6d795
CMSSW: CMSSW_13_0_X_2022-12-16-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40315/29659/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3557521
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3557496
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

makortel · 2022-12-16T22:10:51Z

+1

cmsbuild · 2022-12-16T22:11:16Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-12-17T08:00:10Z

+1

cmsbuild added this to the CMSSW_13_0_X milestone Dec 14, 2022

cmsbuild added code-checks-pending core-pending orp-pending pending-signatures tests-pending labels Dec 14, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 14, 2022

cmsbuild added tests-started and removed tests-pending labels Dec 14, 2022

makortel reviewed Dec 14, 2022

View reviewed changes

makortel mentioned this pull request Dec 14, 2022

Allow throwing of edm::Exceptions from python cms-sw/framework-team#476

Closed

Allow throwing of edm::Exceptions from python

6e6d795

Only a selected set of edm::errors codes are allowed.

Dr15Jones force-pushed the edmExceptionInPython branch from 1b629a1 to 6e6d795 Compare December 14, 2022 20:27

cmsbuild added code-checks-pending tests-pending and removed tests-started code-checks-approved labels Dec 14, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 14, 2022

cmsbuild added tests-started and removed tests-pending labels Dec 16, 2022

cmsbuild removed the tests-started label Dec 16, 2022

cmsbuild added the tests-approved label Dec 16, 2022

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Dec 16, 2022

cmsbuild added orp-approved and removed orp-pending labels Dec 17, 2022

cmsbuild merged commit d3e9a0c into cms-sw:master Dec 17, 2022

Dr15Jones deleted the edmExceptionInPython branch December 19, 2022 19:11

makortel mentioned this pull request Dec 21, 2022

Improve module type resolver system so that it would be usable for Alpaka modules #40383

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow throwing of edm::Exceptions from python #40315

Allow throwing of edm::Exceptions from python #40315

Dr15Jones commented Dec 14, 2022

cmsbuild commented Dec 14, 2022

cmsbuild commented Dec 14, 2022

Dr15Jones commented Dec 14, 2022

makortel left a comment

makortel Dec 14, 2022

makortel Dec 14, 2022

makortel Dec 14, 2022

makortel Dec 14, 2022

Dr15Jones Dec 14, 2022

makortel Dec 14, 2022

makortel Dec 14, 2022

Dr15Jones Dec 14, 2022

cmsbuild commented Dec 14, 2022

cmsbuild commented Dec 14, 2022

Dr15Jones commented Dec 16, 2022

cmsbuild commented Dec 16, 2022

makortel commented Dec 16, 2022

cmsbuild commented Dec 16, 2022

perrotta commented Dec 17, 2022

	static const std::map<ErrorCodes, std::string> codeMap = {FILLENTRY(CommandLineProcessing),
	FILLENTRY(ConfigFileNotFound),

Allow throwing of edm::Exceptions from python #40315

Allow throwing of edm::Exceptions from python #40315

Conversation

Dr15Jones commented Dec 14, 2022

PR description:

PR validation:

cmsbuild commented Dec 14, 2022

cmsbuild commented Dec 14, 2022

Dr15Jones commented Dec 14, 2022

makortel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Dec 14, 2022

cmsbuild commented Dec 14, 2022

Dr15Jones commented Dec 16, 2022

cmsbuild commented Dec 16, 2022

Comparison Summary

makortel commented Dec 16, 2022

cmsbuild commented Dec 16, 2022

perrotta commented Dec 17, 2022