Remove boost lexical_cast dependency in FWCore #34976

Purva-Chaudhari · 2021-08-23T09:50:11Z

PR description:

Remove boost lexical cast dependency in FWCore with corresponding stl alternatives

PR validation:

Passed on scram -b runtests (FWCore)

if this PR is a backport please specify the original PR and why you need to backport that PR:

@davidlange6 @vgvassilev

cmsbuild · 2021-08-23T09:55:12Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34976/24818

This PR adds an extra 40KB to repository

cmsbuild · 2021-08-23T09:55:34Z

A new Pull Request was created by @Purva-Chaudhari for master.

It involves the following packages:

FWCore/Modules (core)
FWCore/ParameterSet (core)
FWCore/Services (core)
FWCore/Utilities (core)

@makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2021-08-23T12:17:57Z

please test

makortel

Thanks!

makortel · 2021-08-23T12:50:16Z

FWCore/ParameterSet/src/types.cc

      // std::cerr << "to:" << to << std::endl;
-    } catch (boost::bad_lexical_cast&) {
+    } catch (const std::invalid_argument&) {


I think it would be useful to catch also std::out_of_range. The both appear to inherit from std::logic_error, so I'd suggest to catch either that or the std::exception.

Replacing with a catch for std::exception in both the files.

makortel · 2021-08-23T12:50:36Z

FWCore/Services/plugins/ProcInfoFetcher.cc

@@ -210,7 +209,7 @@ namespace edm {
              pinfo.num_threads >> pinfo.itrealvalue >> pinfo.starttime >> pinfo.vsize >> pinfo.rss >> pinfo.rlim >>
              pinfo.startcode >> pinfo.endcode >> pinfo.startstack >> pinfo.kstkesp >> pinfo.kstkeip >> pinfo.signal >>
              pinfo.blocked >> pinfo.sigignore >> pinfo.sigcatch >> pinfo.wchan;
-        } catch (boost::bad_lexical_cast& iE) {
+        } catch (const std::invalid_argument& iE) {


std::out_of_range should be caught also here.

cmsbuild · 2021-08-23T15:13:15Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-df0944/17967/summary.html
COMMIT: 95b936c
CMSSW: CMSSW_12_1_X_2021-08-22-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34976/17967/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 1 inherits from legacy modules warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-df0944/17967/llvm-analysis/legacy-mod-sa.txt for details.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000352
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3000324
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

cmsbuild · 2021-08-23T17:27:47Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34976/24831

This PR adds an extra 40KB to repository

cmsbuild · 2021-08-23T17:28:08Z

Pull request #34976 was updated. @makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please check and sign again.

makortel · 2021-08-23T17:41:26Z

@cmsbuild, please test

cmsbuild · 2021-08-23T20:57:21Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-df0944/17977/summary.html
COMMIT: 37da860
CMSSW: CMSSW_12_1_X_2021-08-23-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34976/17977/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 1 inherits from legacy modules warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-df0944/17977/llvm-analysis/legacy-mod-sa.txt for details.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000352
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3000324
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

makortel · 2021-08-23T21:56:23Z

+1

cmsbuild · 2021-08-23T21:56:48Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-08-24T01:08:42Z

+1

Dr15Jones · 2021-08-24T16:28:38Z

It looks like this broke one of the unit tests: test TestIntegrationParameterSet had ERRORS

cmsRun runAutoGeneratedCfi_cfg.py ------------------------------------------------------------
----- Begin Fatal Exception 24-Aug-2021 17:16:37 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_12_1_X_2021-08-24-0900/src/FWCore/Integration/test/runAutoGeneratedCfi_cfg.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'Configuration' occurred.
Exception Message:
EntryError can not convert representation of doublev1: 2.225073858507201e-308 to value of type double 


At:
  /cvmfs/cms-ib.cern.ch/nweek-02695/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-24-0900/python/FWCore/ParameterSet/Types.py(285): insertInto
  /cvmfs/cms-ib.cern.ch/nweek-02695/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-24-0900/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ib.cern.ch/nweek-02695/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-24-0900/python/FWCore/ParameterSet/Mixins.py(512): insertInto
  /cvmfs/cms-ib.cern.ch/nweek-02695/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-24-0900/python/FWCore/ParameterSet/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/nweek-02695/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-24-0900/python/FWCore/ParameterSet/Config.py(1104): _insertManyInto
  /cvmfs/cms-ib.cern.ch/nweek-02695/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-24-0900/python/FWCore/ParameterSet/Config.py(1319): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
Failure cmsRun runAutoGeneratedCfi_cfg.py: status 90

makortel · 2021-08-24T17:43:46Z

The value 2.225073858507201e-308 is tiny bit less than the smallest positive normal double 2.2250738585072014e-308 (from Wikipedia), and apparently std::stod makes that an error, while boost::lexical_cast happily results denormals or rounds to zero for numbers smaller than the smallest denormal.

Appears like a known annoyance with std::stod, see e.g. https://stackoverflow.com/questions/19626246/parse-and-convert-denorm-numbers.

I'd be inclined to revert back to boost::lexical_cast for that one, but am open for better suggestions. (gcc appears to implement std::from_chars() only in 11)

makortel · 2021-08-24T17:45:25Z

@Purva-Chaudhari Would you be able to change FWCore/ParameterSet/src/types.cc back?

Dr15Jones · 2021-08-24T17:50:57Z

@makortel I'm in the process of trying to see if I can find a better solution.

Dr15Jones · 2021-08-24T18:22:45Z

see ##34999

Purva-Chaudhari · 2021-08-25T18:40:55Z

Thanks @Dr15Jones for the improvements

Remove boost lexical_cast dependency in FWCore

95b936c

cmsbuild added this to the CMSSW_12_1_X milestone Aug 23, 2021

cmsbuild added code-checks-pending core-pending orp-pending pending-signatures tests-pending labels Aug 23, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 23, 2021

cmsbuild added tests-started and removed tests-pending labels Aug 23, 2021

makortel reviewed Aug 23, 2021

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Aug 23, 2021

catch std::exception

37da860

cmsbuild added code-checks-pending tests-pending and removed tests-approved code-checks-approved labels Aug 23, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 23, 2021

cmsbuild added tests-started and removed tests-pending labels Aug 23, 2021

cmsbuild added tests-approved and removed tests-started labels Aug 23, 2021

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Aug 23, 2021

cmsbuild added orp-approved and removed orp-pending labels Aug 24, 2021

cmsbuild merged commit 32e5cbf into cms-sw:master Aug 24, 2021

perrotta mentioned this pull request Aug 24, 2021

Removed lexical_cast boost dependency in CondCore #34932

Merged

cmsbuild mentioned this pull request Aug 27, 2021

DataFormats/FEDRawData UBSAN warning fixed #35039

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove boost lexical_cast dependency in FWCore #34976

Remove boost lexical_cast dependency in FWCore #34976

Purva-Chaudhari commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

Dr15Jones commented Aug 23, 2021

makortel left a comment

makortel Aug 23, 2021

Purva-Chaudhari Aug 23, 2021 •

edited

makortel Aug 23, 2021

cmsbuild commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

makortel commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

makortel commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

qliphy commented Aug 24, 2021

Dr15Jones commented Aug 24, 2021

makortel commented Aug 24, 2021 •

edited

makortel commented Aug 24, 2021

Dr15Jones commented Aug 24, 2021

Dr15Jones commented Aug 24, 2021

Purva-Chaudhari commented Aug 25, 2021 •

edited

Remove boost lexical_cast dependency in FWCore #34976

Remove boost lexical_cast dependency in FWCore #34976

Conversation

Purva-Chaudhari commented Aug 23, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

Dr15Jones commented Aug 23, 2021

makortel left a comment

Choose a reason for hiding this comment

makortel Aug 23, 2021

Choose a reason for hiding this comment

Purva-Chaudhari Aug 23, 2021 • edited

Choose a reason for hiding this comment

makortel Aug 23, 2021

Choose a reason for hiding this comment

cmsbuild commented Aug 23, 2021

Comparison Summary

cmsbuild commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

makortel commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

Comparison Summary

makortel commented Aug 23, 2021

cmsbuild commented Aug 23, 2021

qliphy commented Aug 24, 2021

Dr15Jones commented Aug 24, 2021

makortel commented Aug 24, 2021 • edited

makortel commented Aug 24, 2021

Dr15Jones commented Aug 24, 2021

Dr15Jones commented Aug 24, 2021

Purva-Chaudhari commented Aug 25, 2021 • edited

Purva-Chaudhari Aug 23, 2021 •

edited

makortel commented Aug 24, 2021 •

edited

Purva-Chaudhari commented Aug 25, 2021 •

edited