Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

throw a meaninful error if LFN isn't found in FileCatalog #27695

Merged
merged 2 commits into from Aug 6, 2019

Conversation

dan131riley
Copy link

PR description:

When the StreamerFileReader tries to process an LFN that doesn't have a valid PFN translation, it currently tries to open the file "", leading to difficult to diagnose errors at the Tier0, see this thread:

https://hypernews.cern.ch/HyperNews/CMS/get/edmFramework/3876/1/1/1/2/1/1/2/1.html

This PR throws a LogicalFileNameNotFound error if the PFN is empty. Some code changes needed to be made to the error message could include the LFN that resulted in an empty PFN.

PR validation:

Code was tested on the Tier0 repack job configuration from the HN thread, confirming that a working config still worked, and a config with an invalid LFN resulted in a meaningful error message.

In addition the ReadStreamerFile test was repaired (it wasn't running the tests and always returned success), and a new test case added to verify that an invalid LFN results in the correct exception.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27695/11305

  • This PR adds an extra 20KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

A new Pull Request was created by @dan131riley (Dan Riley) for master.

It involves the following packages:

IOPool/Streamer

@cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @wddgit this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@fabiocos
Copy link
Contributor

fabiocos commented Aug 5, 2019

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1838/console Started: 2019/08/05 18:25

<use name="FWCore/ServiceRegistry"/>
<use name="FWCore/Services"/>
<use name="FWCore/ParameterSetReader"/>
<flags TEST_RUNNER_ARGS="all"/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is all a special value known by scram?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that it is an option known by ReadStreamerFile

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it's just an arg to the test program, scram doesn't know anything about the value. Alternatively we could make all the default and get rid of the arg, which might be safer--previously the arg was set to a nonsense string and as a result the tests weren't even being run.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

Comparison job queued.

@Dr15Jones
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 5, 2019

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b02e32/1838/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-b02e32/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 33
  • DQMHistoTests: Total histograms compared: 2715989
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2715659
  • DQMHistoTests: Total skipped: 329
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
  • Checked 137 log files, 14 edm output root files, 33 DQM output files

@fabiocos
Copy link
Contributor

fabiocos commented Aug 6, 2019

+1

@cmsbuild cmsbuild merged commit 90acb56 into cms-sw:master Aug 6, 2019
@dan131riley dan131riley deleted the streamer-noPFN-error branch August 6, 2019 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants