Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move logintpack and libminifloat under DataFormats/Math, and add unit tests for logintpack::pack16 etc #22837

Merged
merged 2 commits into from Apr 11, 2018

Conversation

makortel
Copy link
Contributor

@makortel makortel commented Apr 4, 2018

This PR adds unit tests for logintpack::pack16() etc functions, in the case that base is something smaller than 32768. I did these mainly to understand and make sure the 16-bit packing/unpacking work as I expected with smaller-than-15-bit base.

In addition, after some discussion below, the logintpack and libminifloat are moved under DataFormats/Math to be better usable outside PackedCandidates without adding a dependence on DataFormats/PatCandidates.

Tested in CMSSW_10_1_0, no changes expected.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

DataFormats/PatCandidates

@perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please review it and eventually sign? Thanks.
@gpetruc, @gouskos, @rovere, @cbernet this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

makortel commented Apr 4, 2018

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27264/console Started: 2018/04/04 14:22

@makortel
Copy link
Contributor Author

makortel commented Apr 4, 2018

I'm planning to use liblogintpack for phase2 premixing (from package under Sim*, @civanch), so let me wonder out loud already now if in light of that the liblogintpack and libminifloat are still best placed in this DataFormats/PatCandidates package? (I'm not claiming they should be moved, just asking beforehand the opinions of people)

In practice liblogintpack is header-only, so #includeng it does not create physical dependence, so maybe my pondering is more academical.

@slava77
Copy link
Contributor

slava77 commented Apr 4, 2018 via email

@arizzi
Copy link
Contributor

arizzi commented Apr 4, 2018

@makortel IIRC it is now possible to use the special "comment syntax" of Double32_t in our dict generation.
I did not test myself but Danilo said that now you can do

< class name="myclass" / >
   < field name="_myDataMember" comment="hello" / >
...

and this can be used with this comment syntax:
https://root.cern.ch/root/html/tutorials/io/double32.C.html

The advantage would be that you do not need to do packing/unpacking in your class but it would be handled by root (on the other hand it introduces explicit dependency on root as a backend for this kind of data compression)

@makortel
Copy link
Contributor Author

makortel commented Apr 4, 2018

@arizzi Interesting, thanks for the link. Following it, I'd interpret "x is converted to y bit unsigned integer" such that the mapping from floating point to integer is linear. But I believe in my case the logarithmic mapping of liblogintpack would work better (values span over several orders of magnitude, relative precision is more meaningful than absolute).

@arizzi
Copy link
Contributor

arizzi commented Apr 4, 2018

I think they also have some float truncation that is more similar to our reduced mantissa and liblogintpack, but I never really tested those because we couldn't use them when statred with miniaod.

@gpetruc
Copy link
Contributor

gpetruc commented Apr 4, 2018 via email

@makortel
Copy link
Contributor Author

makortel commented Apr 4, 2018

@arizzi Thanks for point it out, reading the page carefully there is indeed "x converted to Float_t with truncated precision at 6 bits" and "if ... the double word will be converted to a float and its mantissa truncated to nbits significative bits".

I'll try this option out as well (I'm currently packing the values inside a bitfield, so the outcome depends partly on how well ROOT manages to compress the "remaining" part of the bitfield).

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 4, 2018

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22837/27264/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 29
  • DQMHistoTests: Total histograms compared: 2498516
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2498339
  • DQMHistoTests: Total skipped: 176
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.07999999987 KiB( 23 files compared)
  • Checked 119 log files, 9 edm output root files, 29 DQM output files

@makortel makortel changed the title Add unit tests for logintpack::pack16 etc Move logintpack and libminifloat under DataFormats/Math, and add unit tests for logintpack::pack16 etc Apr 5, 2018
@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

@makortel
Copy link
Contributor Author

makortel commented Apr 5, 2018

@cmsbuild, please test

Moved libminifloat and logintpack under DataFormats/Math.

@slava77 I'm done now.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27308/console Started: 2018/04/05 14:35

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

Pull request #22837 was updated. @perrotta, @monttj, @vazzolini, @kmaeshima, @dmitrijus, @cmsbuild, @jfernan2, @slava77, @gpetruc, @vanbesien, @arizzi can you please check and sign again.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22837/27308/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 29
  • DQMHistoTests: Total histograms compared: 2498516
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2498339
  • DQMHistoTests: Total skipped: 176
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.03000000005 KiB( 22 files compared)
  • Checked 119 log files, 9 edm output root files, 29 DQM output files

@slava77
Copy link
Contributor

slava77 commented Apr 5, 2018

+1

for #22837 31eeb78

  • changes are in line with the PR description; no changes in behavior of standard workflows is expected
  • jenkins tests pass and comparisons with the baseline show no differences

@slava77
Copy link
Contributor

slava77 commented Apr 9, 2018

@dmitrijus
a DQM signature is missing for this PR. Please check.
Thank you.

@dmitrijus
Copy link
Contributor

+1

@arizzi
Copy link
Contributor

arizzi commented Apr 10, 2018

btw, would it make sense to add those function to the dictionaries so that they are loadable in PyROOT/FWlite ?

@makortel
Copy link
Contributor Author

btw, would it make sense to add those function to the dictionaries so that they are loadable in PyROOT/FWlite ?

I don't disagree, but could that be left for a subsequent PR (by someone who knows what exactly should be put to classes_def.xml)?

@fabiocos
Copy link
Contributor

+1

@fabiocos
Copy link
Contributor

merge

@cmsbuild cmsbuild merged commit f248aea into cms-sw:master Apr 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants