Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make CPEFast to better reproduce Generic (w/o track angle) #528

Open
wants to merge 19 commits into
base: CMSSW_11_2_X_Patatrack
Choose a base branch
from

Conversation

VinInn
Copy link

@VinInn VinInn commented Aug 3, 2020

This PR improves the error estimation in CPEFast used on GPU providing a simple and fast parameterization in terms of charge, position in x and size in y.

The result is very close to what CPEGeneric provides w/o track-angle (actually a bit more accurate thanks to the estimation of the y-error using the cluster size in y (itself improved with information from the charge unbalance).

It includes also code to recompute y-error using the track-able prior to fit (off by default)

The main effect is a reduction and flattening of the chi2 and a reduction of the tails of the pulls.
Given this improvements I have (for the the being) simplified the cut in chi2 to a single value.

In my opinion further tuning of eff vs fakerate shall be implemented elsewhere (eventually in client code)
It would be also useful to test it against some Simulation with "realistic" APE (such those in the "Legacy Reprocessing")

mariadalfonso and others added 11 commits July 30, 2020 21:21
Move duplicated Eigen code to a common file, and use it for both ECAL and HCAL.
Move HCAL general reconstruction code from the hcal::multifit to the hcal::reconstruction namespace.
CachingDeviceAllocator:
  - add the device allocator status to the public interface
  - monitor the requested amount of bytes in addition to the allocated amount

CUDAMonitoringService:
  - print CUDA memory information after each module, including stats from caching allocator
Update dependent records declarations to use edm::mpl::Vector instead of boost::mpl::vector, following cms-sw#30874.
@VinInn VinInn changed the title New err112 make CPEFast to better reproduce Generic (w/o track angle) Aug 3, 2020
@VinInn VinInn added new feature Pixels Pixels-related developments labels Aug 3, 2020
@AdrianoDee
Copy link

(validating on 11_2_0_pre2)

@AdrianoDee

This comment has been minimized.

@AdrianoDee
Copy link

AdrianoDee commented Aug 6, 2020

Seems there's some compilation error (?). Tried from scratch

cmsrel CMSSW_11_2_0_pre2_Patatrack
cd CMSSW_11_2_0_pre2_Patatrack/src/
git cms-init -x cms-patatrack
git branch CMSSW_11_2_X_Patatrack --track cms-patatrack/CMSSW_11_2_X_Patatrack
git cms-merge-topic -u cms-patatrack:528
scram b -j 16

And got the same issue. Don't know if I'm missing something.

@VinInn
Copy link
Author

VinInn commented Aug 6, 2020

One has to start from master, not CMSSW_11_2_X_Patatrack

v.

@AdrianoDee
Copy link

Ugh sorry, didn't notice. Rerunning.

@fwyzard fwyzard changed the base branch from master to CMSSW_11_2_X_Patatrack August 8, 2020 17:12
@cms-patatrack cms-patatrack deleted a comment from AdrianoDee Aug 8, 2020
@fwyzard
Copy link

fwyzard commented Aug 8, 2020

I've finally synchronised CMSSW_11_2_X and master, and fixed one conflict here (from the MAHI clean up, not fro this code).

For testing, right now it's probably easier to start from CMSSW_11_2_0_pre3 than from CMSSW_11_2_0_pre2_Patatrack.
Of course once CMSSW_11_2_0_pre3_Patatrack is available in a couple of days, that will be the best choice.

@AdrianoDee
Copy link

If you agree I'd directly wait for CMSSW_11_2_0_pre3_Patatrack. What do you think?

@VinInn
Copy link
Author

VinInn commented Aug 9, 2020 via email

@AdrianoDee

This comment has been minimized.

@AdrianoDee
Copy link

Given run2_hlt_2018 conditions have been removed with 129c340ef328a6f2505facc4cc222fc2d36cd1e3 the throughput jobs on Ephemeral have been run with run3_hlt_GRun that should be the same.

@fwyzard fwyzard force-pushed the CMSSW_11_2_X_Patatrack branch 2 times, most recently from bb1d39d to fd98c3d Compare October 20, 2020 20:57
@fwyzard
Copy link

fwyzard commented Nov 18, 2020

Let's re-check on top of the latest branch ...

@dbrehm
Copy link

dbrehm commented Nov 24, 2020

Please see this presentation for run3 GPU pixel only tracking studies on CMSSW_11_2_0_pre8_Patatrack using this PR which show an improvement over the standard way to do error estimation in GPUs. It also shows comparable results to the CPU.
https://indico.cern.ch/event/934814/#57-rechits-cpu-gpu-validation

@fwyzard
Copy link

fwyzard commented Nov 26, 2020

Validation summary

Reference release CMSSW_11_2_0_pre10 at 6c149b2
Development branch cms-patatrack/master at 47423b3
Testing branch cms-patatrack/master at 47423b3 with PRs:

Validation plots

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 11634.5
  • ⚠️ tracking validation plots and summary for workflow 11634.501 are missing
  • tracking validation plots and summary for workflow 11634.502
  • ⚠️ tracking validation plots and summary for workflow 11634.505 are missing
  • tracking validation plots and summary for workflow 11634.506

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 11634.5
  • tracking validation plots and summary for workflow 11634.501
  • tracking validation plots and summary for workflow 11634.502
  • tracking validation plots and summary for workflow 11634.505
  • tracking validation plots and summary for workflow 11634.506

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 11634.5
  • tracking validation plots and summary for workflow 11634.501
  • tracking validation plots and summary for workflow 11634.502
  • tracking validation plots and summary for workflow 11634.505
  • tracking validation plots and summary for workflow 11634.506

Validation plots (CPU vs GPU)

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

  • ⚠️ tracking validation plots and summary for workflows 11634.502 and 11634.501 are missing
  • ⚠️ tracking validation plots and summary for workflows 11634.506 and 11634.505 are missing

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflows 11634.502 and 11634.501
  • tracking validation plots and summary for workflows 11634.506 and 11634.505

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflows 11634.502 and 11634.501
  • tracking validation plots and summary for workflows 11634.506 and 11634.505

Throughput plots

/EphemeralHLTPhysics1/Run2018D-v1/RAW run=323775 lumi=53

scan-136.885502.png
zoom-136.885502.png
scan-136.885512.png
zoom-136.885512.png
scan-136.885522.png
zoom-136.885522.png

logs and nvprof/nvvp profiles

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 11634.5
  • development release, workflow 11634.5
  • development release, workflow 11634.501
  • development release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 11634.505
  • development release, workflow 11634.506
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 11634.511
  • development release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • development release, workflow 11634.521
  • development release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • development release, workflow 136.885522
  • testing release, workflow 11634.5
  • testing release, workflow 11634.501
  • testing release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.505
  • testing release, workflow 11634.506
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.511
  • testing release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • testing release, workflow 11634.521
  • testing release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512
  • testing release, workflow 136.885522

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

  • reference release, workflow 11634.5
  • development release, workflow 11634.5
  • development release, workflow 11634.501
  • development release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 11634.505
  • development release, workflow 11634.506
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 11634.511
  • development release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • development release, workflow 11634.521
  • development release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • development release, workflow 136.885522
  • testing release, workflow 11634.5
  • testing release, workflow 11634.501
  • testing release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.505
  • testing release, workflow 11634.506
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.511
  • testing release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • testing release, workflow 11634.521
  • testing release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512
  • testing release, workflow 136.885522

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 11634.5
  • development release, workflow 11634.5
  • development release, workflow 11634.501
  • development release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 11634.505
  • development release, workflow 11634.506
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 11634.511
  • development release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • development release, workflow 11634.521
  • development release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • development release, workflow 136.885522
  • testing release, workflow 11634.5
  • testing release, workflow 11634.501
  • testing release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.505
  • testing release, workflow 11634.506
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.511
  • testing release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • testing release, workflow 11634.521
  • testing release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512
  • testing release, workflow 136.885522

Logs

The full log is available at https://patatrack.web.cern.ch/patatrack/validation/pulls/401d98ecf289d729b8725fb5d27494419b2ec103/log .

@fwyzard
Copy link

fwyzard commented Nov 27, 2020

There is no impact on the tracking efficiency, but there is a small increase in fakes:
image

The chi² distribution changes significantly, especially at high pT:
image

Maybe the track quality cuts need to be retuned after making these changes ?

@VinInn
Copy link
Author

VinInn commented Nov 27, 2020

Yes, this is expected.
The chi2 changes because the error is different now (and shows how the chi2 is (over)-sensitive to non-gaussian tails)
I loosen the cut on the chi2 to try to "cure" the loss in efficiency at high-pt.
see http://innocent.home.cern.ch/innocent/RelVal/SingleMuFlatIdeal_gpuPR528/plots_pixel_pixel/effandfakePtEtaPhi.pdf
the same gain is not obvious in ttbar2021:
http://innocent.home.cern.ch/innocent/RelVal/pixOnly2021_gpuPR528/plots_pixel_pixel/effandfakePtEtaPhi.pdf
(there one may notice how in the barrel one can reduce fake rate recomputing the error using the track-angle before the fit (code protected by non-active ifdef in this PR).

In my opinion we need first to understand how to recover that (or at least understand the origin), cure it and eventually retune cuts at low pt

view->m_averageGeometry = m_AverageGeometryStore.get();
view->m_cpeParams = cpeParams;
view->m_hitsModuleStart = hitsModuleStart;

// if empy do not bother
if (0 == nHits) {
if
#ifndef __CUDACC__
#ifdef __cpp_if_constexpr
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that we have C++17 both on CPU and GPU, we can just use if constexpr everywhere

Copy link
Author

@VinInn VinInn Nov 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I think this was already commented in the integration PRs.
Appears in other places as well.

@VinInn
Copy link
Author

VinInn commented Nov 27, 2020

I suggest that, to better assess physics performance, this PR to be tested

  1. for HLT Option0
  2. by PF-Scouting
  3. for electron seeding

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Pixels Pixels-related developments
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants