new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111

kbunkow · 2021-09-01T16:13:54Z

PR description:

This PR contains a new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator: L1Trigger/L1TMuonOverlapPhase1. It should replace the current OMTF emulator L1Trigger/L1TMuonOverlap. The changes in the codes are so significant that we decided to create the new package (L1TMuonOverlapPhase1) rather than updating the old one (L1TMuonOverlap). At some point, L1TMuonOverlap can be removed from the CMSSW (for the moment keeping both packages allows comparing the results easily) .

The OMTF algorithm for the Run3 was significantly improved, these improvements decrease the rate while keeping slightly higher efficiency than the Run2 algorithm.
The L1TMuonOverlapPhase1 can emulate properly the OMTF firmware version with the above-mentioned improvements (this firmware is already deployed in the P5 and was used during the CRUZET).
Additionally, L1TMuonOverlapPhase1 emulates properly the Run2 OMTF firmware – the data-to-emulator agreement on the 2018D data is slightly better for the L1TMuonOverlapPhase1 than for the L1TMuonOverlap.

Besides the new package there are also changes in the CondFormats needed to configure the OMTF emulator:
CondFormats/L1TObjects/interface/L1TMuonOverlapParams.h
There is one more parameter added in the L1TMuonOverlapParams, which is needed for Run3 OMTF algorithm. This change is fully backward compatible, i.e. both L1TMuonOverlapPhase1 and L1TMuonOverlap work correctly with the Run2 records which don’t have this new parameter.
L1TMuonOverlapParams record for the Run3 OMTF algorithm is not yet in the database, if needed the L1TMuonOverlapPhase1 can be configured to emulate the Run3 algorithm with the use of the L1Trigger/L1TMuonOverlapPhase1/python/fakeOmtfParams_cff.py

These two files are also updated:
DQM/L1TMonitor/python/L1TStage2Emulator_cff.py
and
L1Trigger/L1TMuon/python/simDigis_cff.py
in both cases switching to use the new emulator i.e. L1Trigger.L1TMuonOverlapPhase1.simOmtfDigis_cfi

PR validation:

I run the recommended test:
runTheMatrix.py -l limited -i all –ibeos
and the result is:
39 38 37 28 18 4 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 failed

Also, I run the DQM data-to-emulator comparison (cmsRun DQM/Integration/python/clients/l1tstage2emulator_dqm_sourceclient-live_cfg.py with) for a few data files. Both for the recent cosmic run in which the updated OMTF firmware was used, and for the 2018D collision data the data-to-emulator disagreements are at the level of 1%, so similar (or slightly less) than in the case of the old emulator L1TMuonOverlap. The reasons for the disagreements are understood (and unfortunately are very difficult to fix).

if this PR is a backport please specify the original PR and why you need to backport that PR:

This PR is not a backport

cmsbuild · 2021-09-01T16:21:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35111/25002

This PR adds an extra 672KB to repository
Found files with invalid states:
- L1Trigger/L1TMuonOverlapPhase1/interface/Tools/DataROOTDumper.h:
  - Added: 70ff9c2
  - Deleted: e53dc5a
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats_stubsMatching.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats_efficiencyAnalyser.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuon/data/omtf_config/Patterns_0x00012_oldSample_3_30Files_grouped1_classProb17_recalib2.xml:
  - Added: d25d8e0
  - Deleted: 93fb0bf
- L1Trigger/L1TMuonOverlapPhase1/interface/Tools/PatternsPtAssignment.h:
  - Added: 70ff9c2
  - Deleted: e53dc5a
- L1Trigger/L1TMuon/data/omtf_config/.gitignore:
  - Added: d25d8e0
  - Deleted: 93fb0bf
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_run3_realData.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/src/Tools/DataROOTDumper.cc:
  - Added: 70ff9c2
  - Deleted: e53dc5a
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_0x0006_efficiencyAnalyser.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runOMTFPatternMaker.py:
  - Added: 70ff9c2
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/python/simOmtfPhase1Digis_cfi.py:
  - Added: 70ff9c2
  - Modified: 86ea298, ad7c4ad
  - Deleted: bbdb383
- L1Trigger/L1TMuonOverlapPhase1/src/Omtf/OMTFSorterWithThreshold.cc:
  - Added: 70ff9c2
  - Deleted: e53dc5a
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_cosmic2021_realData.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_nn1.py:
  - Added: d25d8e0
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/src/Tools/PatternsPtAssignment.cc:
  - Added: 70ff9c2
  - Deleted: e53dc5a
- L1Trigger/L1TMuonOverlapPhase1/test/runMuonOverlapWriteToXml.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8a422fe
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/mergeGPsXMLFiles.py:
  - Added: 70ff9c2
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats_matcherHist.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runPatternsPtAssignment.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
- L1Trigger/L1TMuonOverlapPhase1/interface/Omtf/OMTFSorterWithThreshold.h:
  - Added: 70ff9c2
  - Deleted: e53dc5a
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_nn.py:
  - Added: 70ff9c2
  - Modified: 86ea298, bbdb383
  - Deleted: 8790005
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/L1TMonitor/python/L1TStage2Emulator_cff.py modified in PR(s): Move CSC trigger and GEM-CSC trigger lookup tables into eventsetup objects (CCLUT-17) #34779
- File L1Trigger/L1TMuon/python/simDigis_cff.py modified in PR(s): Move CSC trigger and GEM-CSC trigger lookup tables into eventsetup objects (CCLUT-17) #34779

cmsbuild · 2021-09-01T16:22:09Z

A new Pull Request was created by @kbunkow for master.

It involves the following packages:

CondFormats/L1TObjects (l1, db, alca)
DQM/L1TMonitor (dqm)
L1Trigger/L1TMuon (l1)
L1Trigger/L1TMuonOverlapPhase1 (****)

The following packages do not have a category, yet:

L1Trigger/L1TMuonOverlapPhase1
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@malbouis, @andrius-k, @yuanchao, @kmaeshima, @ErnestaP, @ahmad3213, @rvenditti, @cmsbuild, @rekovic, @jfernan2, @ggovi, @francescobrivio, @cecilecaillol, @tvami can you please review it and eventually sign? Thanks.
@tocheng, @Martin-Grunewald, @thomreis, @dinyar, @mmusich, @seemasharmafnal this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2021-09-01T16:31:23Z

Hi @kbunkow please change the title to something that's more descriptive of what's in the PR. Also can you please squash the commits into fewer commits? (For example "a small fix", "changes from scram build code-format", etc could go into one commit, and so on)

cmsbuild · 2021-09-02T11:01:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35111/25022

This PR adds an extra 680KB to repository
Found files with invalid states:
- L1Trigger/L1TMuonOverlapPhase1/interface/Tools/DataROOTDumper.h:
  - Added: 70ff9c2
  - Deleted: 6623bc0
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats_stubsMatching.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats_efficiencyAnalyser.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuon/data/omtf_config/Patterns_0x00012_oldSample_3_30Files_grouped1_classProb17_recalib2.xml:
  - Added: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/interface/Tools/PatternsPtAssignment.h:
  - Added: 70ff9c2
  - Deleted: 6623bc0
- L1Trigger/L1TMuon/data/omtf_config/.gitignore:
  - Added: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_run3_realData.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/src/Tools/DataROOTDumper.cc:
  - Added: 70ff9c2
  - Deleted: 6623bc0
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_0x0006_efficiencyAnalyser.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runOMTFPatternMaker.py:
  - Added: 70ff9c2
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/python/simOmtfPhase1Digis_cfi.py:
  - Added: 70ff9c2
  - Deleted: b40f7bf
- L1Trigger/L1TMuonOverlapPhase1/src/Omtf/OMTFSorterWithThreshold.cc:
  - Added: 70ff9c2
  - Deleted: 6623bc0
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_cosmic2021_realData.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_nn1.py:
  - Added: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/src/Tools/PatternsPtAssignment.cc:
  - Added: 70ff9c2
  - Deleted: 6623bc0
- L1Trigger/L1TMuonOverlapPhase1/test/runMuonOverlapWriteToXml.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/mergeGPsXMLFiles.py:
  - Added: 70ff9c2
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_newPats_matcherHist.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runPatternsPtAssignment.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
- L1Trigger/L1TMuonOverlapPhase1/interface/Omtf/OMTFSorterWithThreshold.h:
  - Added: 70ff9c2
  - Deleted: 6623bc0
- L1Trigger/L1TMuonOverlapPhase1/test/expert/omtf/runMuonOverlap_nn.py:
  - Added: 70ff9c2
  - Modified: b40f7bf
  - Deleted: 6876446
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/L1TMonitor/python/L1TStage2Emulator_cff.py modified in PR(s): Move CSC trigger and GEM-CSC trigger lookup tables into eventsetup objects (CCLUT-17) #34779
- File L1Trigger/L1TMuon/python/simDigis_cff.py modified in PR(s): Move CSC trigger and GEM-CSC trigger lookup tables into eventsetup objects (CCLUT-17) #34779

cmsbuild · 2021-09-02T11:01:49Z

Pull request #35111 was updated. @malbouis, @andrius-k, @yuanchao, @kmaeshima, @ErnestaP, @ahmad3213, @rvenditti, @cmsbuild, @rekovic, @jfernan2, @ggovi, @francescobrivio, @cecilecaillol, @tvami can you please check and sign again.

kbunkow · 2021-09-02T11:06:41Z

Hi @tvami
I squshed the original commits into 6 commits, I hope it is ok now.

tvami · 2021-09-02T12:43:29Z

test parameters:

pull_request = Adding L1Trigger/L1TMuonOverlapPhase1 to L1 category cms-bot#1620

tvami · 2021-09-02T12:44:49Z

@cmsbuild , please test

cmsbuild · 2021-09-24T20:18:37Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b76fe9/18905/summary.html
COMMIT: b8d8a96
CMSSW: CMSSW_12_1_X_2021-09-24-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35111/18905/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 40
DQMHistoTests: Total histograms compared: 3211080
DQMHistoTests: Total failures: 224
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3210834
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 39 files compared)
Checked 169 log files, 37 edm output root files, 40 DQM output files
TriggerResults: no differences found

tvami · 2021-09-24T20:23:43Z

+alca

see new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111 (comment)

tvami · 2021-09-24T20:24:02Z

+db

see new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111 (comment)

jfernan2 · 2021-09-25T06:57:18Z

+1

rekovic · 2021-09-27T22:24:58Z

+1

cmsbuild · 2021-09-27T22:25:18Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-10-01T15:09:12Z

@kbunkow could you please provide some ETA for the full replacement of the OMTF emulator? IOW, when do you plan to fully remove from CMSSW the old one, now replaced by this L1TMuonOverlapPhase1? Supposing this PR enters in pre4, could the removal of the old code be already made effective within the following pre5 (last open pre)?

@jfernan2 could you please confirm that the differences visible in the L1ETMU DQM plots are compatible with the replacement of the old emulator with the new one? In particular, why so many more entries in the summary plots (9/2 times the ones that were there with the old emulator)?

jfernan2 · 2021-10-04T06:57:22Z

@jfernan2 could you please confirm that the differences visible in the L1ETMU DQM plots are compatible with the replacement of the old emulator with the new one? In particular, why so many more entries in the summary plots (9/2 times the ones that were there with the old emulator)?

I am afraid I don't know the answer, sorry. I understand the number of entries are improved by the new emulator, @kbunkow any explanation?

kbunkow · 2021-10-04T10:47:58Z

@kbunkow could you please provide some ETA for the full replacement of the OMTF emulator? IOW, when do you plan to fully remove from CMSSW the old one, now replaced by this L1TMuonOverlapPhase1? Supposing this PR enters in pre4, could the removal of the old code be already made effective within the following pre5 (last open pre)?

Regarding the ETA for the full replacement of the OMTF emulator: if old (L1TMuonOverlap) and new (L1TMuonOverlapPhase1) versions of the emulator are both in at least one pre-release (let’s say CMSSW_12_1_0_pre4) then it is OK for us. So we have nothing against removing the L1TMuonOverlap from the next pre-release (e.g. pre5). However, it would be good if @rekovic commented if there are no other constraints.

@jfernan2 could you please confirm that the differences visible in the L1ETMU DQM plots are compatible with the replacement of the old emulator with the new one? In particular, why so many more entries in the summary plots (9/2 times the ones that were there with the old emulator)?

Regarding the differences in the DQM:
the differences that can be seen e.g. here:
wf136_731_base
wf136_731_pr
comes entirely from the fix in the DQM/L1TMonitor/python/L1TStage2Emulator_cff.py , i.e. adding the two lines:
bxMin = -3,
bxMax = 4
Without these two lines, the emulator was emulating the data only in the BX=0, and since the OMTF DAQ records 8 BXes for each event, the “BX range mismatches” appear in e.g. this plot https://tinyurl.com/yzmuyfo4. With the above fix, it is emulating all available BX-es for a given event and there are no “BX range mismatches” (e.g. https://tinyurl.com/yz2ckee2). If the old emulator was used with the above setting, the effect would be exactly the same.
The new emulator gives slightly better data-to-emulator agreement, but this difference cannot be observed on the RelVal plots, as there the statistic is small.

perrotta · 2021-10-05T10:37:15Z

+1

This L1TMuonOverlapPhase1 replaces the current L1TMuonOverlap as L1T OMTF emulator. The code for the new emulator is mostly cloned from the previous one, without any attempt to optimize it for the cohexistence phase of the two. As such, the presence of the two packageses together in the release must be considered redundant: but it is planned to remove the old one as soon as the new one is fully tested in at least one pre-release (e.g. 12_1_0_pre4).
Differences observed in the DQM comparisons are due to an additional fix implemented in this PR, as also explained in new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111 (comment). Real differences due to the new emulator would have been hardly distingushable otherwise because of the limited statistic of those tests
- For future integrations I would suggest to keep separate the inclusion of brand new code (a whole package in this case) from bare fixes of the existing DQM, that could have been more conveniently submitted (and tested) separately.

cmsbuild added this to the CMSSW_12_1_X milestone Sep 1, 2021

cmsbuild added alca-pending code-checks-pending db-pending dqm-pending l1-pending new-package-pending orp-pending pending-signatures tests-pending labels Sep 1, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 1, 2021

kbunkow changed the title ~~From cmssw 12 1 x kb v1~~ new pakage L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) Sep 1, 2021

kbunkow force-pushed the from-CMSSW_12_1_X_KB_v1 branch from 8a422fe to 6876446 Compare September 2, 2021 10:53

cmsbuild added code-checks-pending and removed code-checks-approved labels Sep 2, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 2, 2021

tvami mentioned this pull request Sep 2, 2021

Adding L1Trigger/L1TMuonOverlapPhase1 to L1 category cms-sw/cms-bot#1620

Merged

cmsbuild added the requires-external label Sep 2, 2021

cmsbuild added tests-started and removed tests-pending labels Sep 2, 2021

cmsbuild added the tests-started label Sep 24, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 24, 2021

cmsbuild added alca-approved db-approved and removed alca-pending db-pending labels Sep 24, 2021

cmsbuild added dqm-approved and removed dqm-pending labels Sep 25, 2021

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Sep 27, 2021

cmsbuild added orp-approved and removed orp-pending labels Oct 5, 2021

cmsbuild merged commit ffe0bbf into cms-sw:master Oct 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111

new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111

kbunkow commented Sep 1, 2021

cmsbuild commented Sep 1, 2021

cmsbuild commented Sep 1, 2021

tvami commented Sep 1, 2021

cmsbuild commented Sep 2, 2021

cmsbuild commented Sep 2, 2021

kbunkow commented Sep 2, 2021

tvami commented Sep 2, 2021

tvami commented Sep 2, 2021

cmsbuild commented Sep 24, 2021

tvami commented Sep 24, 2021

tvami commented Sep 24, 2021

jfernan2 commented Sep 25, 2021

rekovic commented Sep 27, 2021

cmsbuild commented Sep 27, 2021

perrotta commented Oct 1, 2021

jfernan2 commented Oct 4, 2021

kbunkow commented Oct 4, 2021

perrotta commented Oct 5, 2021

new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111

new package L1Trigger/L1TMuonOverlapPhase1 (new version of the L1 Trigger Overlap Muon Track Finder (OMTF) emulator) #35111

Conversation

kbunkow commented Sep 1, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Sep 1, 2021

cmsbuild commented Sep 1, 2021

tvami commented Sep 1, 2021

cmsbuild commented Sep 2, 2021

cmsbuild commented Sep 2, 2021

kbunkow commented Sep 2, 2021

tvami commented Sep 2, 2021

tvami commented Sep 2, 2021

cmsbuild commented Sep 24, 2021

Comparison Summary

tvami commented Sep 24, 2021

tvami commented Sep 24, 2021

jfernan2 commented Sep 25, 2021

rekovic commented Sep 27, 2021

cmsbuild commented Sep 27, 2021

perrotta commented Oct 1, 2021

jfernan2 commented Oct 4, 2021

kbunkow commented Oct 4, 2021

perrotta commented Oct 5, 2021