Deep flavour PR #17247

mverzett · 2017-01-23T10:44:30Z

This PR introduces the DeepCSV and DeepCMVA (disabled) taggers into the RECO sequences.

The tagger outputs are not stored in the default PAT sequence yet as they would make it crash due to the lack of the TagInfo in the AOD event. All basic tests on my side are passing.
This PR depends on external data being available in the IB.
@slava77 suggested memory and CPU footprint checks, I was hoping jenkins would take care of it.
We would like to back-port this PR (disabling the changes to the RECO format) once this is approved. Can I already make a backport PR?

cmsbuild · 2017-01-23T10:44:51Z

A new Pull Request was created by @mverzett (Mauro Verzetti) for CMSSW_9_0_X.

It involves the following packages:

DataFormats/BTauReco
PhysicsTools/PatAlgos
RecoBTag/Combined
RecoBTag/Configuration
RecoBTag/SecondaryVertex

@cmsbuild, @cvuosalo, @slava77, @monttj, @davidlange6 can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @imarches, @ahinzmann, @acaudron, @mmarionncern, @rappoccio, @jdolen, @nhanvtran, @gpetruc, @gkasieczka, @schoef, @ferencek, @mverzett, @mariadalfonso, @pvmulder, @JyothsnaKomaragiri this is something you requested to watch as well.
@davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

slava77 · 2017-01-23T14:18:38Z

@mverzett jenkins does not provide memory checks
Based on the size of the .json file I'm not expecting a large increase in memory

slava77 · 2017-01-23T14:30:54Z

I'll wait for the next IB to start the tests. The .json files should show up in the -2300 IB

slava77 · 2017-01-24T01:07:09Z

@cmsbuild please test

cmsbuild · 2017-01-24T01:07:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17393/console Started: 2017/01/24 02:08

cmsbuild · 2017-01-24T02:57:19Z

-1

Tested at: aadf047

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17247/17393/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

10021.0 step3

runTheMatrix-results/10021.0_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step3_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D7_GenSimHLBeamSpotFull14+DigiFull_2023D7+RecoFullGlobal_2023D7+HARVESTFullGlobal_2023D7/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D7_GenSimHLBeamSpotFull14+DigiFull_2023D7+RecoFullGlobal_2023D7+HARVESTFullGlobal_2023D7.log

50202.0 step3

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step3_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D4_GenSimHLBeamSpotFull14+DigiFull_2023D4+RecoFullGlobal_2023D4+HARVESTFullGlobal_2023D4/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D4_GenSimHLBeamSpotFull14+DigiFull_2023D4+RecoFullGlobal_2023D4+HARVESTFullGlobal_2023D4.log

23234.0 step3

runTheMatrix-results/23234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D8_GenSimHLBeamSpotFull14+DigiFull_2023D8+RecoFullGlobal_2023D8+HARVESTFullGlobal_2023D8/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D8_GenSimHLBeamSpotFull14+DigiFull_2023D8+RecoFullGlobal_2023D8+HARVESTFullGlobal_2023D8.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@Fake,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Tue Jan 24 03:38:50 2017-date Tue Jan 24 03:38:01 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:49:33 2017-date Tue Jan 24 03:38:03 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:49:50 2017-date Tue Jan 24 03:38:09 2017 s - exit: 23040
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@relval25ns,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns --magField 38T_PostLS1 : FAILED - time: date Tue Jan 24 03:39:06 2017-date Tue Jan 24 03:38:14 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:48:30 2017-date Tue Jan 24 03:38:15 2017 s - exit: 23040
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@relval2016,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 --magField 38T_PostLS1 : FAILED - time: date Tue Jan 24 03:47:43 2017-date Tue Jan 24 03:38:19 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:48:33 2017-date Tue Jan 24 03:38:53 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:47:19 2017-date Tue Jan 24 03:38:57 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:52:04 2017-date Tue Jan 24 03:47:24 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:49:54 2017-date Tue Jan 24 03:48:10 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:52:41 2017-date Tue Jan 24 03:48:35 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:50:13 2017-date Tue Jan 24 03:48:40 2017 s - exit: 23040
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Tue Jan 24 03:55:07 2017-date Tue Jan 24 03:49:42 2017 s - exit: 23040

cmsbuild · 2017-01-24T02:57:21Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

slava77 · 2017-01-24T03:47:25Z

looks like I started the tests too early: jenkins was using -1100 IB which doesn't have the training files

mverzett · 2017-01-24T13:31:40Z

@slava77 can you poke CMS bot again? I don't think I have the authority to do it myself

@cmsbuild please test

monttj · 2017-01-24T13:53:12Z

@cmsbuild please test

cmsbuild · 2017-01-24T13:53:31Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17402/console Started: 2017/01/24 14:55

cmsbuild · 2017-02-06T15:10:11Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17638/console Started: 2017/02/06 16:11

cmsbuild · 2017-02-06T16:47:02Z

+1
Tested at: 6f1e3b6
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17247/17638/summary.html

cmsbuild · 2017-02-06T16:47:05Z

Comparison job queued.

cmsbuild · 2017-02-06T17:27:58Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17247/17638/summary.html

slava77 · 2017-02-06T18:02:09Z

+1

for #17247 6f1e3b6

code changes are in line with the PR description and the follow up code review; new b-tagging producers/products are added
jenkins tests pass and comparisons with baseline show no difference (the new products are not monitored)
local tests were done with the earlier version of this PR and show roughly expected performance in the saved quantities and rather small impact on CPU and memory Deep flavour PR #17247 (comment)

mverzett · 2017-02-06T18:36:56Z

@slava77 , can I start a backport to 80X? I got a bit lost for the re-miniAOD schedule, am I still in time? (guess not, but worth trying)

slava77 · 2017-02-06T18:47:35Z

On 2/6/17 10:36 AM, Mauro Verzetti wrote: @slava77 <https://github.com/slava77> , can I start a backport to 80X? I got a bit lost for the re-miniAOD schedule, am I still in time? (guess not, but worth trying)

It would be more practical to wait until this PR is merged in 90X to submit a backport, just in case you receive more comments. re-mniAOD started last Friday. Since this PR changes data and MC, it will wait for the data+MC re-miniAOD. I'm expecting that it will happen in connection with the legacy data rereco. The target for this is some time in March.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17247 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbvWwi-wm_8pOyo7RJcihAIrrFh5Lks5rZ2hJgaJpZM4Lq5J4>.

slava77 · 2017-02-07T15:57:00Z

@monttj
please check this PR, your signature is needed here.
Thank you.

monttj · 2017-02-08T10:27:53Z

+1
@mverzett thank you for new development!

cmsbuild · 2017-02-08T10:28:14Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_9_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

davidlange6 · 2017-02-09T16:47:59Z

+1

cmsbuild added this to the Next CMSSW_9_0_X milestone Jan 23, 2017

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jan 23, 2017

slava77 mentioned this pull request Jan 23, 2017

updates data-RecoBTag-Combined to version v01-00-03 cms-sw/cmsdist#2783

Merged

cmsbuild added tests-started and removed tests-pending labels Jan 24, 2017

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jan 24, 2017

cmsbuild added comparison-pending tests-pending and removed comparison-notrun tests-rejected labels Jan 24, 2017

cmsbuild removed the tests-pending label Jan 24, 2017

cmsbuild added tests-started and removed tests-pending labels Feb 6, 2017

cmsbuild added tests-approved and removed tests-started labels Feb 6, 2017

cmsbuild added comparison-available and removed comparison-pending labels Feb 6, 2017

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Feb 6, 2017

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Feb 8, 2017

cmsbuild added orp-approved and removed orp-pending labels Feb 9, 2017

cmsbuild merged commit c223087 into cms-sw:CMSSW_9_0_X Feb 9, 2017

mverzett pushed a commit to mverzett/cmssw that referenced this pull request Feb 10, 2017

implemented PR cms-sw#17247 comments

a438428

mverzett pushed a commit to mverzett/cmssw that referenced this pull request Feb 21, 2017

implemented PR cms-sw#17247 comments

891cfbb

mverzett mentioned this pull request Feb 21, 2017

Deep flavour PR for cmssw 8_0_X #17584

Merged

mverzett pushed a commit to mverzett/cmssw that referenced this pull request Mar 4, 2017

implemented PR cms-sw#17247 comments

dcda37d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deep flavour PR #17247

Deep flavour PR #17247

mverzett commented Jan 23, 2017

cmsbuild commented Jan 23, 2017

slava77 commented Jan 23, 2017

slava77 commented Jan 23, 2017

slava77 commented Jan 24, 2017

cmsbuild commented Jan 24, 2017 •

edited

cmsbuild commented Jan 24, 2017

cmsbuild commented Jan 24, 2017

slava77 commented Jan 24, 2017 •

edited

mverzett commented Jan 24, 2017

monttj commented Jan 24, 2017

cmsbuild commented Jan 24, 2017 •

edited

cmsbuild commented Feb 6, 2017 •

edited

cmsbuild commented Feb 6, 2017

cmsbuild commented Feb 6, 2017

cmsbuild commented Feb 6, 2017

slava77 commented Feb 6, 2017

mverzett commented Feb 6, 2017

slava77 commented Feb 6, 2017 via email

slava77 commented Feb 7, 2017

monttj commented Feb 8, 2017

cmsbuild commented Feb 8, 2017

davidlange6 commented Feb 9, 2017

Deep flavour PR #17247

Deep flavour PR #17247

Conversation

mverzett commented Jan 23, 2017

cmsbuild commented Jan 23, 2017

slava77 commented Jan 23, 2017

slava77 commented Jan 23, 2017

slava77 commented Jan 24, 2017

cmsbuild commented Jan 24, 2017 • edited

cmsbuild commented Jan 24, 2017

cmsbuild commented Jan 24, 2017

slava77 commented Jan 24, 2017 • edited

mverzett commented Jan 24, 2017

monttj commented Jan 24, 2017

cmsbuild commented Jan 24, 2017 • edited

cmsbuild commented Feb 6, 2017 • edited

cmsbuild commented Feb 6, 2017

cmsbuild commented Feb 6, 2017

cmsbuild commented Feb 6, 2017

slava77 commented Feb 6, 2017

mverzett commented Feb 6, 2017

slava77 commented Feb 6, 2017 via email

slava77 commented Feb 7, 2017

monttj commented Feb 8, 2017

cmsbuild commented Feb 8, 2017

davidlange6 commented Feb 9, 2017

cmsbuild commented Jan 24, 2017 •

edited

slava77 commented Jan 24, 2017 •

edited

cmsbuild commented Jan 24, 2017 •

edited

cmsbuild commented Feb 6, 2017 •

edited