a subprocess cannot request a product from an unscheduled parent process

[fwyzard]

While writing some test configurations, I ran into an issue with subprocesses and unscheduled execution.

I have a job with a process and a subprocess.
The parent process is configured to run unscheduled, and defines some producers whose products are not needed within the process itself.
If a subprocess tries to request one of those products from the parent process, the job ends with

----- Begin Fatal Exception 15-Dec-2016 10:55:01 CET-----------------------
An exception of category 'LogicError' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'subpath'
   [2] Calling event method for module SampleAnalyzer/'sampleAnalyzer'
   [3] Prefetching for unscheduled module SampleAnalyzer/'sampleAnalyzer'
   [4] Prefetching for unscheduled module SampleMerger/'sampleMerger'
   [5] Calling method for unscheduled module SampleProducer/'sampleProducer'
Exception Message:
ParentProcessProductResolver::putProduct_() not implemented and should never be called.
Contact a Framework developer
----- End Fatal Exception -------------------------------------------------

I think it is fair for the subprocess to fail, but the error message should be less intimidating :-)

To reproduce, prepare an area with

cmsrel CMSSW_9_0_0_pre2
cd CMSSW_9_0_0_pre2/src/
cmsenv
git cms-init
git cms-merge-topic fwyzard:framework-examples
git cms-addpkg DataFormats/Examples FWCore/Examples
scram b -j8

and run the attached python file: bug17046.py.

[cmsbuild]

A new Issue was created by @fwyzard Andrea Bocci.

@davidlange6, @Dr15Jones, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here #13029

[fwyzard]

By the way, I have tried to convince the parent process to produce the product I want to consume in the subprocess with

process.addSubProcess( cms.SubProcess(
    process = subprocess, 
    SelectEvents = cms.untracked.PSet(), 
    outputCommands = cms.untracked.vstring( 'keep *_sampleMerger_*_*' )
) )

but this results in segmentation fault:

#4  <signal handler called>
#5  0x00007f8707ee8727 in edm::ParentProcessProductResolver::prefetchAsync_(edm::WaitingTask*, edm::Principal const&, bool, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const () from /cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre2/lib/slc6_amd64_gcc530/libFWCoreFramework.so
#6  0x00007f8707fcb500 in edm::Worker::prefetchAsync(edm::WaitingTask*, edm::ParentContext const&, edm::Principal const&) () from /cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre2/lib/slc6_amd64_gcc530/libFWCoreFramework.so
#7  0x00007f8707eea664 in void edm::Worker::doWorkAsync<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(edm::WaitingTask*, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::MyPrincipal const&, edm::EventSetup const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*) () from /cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre2/lib/slc6_amd64_gcc530/libFWCoreFramework.so
#8  0x00007f8707ee7603 in edm::UnscheduledProductResolver::prefetchAsync_(edm::WaitingTask*, edm::Principal const&, bool, edm::SharedResourcesAcquirer*, edm::ModuleCallingContext const*) const () from /cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre2/lib/slc6_amd64_gcc530/libFWCoreFramework.so
...

A python file to reproduce this crash is attached:
bug17046_segv.py .

[Dr15Jones]

assign core

[cmsbuild]

New categories assigned: core

@Dr15Jones,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

[Dr15Jones]

@fwyzard have you tried using CSSSW_9_0_DEVEL?

[fwyzard]

Both outcome are reproducible also in CMSSW_9_0_DEVEL_X_2016-12-15-1100 .

[Dr15Jones]

Thanks for checking. I thought for sure I had a unit test that explicitly checked that.

[Dr15Jones]

Fixed in #17064
This time I added a unit test based on your example failing configuration

[Dr15Jones]

+1

[cmsbuild]

This issue is fully signed and ready to be closed.