Provide the ability to do a separate cache-hint for cloning. #6478
Conversation
Cloning is one of the very problematic places for high-latency IO as it reads baskets one-at-a-time. Hence, it makes a lot of sense to use lazy-download for this case. *However*, it may not be desirable to enable lazy-download for all workflows! Nor may it be desirable for the workflow layers to enable lazy-download for all merges (as some sites don't have the disk space). This patch allows us to selectively turn on lazy-download only for merge jobs at a given site by editing site-local-config.xml.
|
A new Pull Request was created by @bbockelm (Brian Bockelman) for CMSSW_7_2_X. Provide the ability to do a separate cache-hint for cloning. It involves the following packages: FWCore/Catalog @cmsbuild, @Dr15Jones, @ktf, @nclopezo can you please review it and eventually sign? Thanks. |
|
@bbockelm can you add a 'link' to the pull request in CMSSW_7_3_X from which this came? That will help in later archeology :). |
|
Greetings, future archeologists! This was previously known as #5983 |
|
+1 |
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_7_2_X IBs unless changes or unless it breaks tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @nclopezo, @ktf, @smuzaffar |
|
@bbockelm This is to deal with the problem seen that the Tier 0, correct? |
|
@Dr15Jones - correct. |
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_7_2_X IBs unless changes (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @nclopezo, @ktf, @smuzaffar |
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_7_2_X IBs unless changes (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @nclopezo, @ktf, @smuzaffar |
|
+1 |
…port Provide the ability to do a separate cache-hint for cloning.
Cloning is one of the very problematic places for high-latency IO
as it reads baskets one-at-a-time. Hence, it makes a lot of sense
to use lazy-download for this case. However, it may not be
desirable to enable lazy-download for all workflows! Nor may it
be desirable for the workflow layers to enable lazy-download for
all merges (as some sites don't have the disk space).
This patch allows us to selectively turn on lazy-download only for
merge jobs at a given site by editing site-local-config.xml.
This commit is a cherry-pick of the b19f156 in 7_3_X.