11891 create mock classdata for mspileup #11905
Conversation
|
Jenkins results:
|
|
@d-ylee Dennis, shouldn't this have fixed those 3 unit tests mentioned in the GH issue? |
|
@amaltaro Should I be replacing the MockRucio calls in those three tests to use MSPileup instead? |
d-ylee
force-pushed
the
11891-create-mock-classdata-for-mspileup
branch
from
8f7c0b7
to
6999d0d
Compare
|
Jenkins results:
|
|
Jenkins results:
|
|
@d-ylee Dennis, we need to ensure that those 3 unit tests are using a patched (mocked) MSPileup instance/data. For instance, this calls the WorkQueue which in turn executes the DataLocationMapper class: Assuming that we are testing a workqueue element with pileup data, it means that DataLocationMapper will try to resolve the pileup location through MSPileup. Hence, we need to patch MSPileup such that it uses mocked data - or uses the mock class. For that, you will have to run that unit test, check what is the value it is trying to look up at MSPileup, and prepare either the mock class or mocked data (MSPileup) to return the data expected by the unit test. If you prefer to have a chat on this tomorrow, just ping me on mattermost please. |
|
Jenkins results:
|
|
test this please |
|
Jenkins results:
|
|
Jenkins results:
|
|
@amaltaro Looking through Jenkins, it seems that the 3 tests in the Issue are passing and using the mocked MSPileup getPileupDocs: WMCore_t.WorkQueue_t.WorkQueue_t.WorkQueueTest:testProcessingWithPileup WMCore_t.WorkQueue_t.Policy_t.Start_t.Block_t.BlockTestCase:testPileupData WMCore_t.WorkQueue_t.WorkQueue_t.WorkQueueTest:testPileupOnProduction |
|
Cool! I see you also fixed a log record and I would like to actually get it deployed in production tomorrow. |
|
Dennis, you will need to rebase this pull request because I merged the url variable fix, see: #11908 , sorry about that. |
d-ylee
force-pushed
the
11891-create-mock-classdata-for-mspileup
branch
from
f27331a
to
09d4398
Compare
Patch locations where getPileupDocs is used Add proper RSE sites in MockMSPileupAPI
Use MSPileupObj schema() instead of a predefined dict Removed ThisBlock from Block_t
d-ylee
force-pushed
the
11891-create-mock-classdata-for-mspileup
branch
from
09d4398
to
7a18a9f
Compare
|
Jenkins results:
|
| return | ||
| if self.mockMSPileup: | ||
| self.msPileupPatchers = [] | ||
| patchMSPileupAt = ['WMCore.MicroService.MSTransferor.MSTransferor.getPileupDocs', # TODO: Need to use Services.MSPileupUtils |
There was a problem hiding this comment.
Dennis, can you please create a new GH issue such that we don't forget to adopt the same MSPileup library across the project - and also to deprecate the one under Tools.Common module? Thanks
|
|
||
| queryDict = queryDict or {} | ||
|
|
||
| pdict = { |
There was a problem hiding this comment.
as data grows we might consider making a json dump for this.
Fixes #11891
Status
In development
Description
Mocks the MSPileupUtils.py getPileupDocs call for testing via an MSPileup Emulator
Is it backward compatible (if not, which system it affects?)
YES
Related PRs
#11870
#11879
External dependencies / deployment changes
No