This WMAgent release has a major change affecting stage in/out, where storage JSON has been adopted and the XML format is now deprecated.
In addition, it has a fully functional WorkflowUpdater component, which continuously updates workflow sandboxes with up-to-date pileup information (artifact of the partial pileup feature). On this pileup feature, the agent no longer resolves pileup data location via Rucio, it now fetches this information solely from the MSPileup service.
Furthermore, the Lexicon file has also been updated to support datatiers with digits (e.g. L1SCOUT). Another feature added is the support to EL9 workflows and the automatic detection of container OS during runtime.
Lastly, a few important bugfixes and enhancements are provided with this release.
Release date: 30 April 2024.
Changes since release: 2.3.1.
WMAgent
Software stack
Features and/or feature changes
- Created MSPileupUtils module and modified GQ CherryPy thread to use MSPileup instead of Rucio (Dennis Lee) #11870
- New function to get dataset locations from MSPileup and its implementation in StartPolicyInterface #11620 (Andrea Piccinelli) #11879
- add support for rhel9. Fix #11985 (Stefano Belforte) #11897
- Add pileup availability logic in WorkflowUpdater component (Valentin Kuznetsov) #11884
- Abandon storage.xml in favor of storage.json for stage-in and stage-out (nhduongvn) #11869 #11917
- Initial implementation of DBSConcurrency module (Valentin Kuznetsov) #11913
- Allow digits in data tier names (German Giraldo) #11930
- Fix regex for Spec generation with alphanumerical data tiers (German Giraldo) #11951
- Add new Merge Repack special cases to AccountantWorker (German Giraldo) #11952
- Allow numbers in DataTier fields in all regular expressions (Todor Ivanov) #11962
- [Container] Addition of the sites with HPC resources for the WMAgent docker deployment (Andrea Piccinelli) #11966
Bug Fixes
- Dont cross check location when data is reported without location (Alan Malta Rodrigues) #11878
- Discover the singularity container OS at runtime. (Todor Ivanov) #11896
- Decode cmsRun stdout (German Giraldo) #11933
- Fix print function in WorkflowUpdater (Alan Malta Rodrigues) #11971
- Add DBS3_READER_URL and WorkflowUpdater.dbsUrl (Valentin Kuznetsov) #11972
- Add check for creating updateBlockInfo return dictionary (Author: Todor Ivanov) #11975
Enhancements
- Increase JobSubmitter queue size from 50k to 100k (Alan Malta Rodrigues) #11889