You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Both job splitting algorithm and main data discovery DAO need to be migrated. Some changes need to be made as well.
The job splitting will change to be based on an input fileset and available files. Data discovery is still going to be non-standard though because it filters on closed lumis, ie. just because a streamer file is available does not mean it can be assigned to a job. Another option is to apply that filter in the data feeders populating the repack input filesets. Then we could get by with a standard data discovery based on available files here.
Data discovery also handles run closeout currently, but we might want to move that to a separate query/DAO. It's somewhat easy to return run status in the current query, but the new one it would be a stretch. Better to break up the queries.
Job splitting will stay more or less the same, but we have to rework where the split parameters are defined. Development can get by with reasonable defaults, but eventually they need to come from the Tier0 configuration.
The text was updated successfully, but these errors were encountered:
CVS:
T0/src/python/T0/JobSplitting/Repack.py
T0/src/python/T0/State/Database/Oracle/ListFilesForRepack.py
Both job splitting algorithm and main data discovery DAO need to be migrated. Some changes need to be made as well.
The job splitting will change to be based on an input fileset and available files. Data discovery is still going to be non-standard though because it filters on closed lumis, ie. just because a streamer file is available does not mean it can be assigned to a job. Another option is to apply that filter in the data feeders populating the repack input filesets. Then we could get by with a standard data discovery based on available files here.
Data discovery also handles run closeout currently, but we might want to move that to a separate query/DAO. It's somewhat easy to return run status in the current query, but the new one it would be a stretch. Better to break up the queries.
Job splitting will stay more or less the same, but we have to rework where the split parameters are defined. Development can get by with reasonable defaults, but eventually they need to come from the Tier0 configuration.
The text was updated successfully, but these errors were encountered: