Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate repack job splitting to SVN #1965

Closed
hufnagel opened this issue Jul 18, 2011 · 3 comments
Closed

migrate repack job splitting to SVN #1965

hufnagel opened this issue Jul 18, 2011 · 3 comments

Comments

@hufnagel
Copy link
Member

CVS:

T0/src/python/T0/JobSplitting/Repack.py
T0/src/python/T0/State/Database/Oracle/ListFilesForRepack.py

Both job splitting algorithm and main data discovery DAO need to be migrated. Some changes need to be made as well.

The job splitting will change to be based on an input fileset and available files. Data discovery is still going to be non-standard though because it filters on closed lumis, ie. just because a streamer file is available does not mean it can be assigned to a job. Another option is to apply that filter in the data feeders populating the repack input filesets. Then we could get by with a standard data discovery based on available files here.

Data discovery also handles run closeout currently, but we might want to move that to a separate query/DAO. It's somewhat easy to return run status in the current query, but the new one it would be a stretch. Better to break up the queries.

Job splitting will stay more or less the same, but we have to rework where the split parameters are defined. Development can get by with reasonable defaults, but eventually they need to come from the Tier0 configuration.

@drsm79
Copy link

drsm79 commented Aug 17, 2011

metson: Milestone T0 2_0_0 deleted

@hufnagel
Copy link
Member Author

hufnagel: Please Review

@evansde77
Copy link

evansde: (In 323fa41) Adds repack jobsplitting, fixes #1965

From: Dirk Hufnagel Dirk.Hufnagel@cern.ch

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants