Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the ability to quicken the transition of STUCK rules to REPLICATING #4634

Closed
dchristidis opened this issue May 20, 2021 · 1 comment · Fixed by #4929
Closed

Add the ability to quicken the transition of STUCK rules to REPLICATING #4634

dchristidis opened this issue May 20, 2021 · 1 comment · Fixed by #4929
Assignees
@joeldierkes
Labels
enhancement Rules
Milestone
1.26.9

Comments

@dchristidis
Copy link
Contributor

Motivation

Following a rucio update-rule --stuck --cancel-requests __RULE__, there might be a considerable delay (up to 8 hours; see here) until the requests are recreated. Currently, one needs to manipulate the updated_at timestamps of the STUCK locks and the replication rule itself.

Modification

Consider adding a new option in the client and the necessary implementation in the various layers to quicken the transition from STUCK to REPLICATING. This operation should be reserved to accounts with elevated privileges.
@bari12
Copy link
Member

bari12 commented May 20, 2021 •
edited

I think we could add a rucio update-rule --boost-rule <ruleid> option.
That option needs to do the following steps:

  • For each STUCK lock it needs to set the updated_at time to 24h prior.
  • Set the updated_at of the rule to 24h earlier.
    This will force, in the next run of the repairer, that the requests are immediately resubmitted.

We need to consider that this might be executed together with --stuck and --cancel-requests thus:
rucio update-rule --stuck --cancel-requests --boost-rule <rule_id> so this needs to be considered in the update_rule part of the core.

@bari12 bari12 assigned joeldierkes and unassigned jwackito Oct 13, 2021
joeldierkes pushed a commit to joeldierkes/rucio that referenced this issue Oct 21, 2021
Core: Add update_rule boost option rucio#4634
e433ef8 
The timeout between the retries of the transitioning of a rule from `STUCK` to
`REPLICATING` are quite big. This commit introduces a feature `--boost-rule` to
allow almost instant transition between the states.
joeldierkes pushed a commit to joeldierkes/rucio that referenced this issue Oct 22, 2021
Core: Add update_rule boost option rucio#4634
7ee4fe6 
The timeout between the retries of the transitioning of a rule from `STUCK` to
`REPLICATING` are quite big. This commit introduces the feature `--boost-rule`
to allow almost instant transition between the states.
joeldierkes pushed a commit to joeldierkes/rucio that referenced this issue Oct 22, 2021
Core: Add update_rule boost option rucio#4634
8c827b8 
The timeout between the retries of the transitioning of a rule from `STUCK` to
`REPLICATING` are quite big. This commit introduces the feature `--boost-rule`
to allow almost instant transition between the states.
@joeldierkes joeldierkes linked a pull request Oct 22, 2021 that will close this issue
Core: Add update_rule boost option #4634 #4929
Merged
joeldierkes pushed a commit to joeldierkes/rucio that referenced this issue Oct 26, 2021
Core: Add update_rule boost option rucio#4634
e065ba8 
The timeout between the retries of the transitioning of a rule from `STUCK` to
`REPLICATING` are quite big. This commit introduces the feature `--boost-rule`
to allow almost instant transition between the states.
bari12 added a commit that referenced this issue Oct 29, 2021
@bari12
Merge pull request #4929 from joeldierkes/feature-4634-Add_the_abilit…
374eff9 
…y_to_quicken_the_transition_of_STUCK_rules_to_REPLICATING

Core: Add update_rule boost option #4634
bari12 pushed a commit that referenced this issue Oct 29, 2021
@bari12
Core: Add update_rule boost option #4634
2c61ae6 
The timeout between the retries of the transitioning of a rule from `STUCK` to
`REPLICATING` are quite big. This commit introduces the feature `--boost-rule`
to allow almost instant transition between the states.
@bari12 bari12 added this to the 1.26.9 milestone Oct 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@joeldierkes joeldierkes

Labels
enhancement Rules
Projects
None yet
Milestone
1.26.9
Development

Successfully merging a pull request may close this issue.

Core: Add update_rule boost option #4634 joeldierkes/rucio
4 participants
@jwackito @bari12 @dchristidis @joeldierkes