Core & Internals: rework tombstone handling. Closes #4491, #4436 and #4188 #4594
Conversation
|
The first commit was submitted as a separate PR (#4577) |
rcarpa
force-pushed
the
patch-4491-default_tombstone_replicas
branch
3 times, most recently
from
b2a1367
to
8ea8d13
Compare
There was a problem hiding this comment.
I think there is a misconception here 😄
The default_tombstone should only be applied for RSEs which have the tombstone_delay set. For all the other RSEs, current behaviour should be kept. As far as as I can see here, we would put the 2 week tombstone on everything which does not have a tombstone_delay set.
For the intermediate replicas of multihops I think we need to do it a bit different. There we should have a global (config table) variable which specifies the tombstone delay.
rcarpa
force-pushed
the
patch-4491-default_tombstone_replicas
branch
from
8ea8d13
to
cd1201b
Compare
|
I updated thing to not set any default tombstone on normal replica creation. And to set it two hours into the future for the multihop tombstone. I'll work on a separate commit now to make the multihop_tombstone_delay also available as a config variable, which will be used for RSEs which don't have it set per-rse. |
rcarpa
force-pushed
the
patch-4491-default_tombstone_replicas
branch
2 times, most recently
from
2d16ac5
to
41a9fa3
Compare
…#4436 and rucio#4188 Allow setting a default tombstone on any replica creation. Introduce two rse attributes which allow to customize the behavior per rse: - One allows to define the normal replica creation tombstone (defaults to 0: "no tombstone") - The other used to configure the multihop temporary replica tombstone (defaults to 2 hours in the future) Use a negative value to set the epoch tombstone. Or 0 to explicitly set a null tombstone. Protect replicas which are used as sources from deletion even if their tombstone is set to epoch. Add an integration test which verifies this behavior. Add a submitter test ensuring that multihop sources are created; and that the temp multihop replicas are created with a tombstone. Remove the redundant behavior which touches the tombstone in the finisher. The 2 remaining protections for intermediate multihop replicas are now: 1) entries in the source table; 2) the default multihop tombstone delay of 2 hours into the future. introduce multihop_transfer_delay configuration: To allow setting a default value for all RSEs used in multihops.
rcarpa
force-pushed
the
patch-4491-default_tombstone_replicas
branch
from
41a9fa3
to
f38bfe3
Compare
|
Merging as a feature (For 1.26) |
Allow setting a default tombstone on any replica creation. Introduce two
rse attributes which allow to customize the behavior per rse:
(defaults to 0: "no tombstone")
(defaults to 2 hours in the future)
Use a negative value to set the epoch tombstone. Or 0 to explicitly
set a null tombstone.
Protect replicas which are used as sources from deletion even if their
tombstone is set to epoch. Add an integration test which verifies this
behavior.
Add a submitter test ensuring that multihop sources are created;
and that the temp multihop replicas are created with a tombstone.
Remove the redundant behavior which touches the tombstone in the
finisher. The 2 remaining protections for intermediate multihop replicas
are now: 1) entries in the source table; 2) the default multihop
tombstone delay of 2 hours into the future.