Delete generated jobsets that are no longer tracked/generated by .jobsets

[basile-henry]

Is your feature request related to a problem? Please describe.
We have a setup where we generate jobsets for each PR. When a PR is closed/merged, the associated jobset is no longer generated by .jobsets so it is disabled and set to hidden automatically by hydra:

hydra/src/lib/Hydra/Helper/AddBuilds.pm

Line 114 in 411e4d0

$project->jobsets->search({ name => { "not in" => \@kept } })->update({ enabled => 0, hidden => 1 });

The issue is that it prevents hydra from garbage collecting any associated build products. This went unnoticed for us for years and is holding on to terabytes worth of useless build products 😅

Describe the solution you'd like
I understand that deleting jobsets (even disabled/hidden ones) can be problematic (see #1006 and #963).
So as an intermediate solution, until these other issues are resolved, I would like to still be able to delete it manually (curl -X DELETE) with the jobset_DELETE endpoint. At the moment no generated jobsets can be deleted:

hydra/src/lib/Hydra/Controller/Jobset.pm

Lines 99 to 101 in 411e4d0

	if (length($c->stash->{project}->declfile)) {
	error($c, "can't modify jobset of declarative project", 403);
	}

Maybe this check could be loosened to allow deleting jobsets that are already disabled and hidden anyway? Or is there some other reason for disallowing modifying generated jobsets I have not considered?

Describe alternatives you've considered
Buying more HDDs 😅