[Improvement] Fix policy version limit being applied per policy instead of globally

[justinmclean]

What would you like to be improved?

PolicyMetaService.deletePolicyVersionsByRetentionCount(versionRetentionCount, limit) applies limit inside a loop for each policy. In production, this is called by the relational garbage collector with GARBAGE_COLLECTOR_SINGLE_DELETION_LIMIT (100). With multiple policies, one GC iteration can delete up to limit * numberOfPolicies, exceeding the intended single-batch cap and causing unexpectedly large cleanup operations.

How should we improve?

Enforce limit as a global budget per method call. Track remaining = limit, and for each policy call softDeletePolicyVersionsByRetentionLine(..., remaining), subtracting deleted rows from remaining; stop when remaining <= 0. This preserves bounded batch behavior while still progressing cleanup across runs.

here a test to help:

@TestTemplate
public void testDeletePolicyVersionsByRetentionCountRespectGlobalLimit() throws IOException {
  BaseMetalake metalake = createAndInsertMakeLake(METALAKE_NAME + "_global_limit");

  PolicyEntity policy1 =
      createPolicy(
          RandomIdGenerator.INSTANCE.nextId(),
          NamespaceUtil.ofPolicy(metalake.name()),
          "policy_limit_1",
          AUDIT_INFO);
  PolicyEntity policy2 =
      createPolicy(
          RandomIdGenerator.INSTANCE.nextId(),
          NamespaceUtil.ofPolicy(metalake.name()),
          "policy_limit_2",
          AUDIT_INFO);
  backend.insert(policy1, false);
  backend.insert(policy2, false);

  PolicyEntity policy1V2 =
      PolicyEntity.builder()
          .withId(policy1.id())
          .withNamespace(policy1.namespace())
          .withName(policy1.name())
          .withPolicyType(policy1.policyType())
          .withComment(policy1.comment())
          .withEnabled(!policy1.enabled())
          .withContent(policy1.content())
          .withAuditInfo(AUDIT_INFO)
          .build();
  PolicyEntity policy2V2 =
      PolicyEntity.builder()
          .withId(policy2.id())
          .withNamespace(policy2.namespace())
          .withName(policy2.name())
          .withPolicyType(policy2.policyType())
          .withComment(policy2.comment())
          .withEnabled(!policy2.enabled())
          .withContent(policy2.content())
          .withAuditInfo(AUDIT_INFO)
          .build();
  backend.update(policy1.nameIdentifier(), Entity.EntityType.POLICY, e -> policy1V2);
  backend.update(policy2.nameIdentifier(), Entity.EntityType.POLICY, e -> policy2V2);

  int deletedCount = PolicyMetaService.getInstance().deletePolicyVersionsByRetentionCount(1L, 1);

  assertEquals(1, deletedCount);
}