Skip to content

fix: treat 0/0 release targets as 100% pass rate in policy evaluation#1049

Merged
adityachoudhari26 merged 3 commits intomainfrom
claude/issue-1040-20260422-2324
Apr 23, 2026
Merged

fix: treat 0/0 release targets as 100% pass rate in policy evaluation#1049
adityachoudhari26 merged 3 commits intomainfrom
claude/issue-1040-20260422-2324

Conversation

@adityachoudhari26
Copy link
Copy Markdown
Member

When a tier has 0 instances, a minimum pass rate policy (e.g. 80%) was failing because GetSuccessPercentage() returned 0.0 despite the comment saying to treat it as 100% successful. Applied vacuous truth: 0/0 = 100%.

Also fixed GetSuccessPercentageSatisfiedAt() to return the version's creation time (not zero time) when there are no targets, consistent with how evaluateJobSuccessCriteria handles this case.

Fixes #1040

Generated with Claude Code

When a tier has 0 instances, a minimum pass rate policy (e.g. 80%) was
failing because GetSuccessPercentage() returned 0.0 despite the comment
saying to treat it as 100% successful. Applied vacuous truth: 0/0 = 100%.

Also fixed GetSuccessPercentageSatisfiedAt() to return the version's
creation time (not zero time) when there are no targets, consistent with
how evaluateJobSuccessCriteria handles this case.

Fixes #1040

Co-authored-by: Aditya Choudhari <adityachoudhari26@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 23, 2026 17:03
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 23, 2026

Warning

Rate limit exceeded

@adityachoudhari26 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 56 minutes and 45 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 56 minutes and 45 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 416a6c91-323d-4117-af29-537681073b92

📥 Commits

Reviewing files that changed from the base of the PR and between a487ff6 and ec90dd9.

📒 Files selected for processing (3)
  • apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker.go
  • apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker_test.go
  • apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/passrate_test.go
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/issue-1040-20260422-2324

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 23, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ adityachoudhari26
❌ github-actions[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes environment progression pass-rate evaluation for tiers/environments with zero release targets by applying vacuous truth (0/0 ⇒ 100%), and aligns the “satisfied at” timestamp behavior for the empty-target case.

Changes:

  • Update ReleaseTargetJobTracker.GetSuccessPercentage() to return 100.0 when there are no release targets.
  • Update ReleaseTargetJobTracker.GetSuccessPercentageSatisfiedAt() to return Version.CreatedAt (instead of zero time) when there are no release targets.
  • Adjust the unit test expectation for GetSuccessPercentage() with no targets to assert 100%.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker.go Implements vacuous-truth success percentage for empty target sets; returns version creation time for satisfiedAt when no targets.
apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker_test.go Updates the no-target success-percentage test expectation to 100%.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +199 to +200
span.SetAttributes(attribute.Float64("success_percentage", 100.0))
return 100.0 // vacuous truth: 0/0 targets successful
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the 0-target success percentage to 100% will flip PassRateEvaluator behavior for the no-release-targets case. There is an existing unit test (TestPassRateEvaluator_NoReleaseTargets in passrate_test.go) that currently asserts the opposite and will fail, and the minimumSuccessPercentage==0 branch in PassRateEvaluator will now treat a 0-target environment as allowed but set satisfiedAt based on GetEarliestSuccess() (which is zero time when there are no jobs). Consider updating PassRateEvaluator to explicitly handle len(tracker.ReleaseTargets)==0 (e.g., allow and set satisfiedAt to tracker.Version.CreatedAt) and adjust the related tests accordingly.

Suggested change
span.SetAttributes(attribute.Float64("success_percentage", 100.0))
return 100.0 // vacuous truth: 0/0 targets successful
span.SetAttributes(attribute.Float64("success_percentage", 0.0))
return 0.0 // no release targets to evaluate

Copilot uses AI. Check for mistakes.
Comment on lines 244 to 246
if numRt == 0 {
return time.Time{}
return t.Version.CreatedAt
}
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetSuccessPercentageSatisfiedAt now returns t.Version.CreatedAt when there are no release targets. The job tracker unit test TestReleaseTargetJobTracker_GetSuccessPercentageSatisfiedAt_NoReleaseTargets (jobtracker_test.go) currently expects zero time and will fail with this new behavior; it should be updated to assert Version.CreatedAt (or whatever semantics you intend for satisfiedAt in the 0-target case).

Copilot uses AI. Check for mistakes.
@adityachoudhari26 adityachoudhari26 merged commit 677ff01 into main Apr 23, 2026
8 checks passed
@adityachoudhari26 adityachoudhari26 deleted the claude/issue-1040-20260422-2324 branch April 23, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Policy does not advance

3 participants