Skip to content

ci: add nightly contract tests with long-running tests and Slack notification#136

Merged
tanderson-ld merged 7 commits intomainfrom
devin/1770832864-nightly-contract-tests
Feb 12, 2026
Merged

ci: add nightly contract tests with long-running tests and Slack notification#136
tanderson-ld merged 7 commits intomainfrom
devin/1770832864-nightly-contract-tests

Conversation

@devin-ai-integration
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Feb 11, 2026

Requirements

Related issues

N/A

Describe the solution you've provided

Adds a new GitHub Actions workflow that runs the Java server SDK contract tests nightly (4am UTC) with the -enable-long-running-tests flag enabled. If the tests fail, a Slack notification is sent to alert the team.

Key details:

  • Schedule: Runs daily at 4am UTC via cron, with workflow_dispatch for manual triggering (supports custom branch and Slack test notification inputs)
  • Test execution: Reuses the existing ci and contract-tests composite actions; the contract-tests action now accepts a test_harness_params input to pass -enable-long-running-tests
  • Slack notification: Uses slackapi/slack-github-action (pinned to v2.1.1 SHA) with an incoming webhook (SLACK_WEBHOOK_URL secret) to post to Slack on failure, including a link to the failed run
  • Test Slack job: A separate test-slack-notification job can be triggered manually via workflow_dispatch to verify the Slack integration is working
  • Contract test service updates: Added server-side-polling capability and polling parameter support to the test service entity
  • FDv2 suppressions: Three streaming/fdv2 fallback/recovery tests are suppressed pending spec alignment between Go and Java implementations

Items for reviewer attention:

  1. SLACK_WEBHOOK_URL secret — must be configured in the repo settings with a webhook routed to #sdks-java. The channel is determined by the webhook configuration, not the workflow.
  2. No timeout-minutes — long-running tests could potentially hang indefinitely. Consider adding a timeout.
  3. Notification scope — the Slack job triggers on failure only, not cancelled or timed_out. Decide if those states should also notify.
  4. Contract test service changes — verify the new polling support in SdkClientEntity.java is correct, particularly the conditional logic for params.polling != null && params.dataSystem == null and the base URI endpoint wiring.

Describe alternatives you've considered

  • Could use rtCamp/action-slack-notify (used by ldcli) instead of slackapi/slack-github-action, but the official Slack action is more widely used across LD repos (terraform-provider-launchdarkly, streamer, ld-docs-private).

Updates since initial revision

  • Extended contract-tests composite action with test_harness_params input instead of splitting out individual make targets (addresses original item ci: adding release please support #4)
  • Added workflow_dispatch inputs for branch selection and Slack notification testing
  • Added test-slack-notification job for verifying Slack integration
  • Added server-side-polling capability and polling params to the contract test service
  • Added 3 FDv2 streaming fallback/recovery test suppressions (pending spec alignment)
  • Pinned slackapi/slack-github-action to v2.1.1 SHA (91efab103c0de0a537f72a35f6b8cda0ee76bf0a)

Additional context

Link to Devin run
Requested by: tanderson@launchdarkly.com


Note

Medium Risk
Primarily CI/workflow changes, but it also adjusts contract-test service configuration logic for polling which could alter how tests initialize and exercise the SDK.

Overview
Adds a new Nightly Contract Tests GitHub Actions workflow that runs server SDK contract tests nightly (and on manual dispatch), with an optional branch override and an optional Slack test-notification path.

Extends the contract-tests composite action to accept test_harness_params and pass them through to make, enabling the nightly workflow to run the harness with -enable-long-running-tests, and posts a Slack alert via an incoming webhook when the nightly job fails.

Updates the contract test service to advertise server-side-polling, accept polling config (including a filter), and wire polling endpoints/data source when streaming isn’t used; also adds FDv2 suppressions for several streaming fallback/recovery tests.

Written by Cursor Bugbot for commit 69b9f64. This will update automatically on new commits. Configure here.

…fication

Co-Authored-By: tanderson@launchdarkly.com <tanderson@launchdarkly.com>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

tanderson-ld and others added 3 commits February 11, 2026 15:54
Co-authored-by: semgrep-code-launchdarkly[bot] <167133144+semgrep-code-launchdarkly[bot]@users.noreply.github.com>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewers: These tests are not passing because they are written against the Go impl. I need to update the FDv2 Data System spec to specify the edge case behavior and timings to either match Go or update Go and then update the Java, Dotnet, and Node impls to match that.

@tanderson-ld tanderson-ld marked this pull request as ready for review February 12, 2026 21:07
@tanderson-ld tanderson-ld requested a review from a team as a code owner February 12, 2026 21:07
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.


if (params.polling != null && params.polling.baseUri != null && !params.polling.baseUri.toString().trim().isEmpty()) {
endpoints.polling(params.polling.baseUri);
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Polling endpoint override placed after serviceEndpoints, reversing precedence

Medium Severity

The endpoints.polling(params.polling.baseUri) call at lines 477–479 is placed after the serviceEndpoints block, so it silently overrides any params.serviceEndpoints.polling value. This is the opposite of the streaming pattern, where endpoints.streaming(params.streaming.baseUri) is set inside the if (params.streaming != null) block (line 397), allowing serviceEndpoints.streaming to take precedence later. Additionally, this endpoint override lacks the params.dataSystem == null guard that protects the data source creation at line 404, meaning it can set the global polling endpoint even when a dataSystem is configured. Moving this call inside the else if block would fix both issues and match the streaming pattern.

Additional Locations (1)

Fix in Cursor Fix in Web

@tanderson-ld tanderson-ld self-requested a review February 12, 2026 21:56
@tanderson-ld tanderson-ld merged commit 7271ae2 into main Feb 12, 2026
21 checks passed
@tanderson-ld tanderson-ld deleted the devin/1770832864-nightly-contract-tests branch February 12, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants