ci: temporary go1.25.8 toolchain pin for tiflow/ticdc unit jobs

[lybcodes]

Summary

This PR adds a temporary CI-side workaround for mixed Go patch toolchain artifacts (go1.25.8 vs older go tool) that cause compile failures in PR jobs.

Why

Some jobs are still running in environments/images that are not fully aligned to Go 1.25.8, while repo-side changes already pull 1.25.8 artifacts. This can trigger errors like:

compile: version "go1.25.8" does not match go tool version "go1.25.x"

Changes

• pipelines/pingcap/tiflow/latest/ghpr_verify.groovy

  • export GOTOOLCHAIN=go1.25.8
  • set per-command GOCACHE
  • run go clean -cache before make
  • add explicit TEMP/TODO comments

• prow-jobs/pingcap/ticdc/latest-presubmits.yaml

• prow-jobs/pingcap/ticdc/latest-presubmits-next-gen.yaml

  • same temporary env/cache workaround for pull-unit-test and pull-unit-test-next-gen

Notes

• This is intentionally temporary and should be reverted after the corresponding CI images/environments are fully upgraded and stable on Go 1.25.8.
• This does not change product code or test logic; it only stabilizes CI execution environment consistency.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a temporary workaround to address build failures in the ghpr_verify CI pipeline caused by an inconsistent Go toolchain environment. By explicitly setting the Go version and clearing the build cache, it ensures a stable and predictable build process, mitigating issues until the underlying CI environment is fully upgraded.

Highlights

• Toolchain Pinning: Temporarily pinned the Go toolchain to go1.25.8 within the ghpr_verify pipeline to prevent issues arising from mixed Go versions in the CI environment.
• Cache Management: Implemented a step to clear the GOCACHE before running make commands, ensuring that stale artifacts from different toolchains do not interfere with builds.
• Debugging Information: Added commands to print go version and relevant Go environment variables (GOTOOLCHAIN, GOROOT, GOCACHE) for improved debugging capabilities in the CI.

Changelog

• pipelines/pingcap/tiflow/latest/ghpr_verify.groovy
  • Modified the sh script block to include explicit Go toolchain pinning.
  • Added GOCACHE clearing to prevent mixed-version build artifacts.
  • Included go version and go env commands for better debugging.

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary
This PR introduces a temporary workaround in the ghpr_verify.groovy pipeline script to pin the Go toolchain to version go1.25.8 during test commands execution. It clears the Go build cache per test matrix entry to avoid stale or mixed artifacts from different Go patch versions, and prints Go environment details for debugging purposes. The approach is straightforward and well-isolated, with clear comments indicating that this is a temporary measure. Overall, the change is low-risk and improves CI stability under the current environment inconsistency.

---

Code Improvements

• Avoid hardcoding toolchain version multiple times

  • File: pipelines/pingcap/tiflow/latest/ghpr_verify.groovy (lines ~50-60)
  • Issue: The GOTOOLCHAIN version go1.25.8 is hardcoded directly in the script. If this version needs to be updated, it requires editing the script in multiple places (e.g., comment and export).
  • Suggestion: Extract the version string into a variable at the start of the pipeline or script block for easier updates and reduce duplication. For example:

    def goToolchainVersion = "go1.25.8"
    ...
    export GOTOOLCHAIN=${goToolchainVersion}

• Ensure that cache clearing is robust

  • File: same as above
  • Issue: The cache clearing step uses rm -rf "$GOCACHE" but does not verify if $GOCACHE is set or non-empty, which could be risky if the environment changes.
  • Suggestion: Add a guard in the script to confirm $GOCACHE is non-empty and points to an expected directory before removing it, e.g.:

    if [[ -n "$GOCACHE" && "$GOCACHE" == /tmp/go-build-* ]]; then
      rm -rf "$GOCACHE"
    fi

    This avoids accidental deletion of unexpected directories.

• Use consistent quoting and escaping

  • File: same
  • Issue: The rm -rf "\$GOCACHE" uses escaped quotes inside a Groovy triple-quoted string, which can be confusing and error-prone.
  • Suggestion: Since this is a triple double-quoted Groovy string, you can use single quotes inside the shell script to simplify:

    rm -rf '$GOCACHE'

---

Best Practices

• Add a TODO issue or link for reverting this temporary fix

  • File: ghpr_verify.groovy (lines ~45-65)
  • Issue: The comment notes this is temporary and should be reverted, but no tracking or issue reference is provided.
  • Suggestion: Add a TODO with a GitHub issue or PR number to ensure this workaround is not forgotten, e.g.:

    // TODO (pingcap/tiflow#XXXX): Remove this pin after ghpr_verify golang image is fully upgraded to go1.25.8.

• Improve logging clarity

  • File: same
  • Issue: The go version and go env ... output is printed but not explicitly labeled in logs, which might make it harder to identify in CI output.
  • Suggestion: Add echo statements or log prefixes to clarify what is being printed, e.g.:

    echo "Using Go version:"
    go version
    echo "Go environment:"
    go env GOTOOLCHAIN GOROOT GOCACHE

• Testing coverage

  • Since this is a pipeline script change, ensure that the pipeline has been tested in the CI environment to verify that the mixed Go versions issue is resolved and that no new failures occur due to toolchain pinning.

---

Minor Style Suggestions

• Add a newline at the end of the script block for better readability.
• Keep consistent indentation inside the multiline shell script block.

---

Overall, the PR is concise and addresses the urgent problem effectively. Incorporating the above improvements will increase maintainability and safety of this temporary workaround.

[gemini-code-assist]

Code Review

This pull request introduces a temporary workaround to pin the Go toolchain version in the ghpr_verify pipeline, addressing build failures caused by mixed Go versions. The changes are clear and well-documented with comments explaining their temporary nature. The suggestion to use go clean -cache for clearing the Go cache is retained as it promotes a more robust and idiomatic approach.

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary
This PR temporarily pins the Go toolchain version to 1.25.8 in the ghpr_verify.groovy Jenkins pipeline script to avoid compilation errors arising from mixed Go versions (1.25.6 and 1.25.8) artifacts. It also clears the Go build cache before make and prints Go version info for debugging. The approach is straightforward and limited in scope, mostly modifying the shell script that runs tests. Overall, the change is clear, purposeful, and well-documented within the script and PR description.

---

Code Improvements

• Pipeline shell script robustness (pipelines/pingcap/tiflow/latest/ghpr_verify.groovy, lines ~49-63)

  • The script uses multiline shell commands inside sh step. To improve maintainability and error visibility, consider:
    • Adding explicit error messages on failure of key commands (e.g., go version, go clean, make).
    • Avoid exporting GOCACHE to a static directory that could cause race conditions if tests run concurrently. Instead, use a unique temporary directory per run/command, e.g.:

      export GOCACHE=$(mktemp -d /tmp/go-build-${TEST_CMD}-XXXXXX)

    • Adding go clean -cache is good, but also consider go clean -modcache if module cache is suspected to cause stale artifacts.
  • This will further reduce potential flaky failures caused by cache pollution.

• Use of set -euo pipefail

  • It's a good practice to avoid silent failures. Ensure this is consistently used in all similar shell commands in the pipeline for uniformity.

---

Best Practices

• Documentation (pipelines/pingcap/tiflow/latest/ghpr_verify.groovy, around line 50)

  • The inline comments are clear and helpful.
  • However, add a reminder or TODO comment about removing this workaround once the environment upgrade is complete, ideally linking to a tracking issue or PR to avoid forgetting temporary fixes.
    Example:

  # TODO: Remove this pin once ghpr_verify environment is fully on go1.25.8 (see PR #12560)

• Testing coverage

  • Since this change affects the CI pipeline itself, ensure that the pipeline is tested/re-run fully after this change. The PR description links a replay test, which is good. Make sure the test covers multiple matrix configurations to validate no cross-version caching issues remain.

---

Minor Suggestions

• Print the Go environment variables in a single command
  Instead of separate go version and go env GOTOOLCHAIN GOROOT GOCACHE, consider:

  go version && go env GOTOOLCHAIN GOROOT GOCACHE

  for slightly cleaner output and execution.

• Avoid potential shell injection
  If ${TEST_CMD} can come from untrusted input in the future, consider quoting or sanitizing it to prevent shell injection. Currently, this is probably safe.

---

No critical issues found as this is a temporary, isolated fix.
Overall, the PR is well-constructed and addresses a specific CI pain point effectively.

[ti-chi-bot]

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary:
This PR introduces a temporary workaround to pin the Go toolchain version to go1.25.8 in multiple CI-related scripts (ghpr_verify.groovy and two ticdc presubmit YAML files) to avoid version mismatch errors caused by mixed Go versions (go1.25.6 vs go1.25.8). It also clears per-matrix GOCACHE before running make and adds debugging output for Go environment details. The changes are clearly scoped as temporary and include TODO comments for later removal. Overall, the approach is practical and addresses the immediate CI instability issue effectively.

---

1. Critical Issues

• No critical bugs or security issues found.

---

2. Code Improvements

• Consistency in go clean command:

  • Files updated:
    • pipelines/pingcap/tiflow/latest/ghpr_verify.groovy line ~51
    • prow-jobs/pingcap/ticdc/latest-presubmits.yaml line ~160
    • prow-jobs/pingcap/ticdc/latest-presubmits-next-gen.yaml line ~112
  • Issue:
    • go clean -cache is used, but the correct flag is -cache (single dash). The current usage seems correct, but to clarify:
      • go clean -cache is valid, but usually go clean -cache is equivalent to go clean -cache=true.
    • However, calling go clean -cache is fine as per go clean docs.
  • Suggestion:
    • Add -modcache cleanup if module cache could also cause stale artifacts, e.g. go clean -cache -modcache.
    • Or explicitly clear GOCACHE directory by deleting it (e.g., rm -rf $GOCACHE) before running make for stronger guarantee.

• Avoid redundant environment variable export:

  • In ghpr_verify.groovy, export GOCACHE=/tmp/go-build-${TEST_CMD} is set, but GOCACHE is also set in ticdc presubmit scripts to fixed paths.
  • Suggestion:
    • Consider consistent naming or document why different paths are used per context.

• Shell script improvements in groovy:

  • In ghpr_verify.groovy (around line 51), the shell block uses set -euo pipefail but no IFS setting.
  • Suggestion:
    • Add IFS=$'\n\t' to make shell scripts more robust to whitespace issues.

• Print full Go environment variables for debugging:

  • Currently only prints GOTOOLCHAIN, GOROOT, and GOCACHE.
  • Suggestion:
    • Consider printing go env fully or key variables like GOPATH, GO111MODULE to help debugging.

---

3. Best Practices

• Documentation & comments:

  • Good TODO comments are present.
  • Suggestion:
    • Add a short comment on why GOCACHE is set to a unique path per matrix (/tmp/go-build-${TEST_CMD}) to help future maintainers.

• Testing coverage:

  • Since this is a CI pipeline change, verify that the pipeline runs succeed and are stable.
  • Suggestion:
    • Add a note or link to test runs in PR description (which is done, good).

• Styling:

  • In YAML files, indentations appear consistent.
  • In Groovy, multi-line shell script is well formatted.

• Naming:

  • Variables like GOTOOLCHAIN, GOCACHE are standard and used appropriately.

---

Summary of suggestions for next steps:

# In shell scripts, consider stronger cache cleanup:
export GOCACHE=/tmp/go-build-${TEST_CMD}
rm -rf "$GOCACHE"
go clean -cache -modcache

# Add IFS setting for robustness:
set -euo pipefail
IFS=$'\n\t'

# Print more go env info:
go env

And add clarifying comments about the per-matrix cache paths.

---

This PR addresses an urgent CI flakiness problem with a clean and revertible workaround. The above improvements can make the temporary changes more robust and easier to maintain.

[dillon-zheng]

lgtm, for Urgent code freese

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dillon-zheng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• pipelines/OWNERS [dillon-zheng]
• prow-jobs/OWNERS [dillon-zheng]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-03-17 11:04:10.513769468 +0000 UTC m=+268577.601427015: ☑️ agreed by dillon-zheng.