Skip to content

fix(storage): skip docdb watcher reconfiguration on empty file read - #1769

Merged
tekton-robot merged 1 commit into
tektoncd:mainfrom
infernus01:fix-docdb
Jul 8, 2026
Merged

fix(storage): skip docdb watcher reconfiguration on empty file read#1769
tekton-robot merged 1 commit into
tektoncd:mainfrom
infernus01:fix-docdb

Conversation

@infernus01

Copy link
Copy Markdown
Member

Changes

Fix flaky TestWatchBackend that fails intermittently in the release pipeline but passes locally.

Problem

os.WriteFile is not atomic — it truncates the file (O_TRUNC) writing new content. When the fsnotify watcher goroutine reads the file during this truncation window, it gets an empty string. This causes:

  1. MONGO_SERVER_URL env var set to ""
  2. docstore.OpenCollection fails: "MONGO_SERVER_URL environment
    variable is not set"
  3. Watcher sends nil to the unbuffered backendChan and blocks
  4. Subsequent fsnotify events can never be processed — deadlock

The race is timing-dependent: on a fast local machine the watcher processes events before the next file write starts; on a busy CI node the truncation window is wide enough to hit.

Fix

Skip reconfiguration when the file read returns an empty value. An empty MONGO_SERVER_URL is never a valid configuration — reconfiguring with it always fails. In production, Kubernetes secret mounts use atomic symlink swaps (..data), so transient empty reads don't occur; this guard makes the watcher robust to non-atomic writes without changing any valid production behavior.

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs included if any changes are user facing
  • Has Tests included if any functionality added or changed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including
    functionality, content, code)
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

/kind bug

Signed-off-by: Shubham Bhardwaj <shubbhar@redhat.com>
@tekton-robot tekton-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jul 7, 2026
@tekton-robot
tekton-robot requested review from vdemeester and wlynch July 7, 2026 14:50
@tekton-robot tekton-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jul 7, 2026
@jkhelil

jkhelil commented Jul 8, 2026

Copy link
Copy Markdown
Member

/approve

@tekton-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jkhelil

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 8, 2026
@jkhelil

jkhelil commented Jul 8, 2026

Copy link
Copy Markdown
Member

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 8, 2026
@tekton-robot
tekton-robot merged commit 7ebcb78 into tektoncd:main Jul 8, 2026
19 of 20 checks passed
@jkhelil

jkhelil commented Jul 8, 2026

Copy link
Copy Markdown
Member

/cherry-pick release-v0.28.x

@tekton-robot

Copy link
Copy Markdown

Cherry-pick to release-v0.28.x successful!

A new pull request has been created to cherry-pick this change to release-v0.28.x.

PR: #1770

Please review and merge the cherry-pick PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants