Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(github): Add 30 minutes debounce between cache syncs #27058

Merged

Conversation

zharinov
Copy link
Collaborator

@zharinov zharinov commented Feb 4, 2024

Changes

  • We always fetch at least the first page during each run for each package
  • Most packages contain <100 releases/tags, so effectively we always fetch all the data (except for repos like Renovate)

This PR prevents any sync for 30 minutes time period, if the package is public, by caching additional flag related to package ability to be persistent (i.e. when it's public).

Context

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@zharinov zharinov requested a review from rarkins February 4, 2024 19:33
@zharinov zharinov requested a review from viceice February 4, 2024 19:51
Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blank review to ensure I test it thoroughly!

viceice
viceice previously approved these changes Feb 4, 2024
@rarkins
Copy link
Collaborator

rarkins commented Feb 8, 2024

When running against renovate-reproductions/t2 I get:

 WARN: Package lookup failures (repository=renovate-reproductions/t2, branch=renovate/actions-upload-artifact-4.x)
       "warnings": [
         "Could not determine new digest for update (github-tags package actions/cache)",
         "Could not determine new digest for update (github-tags package DavidAnson/markdownlint-cli2-action)",
         "Could not determine new digest for update (github-tags package codecov/codecov-action)",
         "Could not determine new digest for update (github-tags package actions/setup-node)"
       ],
       "files": [".github/workflows/build.yml"]

I don't get that on main branch

@rarkins
Copy link
Collaborator

rarkins commented Feb 8, 2024

On main branch, dependency.releases.length is 55, on this PR it's 50 and missing v4.0.0.

When I reset my /tmp/renovate to clear cache, it worked (both cold and hot cache).

I attempted to reproduce the problem by first running on main and then switching to the PR branch, but the lookup warnings aren't showing anymore. However I did see an autoclosing of this PR which I bet is the same root cause: renovate-reproductions/t2#4

And this PR gets edited to have a lower version:

image

@zharinov
Copy link
Collaborator Author

zharinov commented Feb 8, 2024

I've managed to reproduce it, looking for a cause

Copy link
Member

@viceice viceice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷‍♂️

@rarkins rarkins added this pull request to the merge queue Feb 18, 2024
Merged via the queue into renovatebot:main with commit 3ffebd6 Feb 18, 2024
35 checks passed
@rarkins rarkins deleted the feat/github-graphql-30-min-cache-sync-debounce branch February 18, 2024 16:01
@renovate-release
Copy link
Collaborator

🎉 This PR is included in version 37.197.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cache public github tags/releases
5 participants