Renovate 37.219.0 cannot find its dashboard anymore in repositories with a lot of PRs/issues

[mtardy]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us what version of Renovate you run.

37.219.0

If you're self-hosting Renovate, select which platform you are using.

github.com

Was this something which used to work for you, and then stopped?

It used to work, and then stopped

Describe the problem

I originally posted here, in the presumed culprit PR #26793 but I think this is a bug and might be appropriate to have a discussion.

Hey, this morning, 37.219.0 ran on my repository and I think this is the reason why Renovate didn't find its dashboard issue and recreated a new one.

Yesterday evening I got:

DEBUG: findIssue(Dependency Dashboard) (repository=cilium/tetragon)
DEBUG: Retrieving issueList (repository=cilium/tetragon)
DEBUG: Retrieved 1 issues (repository=cilium/tetragon)
DEBUG: Found issue 1054 (repository=cilium/tetragon)
DEBUG: http cache: saving https://api.github.com/repos/cilium/tetragon/issues/1054 (etag=W/"4f3474e44487e1c736d399d19e372a5f6d8280f1e8e733bbbc2e4a46666820d3", lastModified=Tue, 27 Feb 2024 17:08:41 GMT) (repository=cilium/tetragon)

This morning:

DEBUG: findIssue(Dependency Dashboard) (repository=cilium/tetragon)
DEBUG: Retrieving issueList (repository=cilium/tetragon)
DEBUG: http cache: saving https://api.github.com/repos/cilium/tetragon/issues?creator=cilium-renovate[bot]&state=all (etag=W/"f6c20b2f8fe7c51530b52293ea1fb943ee97c3d570f3310591903b76fb27c79f", lastModified=undefined) (repository=cilium/tetragon)
DEBUG: Retrieved 300 issues (repository=cilium/tetragon)

So it recreated a new issue and now it seems the original one is not closed but abandoned (because it doesn't see it):

• original one Dependency Dashboard cilium/tetragon#1054
• new one Dependency Dashboard cilium/tetragon#2159

I'm not sure what will happen if I rollback and keep both dashboards.

I can't create a minimal repro without creating hundreds of renovate PRs I guess. You can check https://github.com/cilium/tetragon.

I think the issue was already highlighted in the PR:

sadly the rest API returns pulls as issues 😔 so I don't think this will really help as @zharinov already said.

maybe hide the change behind a feature flag, so the app can test it on some orgs first?

A colleague of mine mentioned that it might be the issue:

graphql -> listing issues only
rest api -> lists issues and PRs

That + a pagination issue might explain why it could not find its own "old" dashboard.

Relevant debug logs

No response

Have you created a minimal reproduction repository?

I have explained in the description why a minimal reproduction is impossible

[rarkins]

Renovate fetches at most 1000 results (100 * 10 pages), and I see that there's a gap of 1004 between the two issue numbers you link to, which is potentially not a coincidence.

Related:

• The query should filter by creator though. How many issues/PRs are there by your renovate bot?
• I note you're using an app token, this could be relevant
• 300 queries is a suspiciously round number. Maybe one bug here is we're defaulting to 30 per page so that's 30*10
• Does the bot find the new issue on the next run? i.e. not creating a new one over and over?

I think a fix from 30 per page to 100 per page may work around this problem for some time

[mtardy]

The query should filter by creator though. How many issues/PRs are there by your renovate bot?

Searching for authored issues/PRs by renovate, it's a bit beyond 300, total 313.

Does the bot find the new issue on the next run? i.e. not creating a new one over and over?

Yes (see full log here), it seems that it just cannot find the previous one:

DEBUG: findIssue(Dependency Dashboard) (repository=cilium/tetragon)
DEBUG: Retrieving issueList (repository=cilium/tetragon)
DEBUG: http cache: saving https://api.github.com/repos/cilium/tetragon/issues?creator=cilium-renovate[bot]&state=all (etag=W/"97c945840672d2f438f295cdf09170307c4e97c261aadaed507344b3aed8ad6f", lastModified=undefined) (repository=cilium/tetragon)
DEBUG: Retrieved 300 issues (repository=cilium/tetragon)
DEBUG: Found issue 2159 (repository=cilium/tetragon)

[mtardy]

I don't know the details but it seems that the GraphQL query only fetched the issues and thus you ended up not needing to access a lot of pages. It seemed you used REST for caching reasons, maybe there's a way to filter by issue as well using REST? not sure...

[rarkins]

I couldn't find any available option in the issues REST api for saying pulls=false so have reverted the refactor for now. Thanks for reporting the problem. I think you'll find after upgrade that Renovate will autoclose the older one

[mtardy]

Indeed, I'm almost sad for the old dashboard.

[rarkins]

You could close the new one and reopen the old

[rarkins]

I'm going to revert this change for now: #27612