Commit statuses/Pull requests checks not available for old commits/Pull request

[smuzaffar]

Select Topic Area

Bug

Body

We noticed that our CI system re-trigger a test for a 2 years old already merged Pull requests. This happened because github did not return any statuses for the commit. I checked antoher PR and commit status/PR checks for that are also missing e.g. see https://api.github.com/repos/cms-sw/cmssw/commits/2ce0ce793d7b89a0e4691f4dfcad2dcf83e09a2c/statuses . This commit was part of cms-sw/cmssw#40608 PR and pull requests checks for this PR are also missing

Is there any global issue with github commits status for old commits?

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[smuzaffar]

has any one else encounter such issue i.e. PR checks/commit statuses for already merged PR start to disappear?

[smuzaffar]

Hi, any one else has seen such issues? All of a sudden all the commit statuses and Pull Requests checks for older PRs have disappeared. I also have opened a support ticket but looks like chat bot there persists and claims that every thing is good but our CI system have been creating commit statuses for over 5 years and I see PRs/commits have disappeared for PRs/commits from 2023

[smuzaffar]

I have checked multiple repositories and looks like commit statuses and pull request checks for commits/pull request before Feb 2024 are not available e.g AIDASoft/DD4hep#1238 pull request ( which was merged on 29th Feb 2024) does contain the Pull Requests checks and commits statuses but AIDASoft/DD4hep#1231 (which was merged on 16th Feb 2024) does not contain Pull request checks and commits statuses

[smuzaffar]

Looks like the 400 days Checks Data Retention Policy which should only apply to checks created by GitHub Actions and third-party GitHub Apps is also being applied to commit statuses.

For https://github.com/cms-sw/cmssw pull requests we do not create pull requests checks but use commit statuses. Any idea why commit statuses are also disappearing after 400 days?

[smuzaffar]

Reply from GH support

I checked internally and can confirm that our retention policy archives and deletes check suites, Actions workflow runs, statuses, and associated data after 400 days.
 
It's important to note that when workflows run, GitHub Actions creates checks—not commit statuses. The Checks tab in pull requests is populated with these checks if you've set them up in your repository. Consequently, the checks created by the workflow runs in your repository and PRs fall within the scope of this policy.
 
As you observed, if a pull request requires checks that have been archived, you'll need to rerun them before you can merge the PR.
 
I hope this clears up the issue. Please let me know if you have any further questions or need additional assistance.

so looks like the 400 days Checks Data Retention Policy also apply to commit statuses

[smuzaffar]

and github support confirms that

The retention policy applies to commit statuses created via the GitHub API as well. The Status API was introduced before the Checks API,
so when we implemented the [400-day Checks Data Retention Policy](https://github.blog/changelog/2022-01-14-updates-to-the-checks-data-retention-policy/?utm_source=chatgpt.com), 
it was applied to commit statuses created by both APIs, including those from GitHub Apps (and Actions, which is a type of GitHub App). 
I confirmed this with our engineering team.