Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regression tester regressed 629865 #15698

Closed
tamiko opened this issue Jul 10, 2023 · 6 comments
Closed

Regression tester regressed 629865 #15698

tamiko opened this issue Jul 10, 2023 · 6 comments
Labels
Regression tester Issues reported by the regression tester bot

Comments

@tamiko
Copy link
Member

tamiko commented Jul 10, 2023 •
edited

[start automated regression testsuite report]

Dear all,

this is the automated regression testsuite reporting a new regression between

Summary:

I have identified the following pull requests as possible candidates:

Notes:

[end automated regression testsuite report]
@tamiko tamiko added Regression tester Issues reported by the regression tester bot High Priority ⚠️ labels Jul 10, 2023
@tamiko tamiko pinned this issue Jul 10, 2023
@peterrum
Copy link
Member

I have found the issue. I'll create a PR in a few minutes.

@peterrum peterrum mentioned this issue Jul 10, 2023
Merged
@drwells
Copy link
Member

drwells commented Jul 10, 2023

Is the plan to have the CI automatically close these once the issue is resolved?

@tamiko tamiko unpinned this issue Jul 10, 2023
@tamiko
Copy link
Member Author

tamiko commented Jul 10, 2023 •
edited

@drwells The current logic of the regression tester CI is the following:

  • it opens an issue (such as this one) if it can identify a regression compared to the previous run 12h prior.
  • it reopens an issue (such as this one) if the testsuite run is not complete and it found an additional regression.
  • (soon) it opens a general "I am unhappy" issue if there are (a) no open regression tester issues and (b) if it regressed compared to a baseline but cannot establish a regression between the last two runs.
  • it closes and unpins all issues starting with "Regression tester regressed " if the last run is complete and compares cleanly to a baseline.

So, you can keep this issue open and it should close it automatically provided that Peter's fix is all we need 😃

@drwells
Copy link
Member

drwells commented Jul 10, 2023

Thanks. Could you add that text to the message it prints up above? I'm probably going to forget how it works at some point in the future.

@tamiko
Copy link
Member Author

tamiko commented Jul 10, 2023

@drwells Done.

@tamiko tamiko mentioned this issue Jul 10, 2023
Merged
@tamiko
Copy link
Member Author

tamiko commented Jul 11, 2023

[automated regression testsuite]

Regression tester reported a clean run compared to baseline. 😊
Closing all issues. 😎

@tamiko tamiko closed this as completed Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Regression tester Issues reported by the regression tester bot
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tamiko @drwells @peterrum