Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Our tests should run in a certain order to save resources #3906

Closed
1 task done
waynehamadi opened this issue May 6, 2023 · 3 comments
Closed
1 task done

Our tests should run in a certain order to save resources #3906

waynehamadi opened this issue May 6, 2023 · 3 comments

Comments

@waynehamadi
Copy link
Contributor

Duplicates

  • I have searched the existing issues

Summary 💡

Create a mode in pytest that runs the tests in a certain order and fails the first time any of the tests fails.

We want to use this test to run our challenges. This way we don't waste time on tests that don't even pass unit tests.

Here is a suggestion for the order.
1- UNIT TESTS
2- BAR RAISERS: Challenges that already pass in master but we know are hard to beat.
3- Others.

We can start with this ordering. We could use this test as the bar raiser, since it doesn't look easy to achieve.

Examples 🌈

No response

Motivation 🔦

No response

@Boostrix
Copy link
Contributor

Boostrix commented May 6, 2023

We could use this #3901 as the bar raiser, since it doesn't look easy to achieve.

FWIW, I believe we can work out a way to run an outer agent to mutate those specs in project.txt to create several different versions of the project using different "styles" (think CSS, colors for now) and other attributes (file names), that should then also work with the unit testing portion we're currently drafting to "test" the result afterwards (file names, validate HTML, run XPath on the DOM of the HTML and do a back-end test using an actuall HTTPServer instance) - so if that works out well enough, the same project.txt /COULD/ be used to create different variations of the same project in a loop - such as adding different HTML elements/widgets, which would obviously a good stress-test while increasing code coverage.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

@github-actions github-actions bot added the Stale label Sep 6, 2023
@github-actions
Copy link
Contributor

This issue was closed automatically because it has been stale for 10 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants