Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How people can contribute to Pharo if some of the tests on the base image are not passing? #5227

Open
VincentBlondeau opened this issue Nov 26, 2019 · 4 comments

Comments

@VincentBlondeau
Copy link
Contributor

@VincentBlondeau VincentBlondeau commented Nov 26, 2019

I do not know how people can contribute to a system with tests that are continuously red?
For a newcomer to this community, all the tests passing means that the code he wrote is ok and his PR could be merged. But if the base image has already some flaws in it, how can he differentiate his failing tests from the base ones?

On the last week, more than half of the build of Pharo 8.0 were failing. It should be unlikely to happen if the imposed rules of the PR are followed (at least one review, green tests, and benchmarks): almost none of the PR have been merged without a full green check.

image

image

Is it planned to enforce the rules on the PR or should we continue as it is now?

I can concede that some changes are critical and could be passed without all the checks but they should be marginals.

@MarcusDenker

This comment has been minimized.

Copy link
Member

@MarcusDenker MarcusDenker commented Nov 26, 2019

The problem is that the tests are not red because of that PR. They are red because of other problems.

What we do is that we check the failing tests and if the fail is not related, we merge.

Yes, this is not good. But the alternative is to not merge anything.

@MarcusDenker

This comment has been minimized.

Copy link
Member

@MarcusDenker MarcusDenker commented Nov 26, 2019

It would be nice to fix the problems.

The first step is to remove the benchmark run because for the case that the rests are green (which already happens not often) the benchmark is always red.

@VincentBlondeau

This comment has been minimized.

Copy link
Contributor Author

@VincentBlondeau VincentBlondeau commented Nov 26, 2019

Thanks Marcus! So first we may need to solve those deeper problems. Could you link those issues so we can fix them in priority?

I think that the stability of the testing environment is very important.

For the benchmark, it could be still triggered and reported but not flagged as an issue for the test?

@MarcusDenker

This comment has been minimized.

Copy link
Member

@MarcusDenker MarcusDenker commented Nov 26, 2019

I already added a board for the CI: https://github.com/orgs/pharo-project/projects/11

We can use that to have an overview.

GitHub
GitHub is where people build software. More than 40 million people use GitHub to discover, fork, and contribute to over 100 million projects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
2 participants
You can’t perform that action at this time.