Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add ability to run tests in jenkins #701

Closed
wants to merge 1 commit into from

Conversation

Projects
None yet
2 participants
@pbrezina
Copy link
Member

commented Nov 24, 2018

Jenkins master polls github for new branches and pull requests. When it discovers new pull request or branch or changes to existing pull request or branch it will allocate a jenkins agent and executes pipeline defined in ./Jenkinsfile (in SSSD source) on this agent.

The pipeline executes ./contrib/test-suite/run.sh and archives logs when testing is finished. Script ./contrib/test-suite/run.sh prepares sssd-test-suite, starts the vagrant machines and copy SSSD source code to the client machine. Then it calls ./contrib/test-suite/run-client.sh on the client machine which runs continuous integration tests.

Extending current tests

To extend current testing capabilities, modify ./contrib/test-suite/run.sh and ./contrib/test-suite/run-client.sh to new requirements. These files can be modified by anyone but are considered untrusted from contributor that is not an administrator of SSSD repository. This means that if a public contributor submits a pull request that changes those files, Jenkins will refuse to run tests.

Adding additional distribution to test on

You need to modify ./Jenkinsfile. Simply copy, paste and amend existing Fedora 28 stage. This file is also considered untrusted so only administrators can modify it within a pull request.

You also need to extend sssd-test-suite and prepare vagrant boxes for this distro.

@pbrezina

This comment has been minimized.

Copy link
Member Author

commented Nov 24, 2018

CI tests are now finished. You can see sssd-ci and sssd-ci/fedora28 labels. The details tab for sssd-ci points to our internal jenkins instance that is not publicly accessible (sorry folks). But each distribution that we test on produces logs that we store in amazon cloud and these logs are publicly accesible (click the details button).

Tests can be aborted or re-run in Jenkins. Currently it takes about an hour to run them all. Jenkins polls github every 5 minutes, there is 5000/hour request limit on github api but I think we are safe here. If it will not be sufficient we can increase the timeout.

@pbrezina

This comment has been minimized.

Copy link
Member Author

commented Nov 26, 2018

I just realized that Jenkins will fire new testing even if the target branch changes which is nice. So when you push something to master pull request will be re-tested.

failure {
script {
if (untrusted) {
githubNotify status: 'ERROR', context: "$GH_CONTEXT", description: 'Untrusted files were modified.', targetUrl: "$GH_URL"

This comment has been minimized.

Copy link
@jhrozek

jhrozek Nov 28, 2018

Contributor

What does this mean?

This comment has been minimized.

Copy link
@pbrezina

pbrezina Nov 29, 2018

Author Member

This means that untrusted user (not an sssd repo admin) modified scripts under contrib/test-suite and Jenkins therefore refuse to run tests.

Jenkinsfile Outdated
GH_SUCCESS = "Success."
GH_PENDING = "Build is pending."
GH_FAILURE = "Build failed."
GH_URL = "https://sssd-jenkins.duckdns.org/blue/organizations/jenkins/sssd-ci/detail/$BRANCH_NAME/$BUILD_ID/pipeline"

This comment has been minimized.

Copy link
@jhrozek

jhrozek Nov 28, 2018

Contributor

Is this where the artifacts are being uploaded to?

It's not clear to me what the relationship is between this and the AWS base as later the script later rewrites this variable.

btw the certificate seems incorrect, at least my firefox chokes on it :)

This comment has been minimized.

Copy link
@pbrezina

pbrezina Nov 29, 2018

Author Member

This is url into internal jenkins instance. Normally available on internal dns name the we should not make public though. That's why certificate is incorrect.

This comment has been minimized.

Copy link
@pbrezina

pbrezina Nov 29, 2018

Author Member

You can go to internal jenkins for this PR by clicking on details next to sssd-ci and you can see public logs in aws with sssd-ci/fedora28.

This comment has been minimized.

Copy link
@jhrozek

jhrozek Dec 4, 2018

Contributor

I thought the point of having some AWS storage is that we upload the logs there? If the machine is behind the RH firewall, do we need to expose it at all? btw even after adding an exception for the certificate I get asked for credentials...and there's no way I'm putting any valuable password to a machine with an incorrect certificate

This comment has been minimized.

Copy link
@pbrezina

pbrezina Dec 5, 2018

Author Member

This is just a shortcut to the task, it is not necessary to publish it. In that case we could just remove the 'sssd-ci' status report. You can access the jenkins through internal dns name, the certificate will be valid there.

@jhrozek
Copy link
Contributor

left a comment

except for some inline comments, is there a way to re-run some failed test?

@pbrezina

This comment has been minimized.

Copy link
Member Author

commented Nov 29, 2018

Yes, you can re-run it from Jenkins: https://sssd-jenkins.duckdns.org/blue/pipelines

Sometimes, it yields 404. I do not know why, but refreshing the page works.

@pbrezina pbrezina force-pushed the pbrezina:jenkins branch from 7cdebef to ed905e4 Dec 5, 2018

@pbrezina

This comment has been minimized.

Copy link
Member Author

commented Dec 5, 2018

After a discussion with Jakub, we decided to keep sssd-ci status and point it to SSSD homepage (for now), later we will prepare a document that will explain this CI.

@pbrezina

This comment has been minimized.

Copy link
Member Author

commented Dec 5, 2018

The sssd-ci status is important for time when we will test on multiple distros as it will keep the overall status as 'pending' until all test on all distributions are finished.

@jhrozek

This comment has been minimized.

Copy link
Contributor

commented Dec 5, 2018

I'll push the patch if the builds succeed here. What we need next is some documentation..

@jhrozek

jhrozek approved these changes Dec 5, 2018

@jhrozek jhrozek self-assigned this Dec 5, 2018

@jhrozek jhrozek added the Accepted label Dec 5, 2018

@jhrozek

This comment has been minimized.

Copy link
Contributor

commented Dec 5, 2018

@jhrozek jhrozek closed this Dec 5, 2018

@jhrozek jhrozek added the Pushed label Dec 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.