Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let the CI pass the appropriate forkCount #7110

Merged
merged 4 commits into from
Sep 17, 2022

Conversation

Vlatombe
Copy link
Member

@Vlatombe Vlatombe commented Sep 15, 2022

cf. jenkinsci/plugin-pom#269 (comment)

See JENKINS-XXXXX.

Proposed changelog entries

  • Internal change
  • ...

Proposed upgrade guidelines

N/A

Submitter checklist

  • (If applicable) Jira issue is well described
  • Changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developer, depending on the change) and are in the imperative mood. Examples
    • Fill-in the Proposed changelog entries section only if there are breaking changes or other changes which may require extra steps from users during the upgrade
  • Appropriate autotests or explanation to why this change has no tests
  • New public classes, fields, and methods are annotated with @Restricted or have @since TODO Javadoc, as appropriate.
  • New deprecations are annotated with @Deprecated(since = "TODO") or @Deprecated(forRemoval = true, since = "TODO") if applicable.
  • New or substantially changed JavaScript is not defined inline and does not call eval to ease future introduction of Content-Security-Policy directives (see documentation on jenkins.io).
  • For dependency updates: links to external changelogs and, if possible, full diffs

Desired reviewers

@mention

Maintainer checklist

Before the changes are marked as ready-for-merge:

  • There are at least 2 approvals for the pull request and no outstanding requests for change
  • Conversations in the pull request are over OR it is explicit that a reviewer does not block the change
  • Changelog entries in the PR title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood
  • Proper changelog labels are set so that the changelog can be generated automatically
  • If the change needs additional upgrade steps from users, upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the PR title. (example)
  • If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

@MarkEWaite MarkEWaite added the skip-changelog Should not be shown in the changelog label Sep 15, 2022
Copy link
Member

@jglick jglick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess

jenkins/Jenkinsfile

Lines 46 to 60 in e80100a

def mavenOptions = [
'-Pdebug',
'-Penable-jacoco',
'--update-snapshots',
"-Dmaven.repo.local=$m2repo",
'-Dmaven.test.failure.ignore',
'-Dspotbugs.failOnError=false',
'-Dcheckstyle.failOnViolation=false',
'-Dset.changelist',
'help:evaluate',
'-Dexpression=changelist',
"-Doutput=$changelistF",
'clean',
'install',
]
should pass -DforkCount=2 to match?

Copy link
Member

@basil basil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since in its current form this PR is not setting the forkCount for the test/ module to any value, and since the default forkCount is 1, I think this will have the practical effect of doubling the test time for the test/ module, which will likely cause this PR build to time out and fail.

While I do not understand the motivation for this change, I am going to assume (rather than applying the needs-justification label) that this is because it is undesirable to make an assumption about the processing power of the execution environment in the build. Rather it seems desirable to allow the execution environment to set this value, e.g. in its Jenkinsfile (for CI builds) or in the options passed to mvn (for interactive builds).

If that is the motivation, then why not take it to its logical conclusion by also removing forkCount from the core/ module as well? Then, we would need to define forkCount in the Jenkinsfile to something reasonable. Since it is currently 0.5C for the core/ module and 2 for the test/, I think using a value of 2 in the Jenkinsfile would be reasonable, and would also make test runs more deterministic (a plus!).

@Vlatombe
Copy link
Member Author

If that is the motivation, then why not take it to its logical conclusion by also removing forkCount from the core/ module as well? Then, we would need to define forkCount in the Jenkinsfile to something reasonable. Since it is currently 0.5C for the core/ module and 2 for the test/, I think using a value of 2 in the Jenkinsfile would be reasonable, and would also make test runs more deterministic (a plus!).

I missed the fact that it was also set in core. I agree that it should rather be set in the Jenkinsfile. I'll update accordingly.

Jenkinsfile Outdated Show resolved Hide resolved
Jenkinsfile Outdated
@@ -49,6 +49,7 @@ for (i = 0; i < buildTypes.size(); i++) {
'--update-snapshots',
"-Dmaven.repo.local=$m2repo",
'-Dmaven.test.failure.ignore',
'-DforkCount=0.5C',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know how many cores our container test environments have, so I am unable to assess whether this is reasonable or not. I do know for sure that setting this to 2 would retain existing behavior for the test/ module (no regressions) and would likely not impact the core/ module, which only takes a few minutes to run anyway, (no regressions). So while I would be a +1 for a value of 2, I am undecided on a value of 0.5C pending an explanation of what that would translate to in practice on our container agents.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice they seem equivalent (2 tests in parallel). https://github.com/jenkins-infra/documentation/blob/main/ci.adoc#jenkins-on-jenkins doesn't say anything about specs of default containers (unlike VMs), would probably be valuable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it could easily change in the future and cause unexpected breakage. I think an explicit value of 2 would be less fragile.

Copy link
Member

@lemeurherve lemeurherve Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/jenkins-infra/documentation/blob/main/ci.adoc#jenkins-on-jenkins doesn't say anything about specs of default containers (unlike VMs), would probably be valuable.

This page needs an update indeed, issue opened.

Here and below you can see the number of CPU and mem allocated to the kubernetes agents.

Co-authored-by: Jesse Glick <jglick@cloudbees.com>
Copy link
Member

@basil basil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is now ready for merge. We will merge it after approximately 24 hours if there is no negative feedback. Please see the merge process documentation for more information about the merge process. Thanks!

@basil basil added the ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback label Sep 15, 2022
@basil basil merged commit e33b546 into jenkinsci:master Sep 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback skip-changelog Should not be shown in the changelog
Projects
None yet
6 participants