Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make this work on ci.j.io #15

Merged
merged 6 commits into from May 23, 2017

Conversation

4 participants
@daniel-beck
Copy link
Contributor

commented May 18, 2017

Work in progress

/me sighs

daniel-beck added some commits May 18, 2017

@rtyler

rtyler approved these changes May 18, 2017

<id>maven.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/public/</url>
</repository>
<repository>
<id>m.g.o-public</id>

This comment has been minimized.

Copy link
@jglick

jglick May 19, 2017

Delete this.

<mirrors>
<mirror>
<id>repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/public/</url>

This comment has been minimized.

Copy link
@jglick

jglick May 19, 2017

Why is this here? The pom.xml already specifies this repository.

This comment has been minimized.

Copy link
@daniel-beck

daniel-beck May 19, 2017

Author Contributor

This is used in a separate mvn invocation that downloads dependencies of core and all plugins before scanning for extension implementations (perhaps to make it compilable, unclear to me).

My current best guess is that without this, the standalone mvn invocation trying to download jenkins-core dependencies doesn't find the parent pom that's not in the parent dir.

This comment has been minimized.

Copy link
@daniel-beck

daniel-beck May 19, 2017

Author Contributor

Additionally, plugins last released before 2012 or so will reference now obsolete Maven repositories.

@daniel-beck daniel-beck changed the title Add Jenkinsfile Make this work on ci.j.io May 19, 2017

@@ -304,7 +304,7 @@ private MavenRepositoryImpl createRepository() throws Exception {
* Walks over the plugins, record {@link #modules} and call {@link #discover(Module)}.
*/
private void processPlugins(MavenRepository r) throws Exception {
ExecutorService svc = Executors.newFixedThreadPool(4);
ExecutorService svc = Executors.newFixedThreadPool(1);

This comment has been minimized.

Copy link
@kwhetstone

kwhetstone May 19, 2017

Contributor

Does this drastically increase the runtime?

This comment has been minimized.

Copy link
@daniel-beck

daniel-beck May 19, 2017

Author Contributor

Yes and no.

Yes, because things go slower (while they're still going).

No, because it kept locking up on me with stack traces like https://gist.github.com/daniel-beck/401a85db7d58a55b0c78e02d33f971fd

This comment has been minimized.

Copy link
@daniel-beck

daniel-beck May 19, 2017

Author Contributor

FWIW I can try reverting this one once the PR build passes to see whether that can finish with 4 threads. If so, I won't care about local execution.

@@ -143,6 +143,8 @@ private static void downloadDependencies(File pomDir, File destDir) throws IOExc
process = System.getenv("M2_HOME") + "/bin/mvn";
}
ProcessBuilder builder = new ProcessBuilder(process,
"--settings", new File("maven-settings.xml").getAbsolutePath(),

This comment has been minimized.

Copy link
@jglick

jglick May 20, 2017

BTW it is poor style to assume that the CWD of the process has a particular file in it. Rather out maven-settings.xml into src/main/resources/ and use FileUtils to copy the resource to a local temp file.

@@ -143,6 +143,8 @@ private static void downloadDependencies(File pomDir, File destDir) throws IOExc
process = System.getenv("M2_HOME") + "/bin/mvn";
}
ProcessBuilder builder = new ProcessBuilder(process,
"--settings", new File("maven-settings.xml").getAbsolutePath(),
"--update-snapshots",
"dependency:copy-dependencies",

This comment has been minimized.

Copy link
@jglick

jglick May 20, 2017

Maybe try running the validate phase first:

mvn -s … -U -DincludeScope=compile -DoutputDirectory=… validate dependency:copy-dependencies
@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 20, 2017

The previous PR build got to mber, this one is stuck at notification. Not a huge difference.

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 21, 2017

Re-running the PR so I can check whether it's again disk usage.

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 21, 2017

Node monitoring showed that disk space was at just over 4GB when I aborted the build just now (after ~1.5hrs), so it's safe to assume it'd just run out.

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 22, 2017

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2017

@rtyler did some hackery on the highmem agent and currently it reports 362.53 GB free disk space. This should be enough for this build 😃

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2017

It's at 'm', with 348.01 GB free disk space.

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2017

After 3hr 18min it's at 'toolenv', with 340.61 GB free disk space.

@daniel-beck

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2017

3 hours 38 minutes. 🎉

@rtyler

rtyler approved these changes May 23, 2017

@daniel-beck daniel-beck merged commit c0d421e into jenkins-infra:master May 23, 2017

1 check passed

continuous-integration/jenkins/pr-merge This commit looks good
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.