real build is slow, testing fixes locally is slow too #152
Comments
|
read it and weep, here's a log of the slow resolution process. it takes 1-2 seconds per line. note that you have to run dbuild with and that's just resolution. it actually downloads the jars too, every time: |
SethTisue
changed the title
SethTisue
changed the title
|
@retronym comments: "Maybe that plugin should be uberjarred or some such... it could also just add a list of JARs to the build's |
|
one of the issues here is that dbuild lets you tweak the build definition, but it doesn’t seem to offer a way to tweak the meta-project — the build definition’s build definition. (or maybe it does and I just can’t uncover any reference to it, I guess I’d have to ask @cunei to be sure.) and it’s the meta-project that is the problem here |
|
note: I tried upgrading dbuild from 0.9.1 to 0.9.3. it didn't help. |
|
whenever one project needs a resolver, we add it to a global list. presumably this slows down everything. can we add resolvers for a particular project only? |
This was referenced Oct 28, 2015
|
@SethTisue, concerning tweaking the meta-build, check: http://typesafehub.github.io/dbuild/0.9.3/plugins.html#build-levels as well as the following section; it may help. |
SethTisue
added a commit
to SethTisue/scala-jenkins-infra
that referenced
this issue
Jan 29, 2016
when the Scala version bumps, it's now become routine for the build to hit the old 600 minute limit and time out. we have continued to gradually add projects (and additional resolvers), and existing projects can be expected to gradually add dependencies and grow their test suites and so forth, so I'm not too alarmed about it. admittedly, the build is slower than we would like! the ticket on that is scala/community-build#152
SethTisue
added a commit
to SethTisue/scala-jenkins-infra
that referenced
this issue
Jan 29, 2016
when the Scala version bumps, it's now become routine for the build to hit the old 600 minute limit and time out. we have continued to gradually add projects (and additional resolvers), and existing projects can be expected to gradually add dependencies and grow their test suites and so forth, so I'm not too alarmed about it. in practice, the usual way of coping with this is to fix the Scala SHA whenever manually triggering test runs, so that the whole thing doesn't have to be redone from scratch. admittedly, the build is still slower than we would like! the ticket on that is scala/community-build#152
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
slow in actual Jenkins runs:
we keep having to up the limit on how many hours the community builds are allowed to take. lately they've been in the ~8 hour range
slow during testing:
e.g. I'm trying to fix an issue in the scala-xml build. even if I delete all of common.conf and all of community.dbuild except for two projects only (scala and scala-xml), and even once the scala project has been built and cached, it still takes a long time (like 10+ minutes) for the scala-xml build to get going, which means my turnaround time for testing possible fixes precludes any kind of thinking/coding "flow"
most of the time is spent first resolving, then downloading the dependencies, not of scala-xml itself, but of its build definition. the list of jars needed is quite long, and most of them seem to be dependencies of dbuild itself — since one of the mechanisms by which dbuild operates is by injecting its own sbt plugin.
because dbuild sets up a special environment in which ivy.home and sbt.ivy.home are different, the needed JARs don't get cached in
~/.ivy2like they would be in a normal buildand whatever magic dbuild does to make resolution fast during extraction (I confirmed by testing that extraction the 1st time is slow but subsequent extractions are very fast) isn't operable here, not sure why
The text was updated successfully, but these errors were encountered: