Spurious recompilation in multiproject builds with external subprojects. #35

Closed
nuttycom opened this Issue Jun 5, 2011 · 5 comments

Comments

Projects
None yet
2 participants
@nuttycom

nuttycom commented Jun 5, 2011

I'm finding some strange behavior with the new external multiproject support. I have two projects, A and B which both dependOn external project C using RootProject("../c").

If I go to project A and compile it, C is also compiled, and all is well. However, if I then go to project B and compile it, this also causes C to be entirely recompiled. This of course is awful, because C takes a couple of minutes to compile and this happens whenever I switch between A and B.

Here is an example of build output related to the attached projects:

knuttycombe@floorshow sbt (master)]$ ls
a b c
[knuttycombe@floorshow sbt (master)]$ cd c/
[knuttycombe@floorshow c (master)]$ sbt compile
Getting net.java.dev.jna jna 3.2.3 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/80ms)
Getting Scala 2.8.1 (for sbt)...
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
4 artifacts copied, 0 already retrieved (15296kB/873ms)
Getting org.scala-tools.sbt sbt_2.8.1 0.10.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
34 artifacts copied, 0 already retrieved (6012kB/794ms)
[info] Set current project to default (in build file:/Users/knuttycombe/tmp/sbt/c/)
[info] Updating...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[success] Total time: 3 s, completed Jun 4, 2011 9:46:53 PM
[knuttycombe@floorshow c (master)]$ cd ../b
[knuttycombe@floorshow b (master)]$ sbt compile
Getting net.java.dev.jna jna 3.2.3 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/19ms)
Getting Scala 2.8.1 (for sbt)...
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
4 artifacts copied, 0 already retrieved (15296kB/248ms)
Getting org.scala-tools.sbt sbt_2.8.1 0.10.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
34 artifacts copied, 0 already retrieved (6012kB/63ms)
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/b/project/target/scala_2.8.1/classes...
[info] Set current project to b (in build file:/Users/knuttycombe/tmp/sbt/b/)
[info] Updating...
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/b/target/scala-2.8.1.final/classes...
[success] Total time: 3 s, completed Jun 4, 2011 9:47:16 PM
[knuttycombe@floorshow b (master)]$ cd ../a
[knuttycombe@floorshow a (master)]$ sbt compile
Getting net.java.dev.jna jna 3.2.3 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/17ms)
Getting Scala 2.8.1 (for sbt)...
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
4 artifacts copied, 0 already retrieved (15296kB/236ms)
Getting org.scala-tools.sbt sbt_2.8.1 0.10.0 ...
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
34 artifacts copied, 0 already retrieved (6012kB/65ms)
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/a/project/target/scala_2.8.1/classes...
[info] Set current project to a (in build file:/Users/knuttycombe/tmp/sbt/a/)
[info] Updating...
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/a/target/scala-2.8.1.final/classes...
[success] Total time: 3 s, completed Jun 4, 2011 9:47:41 PM
[knuttycombe@floorshow a (master)]$ cd ../b/
[knuttycombe@floorshow b (master)]$ sbt compile
[info] Set current project to b (in build file:/Users/knuttycombe/tmp/sbt/b/)
[info] Compiling 1 Scala source to /Users/knuttycombe/tmp/sbt/c/target/scala-2.8.1.final/classes...
[success] Total time: 6 s, completed Jun 4, 2011 9:47:58 PM
[knuttycombe@floorshow b (master)]$

Something interesting to note: although c is recompiled when I switch back to b, the classes in b that depend on c are not!

@nuttycom

This comment has been minimized.

Show comment
Hide comment
@nuttycom

nuttycom Jun 5, 2011

Haven't been able to figure out how to attach a file for this ticket, so here's the source organization:

./a/project/Build.scala
import sbt._
import Keys._

object ABuild extends Build {
lazy val a = Project("a", file(".")) dependsOn (c)
lazy val c = RootProject(file("../c"))
}

./a/src/main/scala/a/A.scala
package a
import c.C

class A(c: C)

./b/project/Build.scala
import sbt._
import Keys._

object BBuild extends Build {
lazy val b = Project("b", file(".")) dependsOn (c)
lazy val c = RootProject(file("../c"))
}

./b/src/main/scala/b/B.scala
package b
import c.C

case class B(c: C)

./c/build.sbt
name := "c"

./c/src/main/scala/c/C.scala
package c

case class C(value: String)

nuttycom commented Jun 5, 2011

Haven't been able to figure out how to attach a file for this ticket, so here's the source organization:

./a/project/Build.scala
import sbt._
import Keys._

object ABuild extends Build {
lazy val a = Project("a", file(".")) dependsOn (c)
lazy val c = RootProject(file("../c"))
}

./a/src/main/scala/a/A.scala
package a
import c.C

class A(c: C)

./b/project/Build.scala
import sbt._
import Keys._

object BBuild extends Build {
lazy val b = Project("b", file(".")) dependsOn (c)
lazy val c = RootProject(file("../c"))
}

./b/src/main/scala/b/B.scala
package b
import c.C

case class B(c: C)

./c/build.sbt
name := "c"

./c/src/main/scala/c/C.scala
package c

case class C(value: String)

@nuttycom nuttycom closed this Jun 5, 2011

@nuttycom nuttycom reopened this Jun 5, 2011

@harrah

This comment has been minimized.

Show comment
Hide comment
@harrah

harrah Jun 5, 2011

Member

Apparently, you are supposed to paste in a gist and link to it, which is somewhat unfortunate.

The cause of the problem is that by default, sbt uses the project/boot directory in the project you start sbt in and in particular, the Scala files there. So, when you move between projects, sbt sees a different version of scala-library.jar for c (the one in a/project/boot and then in b/project/boot) and thinks the source needs recompiling.

There are two solutions I can think of. First, you can have a global boot directory that is shared by all projects on your machine. You can do this by setting the sbt.boot.directory system property in your launcher script. It can also be passed directly to the sbt script to just try it out at first. For example:

sbt -Dsbt.boot.directory=/home/user/.sbt/boot/

The other option is to have another project that refers to the three projects and then always run sbt on that project. This other project doesn't have to contain the subdirectories.

Member

harrah commented Jun 5, 2011

Apparently, you are supposed to paste in a gist and link to it, which is somewhat unfortunate.

The cause of the problem is that by default, sbt uses the project/boot directory in the project you start sbt in and in particular, the Scala files there. So, when you move between projects, sbt sees a different version of scala-library.jar for c (the one in a/project/boot and then in b/project/boot) and thinks the source needs recompiling.

There are two solutions I can think of. First, you can have a global boot directory that is shared by all projects on your machine. You can do this by setting the sbt.boot.directory system property in your launcher script. It can also be passed directly to the sbt script to just try it out at first. For example:

sbt -Dsbt.boot.directory=/home/user/.sbt/boot/

The other option is to have another project that refers to the three projects and then always run sbt on that project. This other project doesn't have to contain the subdirectories.

@nuttycom

This comment has been minimized.

Show comment
Hide comment
@nuttycom

nuttycom Jun 5, 2011

Having a common SBT boot directory sounds like it might be generally good
practice in any case, because it would minimize startup times for new
projects as well. I'll give that a try, thanks!
On Jun 5, 2011 12:47 AM, "harrah" <
reply@reply.github.com>
wrote:

Apparently, you are supposed to paste in a gist and link to it, which is
somewhat unfortunate.

The cause of the problem is that by default, sbt uses the project/boot
directory in the project you start sbt in and in particular, the Scala files
there. So, when you move between projects, sbt sees a different version of
scala-library.jar for c (the one in a/project/boot and then in
b/project/boot) and thinks the source needs recompiling.

There are two solutions I can think of. First, you can have a global boot
directory that is shared by all projects on your machine. You can do this by
setting the sbt.boot.directory system property in your launcher script. It
can also be passed directly to the sbt script to just try it out at first.
For example:

sbt -Dsbt.boot.directory=/home/user/.sbt/boot/

The other option is to have another project that refers to the three
projects and then always run sbt on that project. This other project doesn't
have to contain the subdirectories.

Reply to this email directly or view it on GitHub:
harrah#35 (comment)

nuttycom commented Jun 5, 2011

Having a common SBT boot directory sounds like it might be generally good
practice in any case, because it would minimize startup times for new
projects as well. I'll give that a try, thanks!
On Jun 5, 2011 12:47 AM, "harrah" <
reply@reply.github.com>
wrote:

Apparently, you are supposed to paste in a gist and link to it, which is
somewhat unfortunate.

The cause of the problem is that by default, sbt uses the project/boot
directory in the project you start sbt in and in particular, the Scala files
there. So, when you move between projects, sbt sees a different version of
scala-library.jar for c (the one in a/project/boot and then in
b/project/boot) and thinks the source needs recompiling.

There are two solutions I can think of. First, you can have a global boot
directory that is shared by all projects on your machine. You can do this by
setting the sbt.boot.directory system property in your launcher script. It
can also be passed directly to the sbt script to just try it out at first.
For example:

sbt -Dsbt.boot.directory=/home/user/.sbt/boot/

The other option is to have another project that refers to the three
projects and then always run sbt on that project. This other project doesn't
have to contain the subdirectories.

Reply to this email directly or view it on GitHub:
harrah#35 (comment)

@nuttycom

This comment has been minimized.

Show comment
Hide comment
@nuttycom

nuttycom Jun 5, 2011

The global sbt boot directory looks like it does the trick. I still have a bunch more questions, but I'll bring them up on the list. Thanks!

nuttycom commented Jun 5, 2011

The global sbt boot directory looks like it does the trick. I still have a bunch more questions, but I'll bring them up on the list. Thanks!

@nuttycom nuttycom closed this Jun 5, 2011

@harrah

This comment has been minimized.

Show comment
Hide comment
@harrah

harrah Jun 6, 2011

Member

To close this out, I added comments about this on Setup and Full Configuration.

Member

harrah commented Jun 6, 2011

To close this out, I added comments about this on Setup and Full Configuration.

eed3si9n pushed a commit to eed3si9n/sbt that referenced this issue Apr 22, 2017

Merge pull request #35 from jroper/1.0-support
Added support for cross building against sbt 1.0

dwijnand added a commit to dwijnand/sbt that referenced this issue Jun 26, 2017

dwijnand added a commit to dwijnand/sbt that referenced this issue Jun 30, 2017

dwijnand added a commit to dwijnand/sbt that referenced this issue Jul 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment