Plugins are not handled by Sbtix yet

[nightkr]

Currently, plugins are still downloaded by SBT from the internet as usual. Ideally Sbtix should download these as well, and add them to the repo.

[ericbmerritt]

Do you have an ideas on how to go about fixing this? My research on sbt seems to indicate that the implementers have given very little thought to non connected development of any strip. My approach in supporting Erlang's build system rebar (which has a similar problem) was to actually patch the build system. The down side of that is that rebar in nix is different then rebar everywhere else. The upside is it is impossible for it to hit the network.

[nightkr]

@ericbmerritt

Sort of. SBT allows you to apply plugins to the "plugins project" (and so on, going on forever). So in the worst case it should be doable, with the caveats that the plugin has to be applied "one level higher" than your highest other plugin, and that it would slow down genNix significantly. It would probably also make it a bit more annoying to invoke...

A better solution would be if we could dynamically run the task in the context of those projects as well, but I'm not sure what to do about that...

Making SBT load it from our repo afterwards should be a much easier affair: AFAIK, SBT loads the repos to use from the launcher config file, which we can replace with our own that only contains the Sbtix repo.

[cessationoftime]

Using SBT's global plugins feature might be useful in solving this problem. One shouldn't have to apply the sbtix plugin to all projects in the plugin project chain then.

sbtix could provide a script that:

• creates a directory (./.sbtix_plugin) which contains the sbtix plugin
• launches sbt while referencing the directory as the location of global plugins by using sbt.global.base option

-Dsbt.global.base=./.sbtix_plugin/

• recurses into all projects and runs the genNix command

[cessationoftime]

Pull request #8 provides a method to easily load the sbtix plugin as a global plugin. It is then possible to run the genNix command against the plugins project with the command: sbtix "reload plugins" genNix.

It fails, but attempts to execute:

[info] Loading project definition from /home/haste-chris/workspace/sbtix/plugin/src/sbt-test/sbtix/simple/project
java.lang.AssertionError: assertion failed: ArrayBuffer((Dependency(com.typesafe.sbt:sbt-native-packager;sbtVersion=0.13;scalaVersion=2.10,1.1.4,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/com.typesafe.sbt/sbt-native-packager/scala_2.10/sbt_0.13/1.1.4/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-native-packager_2.10_0.13/1.1.4/sbt-native-packager-1.1.4.pom)), (Dependency(org.scala-sbt:sbt,0.13.12,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/org.scala-sbt/sbt/0.13.12/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/org/scala-sbt/sbt/0.13.12/sbt-0.13.12.pom)))
        at scala.Predef$.assert(Predef.scala:179)
        at se.nullable.sbtix.CoursierArtifactFetcher.buildNixRepo(CoursierArtifactFetcher.scala:24)
        at se.nullable.sbtix.NixPlugin$$anonfun$genNixRepoTask$1.apply(NixPlugin.scala:21)
        at se.nullable.sbtix.NixPlugin$$anonfun$genNixRepoTask$1.apply(NixPlugin.scala:12)
        at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
        at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
        at sbt.std.Transform$$anon$4.work(System.scala:63)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
        at sbt.Execute.work(Execute.scala:237)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
        at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
[error] (*:genNixRepo) java.lang.AssertionError: assertion failed: ArrayBuffer((Dependency(com.typesafe.sbt:sbt-native-packager;sbtVersion=0.13;scalaVersion=2.10,1.1.4,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/com.typesafe.sbt/sbt-native-packager/scala_2.10/sbt_0.13/1.1.4/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-native-packager_2.10_0.13/1.1.4/sbt-native-packager-1.1.4.pom)), (Dependency(org.scala-sbt:sbt,0.13.12,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/org.scala-sbt/sbt/0.13.12/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/org/scala-sbt/sbt/0.13.12/sbt-0.13.12.pom)))
scala.MatchError: Some((sbt.State@40db6136,Inc(Incomplete(node=Some(ScopedKey(Scope(Select(ProjectRef(file:/home/haste-chris/workspace/sbtix/plugin/src/sbt-test/sbtix/simple/project/,project)),Global,Global,Global),genNixRepo)), tpe=Error, msg=None, causes=List(), directCause=Some(java.lang.AssertionError: assertion failed: ArrayBuffer((Dependency(com.typesafe.sbt:sbt-native-packager;sbtVersion=0.13;scalaVersion=2.10,1.1.4,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/com.typesafe.sbt/sbt-native-packager/scala_2.10/sbt_0.13/1.1.4/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-native-packager_2.10_0.13/1.1.4/sbt-native-packager-1.1.4.pom)), (Dependency(org.scala-sbt:sbt,0.13.12,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/org.scala-sbt/sbt/0.13.12/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/org/scala-sbt/sbt/0.13.12/sbt-0.13.12.pom)))))))) (of class scala.Some)
        at se.nullable.sbtix.NixPlugin$$anonfun$genNixCommand$1$$anonfun$1.apply(NixPlugin.scala:31)
        at se.nullable.sbtix.NixPlugin$$anonfun$genNixCommand$1$$anonfun$1.apply(NixPlugin.scala:30)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
        at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
        at se.nullable.sbtix.NixPlugin$$anonfun$genNixCommand$1.apply(NixPlugin.scala:30)
        at se.nullable.sbtix.NixPlugin$$anonfun$genNixCommand$1.apply(NixPlugin.scala:24)
        at sbt.Command$$anonfun$command$1$$anonfun$apply$1.apply(Command.scala:30)
        at sbt.Command$$anonfun$command$1$$anonfun$apply$1.apply(Command.scala:30)
        at sbt.Command$.process(Command.scala:93)
        at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
        at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
        at sbt.State$$anon$1.process(State.scala:184)
        at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
        at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
        at sbt.MainLoop$.next(MainLoop.scala:96)
        at sbt.MainLoop$.run(MainLoop.scala:89)
        at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:68)
        at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:63)
        at sbt.Using.apply(Using.scala:24)
        at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
        at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
        at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
        at sbt.MainLoop$.runLogged(MainLoop.scala:22)
        at sbt.StandardMain$.runManaged(Main.scala:54)
        at sbt.xMain.run(Main.scala:29)
        at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
        at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
        at xsbt.boot.Launch$.run(Launch.scala:109)
        at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
        at xsbt.boot.Launch$.launch(Launch.scala:117)
        at xsbt.boot.Launch$.apply(Launch.scala:18)
        at xsbt.boot.Boot$.runImpl(Boot.scala:41)
        at xsbt.boot.Boot$.main(Boot.scala:17)
        at xsbt.boot.Boot.main(Boot.scala)
[error] scala.MatchError: Some((sbt.State@40db6136,Inc(Incomplete(node=Some(ScopedKey(Scope(Select(ProjectRef(file:/home/haste-chris/workspace/sbtix/plugin/src/sbt-test/sbtix/simple/project/,project)),Global,Global,Global),genNixRepo)), tpe=Error, msg=None, causes=List(), directCause=Some(java.lang.AssertionError: assertion failed: ArrayBuffer((Dependency(com.typesafe.sbt:sbt-native-packager;sbtVersion=0.13;scalaVersion=2.10,1.1.4,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/com.typesafe.sbt/sbt-native-packager/scala_2.10/sbt_0.13/1.1.4/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-native-packager_2.10_0.13/1.1.4/sbt-native-packager-1.1.4.pom)), (Dependency(org.scala-sbt:sbt,0.13.12,default(compile),Set(),Attributes(jar,),false,true),List(not found: /home/haste-chris/.ivy2/local/org.scala-sbt/sbt/0.13.12/ivys/ivy.xml, not found: https://repo1.maven.org/maven2/org/scala-sbt/sbt/0.13.12/sbt-0.13.12.pom)))))))) (of class scala.Some)

The error seems to be caused by an inability to find the plugins. Maybe more resolvers are needed?

Excerpt from the large error message posted above:

(Dependency(
 com.typesafe.sbt:sbt-native-packager;sbtVersion=0.13;scalaVersion=2.10,1.1.4,default(compile),Set(),Attributes(jar,),false,true),List(
not found: /home/haste-chris/.ivy2/local/com.typesafe.sbt/sbt-native-packager/scala_2.10/sbt_0.13/1.1.4/ivys/ivy.xml,
not found: https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-native-packager_2.10_0.13/1.1.4/sbt-native-packager-1.1.4.pom)), 

(Dependency(
 org.scala-sbt:sbt,0.13.12,default(compile),Set(),Attributes(jar,),false,true),List(
 not found: /home/haste-chris/.ivy2/local/org.scala-sbt/sbt/0.13.12/ivys/ivy.xml, 
 not found: https://repo1.maven.org/maven2/org/scala-sbt/sbt/0.13.12/sbt-0.13.12.pom))

[cessationoftime]

This coursier issue might become a problem: https://github.com/alexarchambault/coursier/issues/292

[cessationoftime]

adding resolvers to simple example's plugins.sbt seems to almost solve this problem:

resolvers += Resolver.typesafeIvyRepo("releases")
resolvers += Resolver.sbtPluginRepo("releases")

With those resolvers in place I get:

[info] Fetching file:/home/haste-chris/.coursier/cache/v1/https/repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/classpath/0.13.12/jars/classpath.jar
path is ‘/nix/store/byqri0fzcsphlcn54wi3vkh42llsjgv9-classpath.jar’
[info] Fetching https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/classpath/0.13.12/jars/classpath.pom
error: unable to download ‘https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/classpath/0.13.12/jars/classpath.pom’: HTTP response code said error (22)
java.lang.RuntimeException: Nonzero exit value: 1
        at scala.sys.package$.error(package.scala:27)
        at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:131)
        at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:101)
        at se.nullable.sbtix.CoursierArtifactFetcher.fetchChecksum(CoursierArtifactFetcher.scala:59)
        at se.nullable.sbtix.CoursierArtifactFetcher.buildArtifact(CoursierArtifactFetcher.scala:54)
        at se.nullable.sbtix.CoursierArtifactFetcher$$anonfun$buildNixRepo$2.apply(CoursierArtifactFetcher.scala:40)
        at se.nullable.sbtix.CoursierArtifactFetcher$$anonfun$buildNixRepo$2.apply(CoursierArtifactFetcher.scala:27)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

and I wonder if it could be because of how sbtix is resolving pom files. Or because it is looking at pom files at all. I think most plugins are stored in ivy repositories and not maven ones because plugins need ivy's extra attributes feature.

[cessationoftime]

Fixed issue #1 and am now able to generate a repo.nix for the plugin project, it finds the ivy.xml, pom and jar artifacts locally. Am probably going to have a problem when I try to use it though, since I need to build Ivy formatted repos.