The classpath for BSP when using the mill dev.launcher is too long

[srdo]

Hi,

When I set up BSP using mill.contrib.BSP/install, from a mill launcher generated with mill dev.launcher, the generated BSP file (.bsp/mill.json) contains an extremely long classpath string, which prevents IntelliJ from loading the project.

/home/stig/mill/dev/resources:/home/stig/mill/out/dev/compile/dest/classes:/home/stig/mill/scalalib/resources:/home/stig/mill/out/scalalib/compile/dest/classes:/home/stig/mill/main/resources:/home/stig/mill/out/main/compile/dest/classes:/home/stig/mill/main/core/resources:/home/stig/mill/out/main/core/compile/dest/classes:/home/stig/mill/main/moduledefs/resources:/home/stig/mill/out/main/moduledefs/compile/dest/classes:/home/stig/mill/main/api/resources:/home/stig/mill/out/main/api/compile/dest/classes:/home/stig/mill/main/client/resources:/home/stig/mill/out/main/client/compile/dest/classes:/home/stig/mill/scalalib/api/resources:/home/stig/mill/out/scalalib/api/compile/dest/classes:/home/stig/mill/scalajslib/resources:/home/stig/mill/out/scalajslib/compile/dest/classes:/home/stig/mill/scalajslib/api/resources:/home/stig/mill/out/scalajslib/api/compile/dest/classes:/home/stig/mill/scalanativelib/resources:/home/stig/mill/out/scalanativelib/compile/dest/classes:/home/stig/mill/scalanativelib/api/resources:/home/stig/mill/out/scalanativelib/api/compile/dest/classes:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.2/scala-library-2.13.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/test-interface/1.0/test-interface-1.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-dynamic_2.13/2.2.1/scalafmt-dynamic_2.13-2.2.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite_2.13.2/2.2.0/ammonite_2.13.2-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/trees_2.13/4.3.7/trees_2.13-4.3.7.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier_2.13/2.0.0-RC6-25/coursier_2.13-2.0.0-RC6-25.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.0.0/jna-5.0.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna-platform/5.0.0/jna-platform-5.0.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/eed3si9n/jarjarabrams/jarjar-abrams-core_2.13/0.3.0/jarjar-abrams-core_2.13-0.3.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.2/scala-compiler-2.13.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/sourcecode_2.13/0.2.1/sourcecode_2.13-0.2.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/os-lib_2.13/0.7.1/os-lib_2.13-0.7.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle_2.13/1.2.0/upickle_2.13-1.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/ipcsocket/ipcsocket/1.0.0/ipcsocket-1.0.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-interfaces/2.2.1/scalafmt-interfaces-2.2.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/interface/0.0.21/interface-0.0.21.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/config/1.3.3/config-1.3.3.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-terminal_2.13/2.2.0/ammonite-terminal_2.13-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-ops_2.13/2.2.0/ammonite-ops_2.13-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-util_2.13/2.2.0/ammonite-util_2.13-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-runtime_2.13.2/2.2.0/ammonite-runtime_2.13.2-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-interp-api_2.13.2/2.2.0/ammonite-interp-api_2.13.2-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-repl-api_2.13.2/2.2.0/ammonite-repl-api_2.13.2-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-interp_2.13.2/2.2.0/ammonite-interp_2.13.2-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-repl_2.13.2/2.2.0/ammonite-repl_2.13.2-2.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/common_2.13/4.3.7/common_2.13-4.3.7.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/thesamet/scalapb/scalapb-runtime_2.13/0.9.6/scalapb-runtime_2.13-0.9.6.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/fastparse_2.13/1.0.1/fastparse_2.13-1.0.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-core_2.13/2.0.0-RC6-25/coursier-core_2.13-2.0.0-RC6-25.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cache_2.13/2.0.0-RC6-25/coursier-cache_2.13-2.0.0-RC6-25.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/alexarchambault/argonaut-shapeless_6.2_2.13/1.2.0-M12/argonaut-shapeless_6.2_2.13-1.2.0-M12.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.3.1/jna-5.3.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/pantsbuild/jarjar/1.7.2/jarjar-1.7.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.2/scala-reflect-2.13.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline/3.14.1/jline-3.14.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.13/0.6.2/geny_2.13-0.6.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ujson_2.13/1.2.0/ujson_2.13-1.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upack_2.13/1.2.0/upack_2.13-1.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-implicits_2.13/1.2.0/upickle-implicits_2.13-1.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fansi_2.13/0.2.9/fansi_2.13-0.2.9.jar:/home/stig/.cache/coursier/v1/
https/repo1.maven.org/maven2/org/scala-lang/modules/scala-collection-compat_2.13/2.1.4/scala-collection-compat_2.13-2.1.4.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/pprint_2.13/0.5.9/pprint_2.13-0.5.9.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/requests_2.13/0.6.5/requests_2.13-0.6.5.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/scopt/scopt_2.13/3.7.1/scopt_2.13-3.7.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/ch/epfl/scala/bsp4j/2.0.0-M6/bsp4j-2.0.0-M6.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/scalaparse_2.13/2.3.0/scalaparse_2.13-2.3.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/javassist/javassist/3.21.0-GA/javassist-3.21.0-GA.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.13/1.3.0/scala-xml_2.13-1.3.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal/3.14.1/jline-terminal-3.14.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal-jna/3.14.1/jline-terminal-jna-3.14.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-reader/3.14.1/jline-reader-3.14.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/javaparser/javaparser-core/3.2.5/javaparser-core-3.2.5.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/thesamet/scalapb/lenses_2.13/0.9.6/lenses_2.13-0.9.6.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.8.0/protobuf-java-3.8.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fastparse_2.13/2.3.0/fastparse_2.13-2.3.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/fastparse-utils_2.13/1.0.1/fastparse-utils_2.13-1.0.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-util_2.13/2.0.0-RC6-25/coursier-util_2.13-2.0.0-RC6-25.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/windows-ansi/windows-ansi/0.0.3/windows-ansi-0.0.3.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/io/argonaut/argonaut_2.13/6.2.4/argonaut_2.13-6.2.4.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/chuusai/shapeless_2.13/2.3.3/shapeless_2.13-2.3.3.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/ant/ant/1.9.9/ant-1.9.9.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm/7.0/asm-7.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-commons/7.0/asm-commons-7.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-plugin-api/3.3.9/maven-plugin-api-3.3.9.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-core_2.13/1.2.0/upickle-core_2.13-1.2.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j.generator/0.8.1/org.eclipse.lsp4j.generator-0.8.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/lsp4j/org.eclipse.lsp4j.jsonrpc/0.8.1/org.eclipse.lsp4j.jsonrpc-0.8.1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.18/jansi-1.18.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.9/ant-launcher-1.9.9.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-tree/7.0/asm-tree-7.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-analysis/7.0/asm-analysis-7.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-model/3.3.9/maven-model-3.3.9.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/maven/maven-artifact/3.3.9/maven-artifact-3.3.9.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.3.2/org.eclipse.sisu.plexus-0.3.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/xtend/org.eclipse.xtend.lib/2.18.0/org.eclipse.xtend.lib-2.18.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/code/gson/gson/2.8.2/gson-2.8.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.4/commons-lang3-3.4.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/org.eclipse.sisu.inject-0.3.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/xtext/org.eclipse.xtext.xbase.lib/2.18.0/org.eclipse.xtext.xbase.lib-2.18.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/org/eclipse/xtend/org.eclipse.xtend.lib.macro/2.18.0/org.eclipse.xtend.lib.macro-2.18.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/javax/annotation/jsr250-api/1.0/jsr250-api-1.0.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar:/home/stig/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/guava/guava/21.0/guava-21.0.jar -i mill.contrib.BSP/start

This command is too long for IntelliJ to run, which causes IntelliJ to fail to open the project properly

env: ‘/home/stig/mill/dev/resources:/home/stig/mill/out/dev/compile/dest/classes:/home/stig/mill/scalalib/resources:(and so on): File name too long

When using a regular Mill release, the corresponding string is much smaller

/home/stig/.cache/mill/download/0.8.0 -i mill.contrib.BSP/start

Would an option be for the dev.launcher command to put the dependency jars in a directory somewhere, so they can be referenced with a wildcard in the classpath?

[srdo]

The long classpath is coming from here https://github.com/lihaoyi/mill/blob/8cd135ad640b2732462739f6b104df4d3ac7edc1/contrib/bsp/src/mill/contrib/BSP.scala#L74-L76

The shell script generated by dev.launcher had a similar problem, which was solved in #791 to address issues on Windows. The reason that fix doesn't affect this issue is that even if Mill uses a classpath jar in the launcher script, the BSP install command will read the expanded classpath through java.class.path, and write the expanded classpath to the .bsp/mill.json file.

I think the BSP install command should do something similar to what Mill's build.sc does, by writing a classpath jar. It would be nice if the BSP script could reuse the classpath jar generated by dev.launcher, but I don't see a good way to pass the path to that jar to the BSP command.

Maybe the best solution is to have BSP/install generate a new classpath jar with the java.class.path in the manifest, write that jar to the .bsp directory, and write the .bsp/mill.json to point to that jar?

Edit: I don't think the above would help. It looks to me like the BSP command is actually expecting java.class.path to contain the path to a Mill executable script (e.g. 0.8.0), rather than a list of jars. For released versions, the Mill executable seems to be a jar with the Mill boot script prepended. The boot script starts Mill, setting the script file itself as the classpath. dev.launcher scripts start Mill with the classpath set to jars in Mill's out directory, which doesn't fit what the BSP command expects.

I think the question is if there's a better way for Mill to tell the BSP command how to start Mill. An option could be for Mill's boot scripts to pass their own location to the Mill process as a system parameter. For now the workaround is probably to avoid using BSP when starting off a dev.launcher script.

[srdo]

Raised a PR with a first attempt at a solution.

Edit: Not sure how best to solve this. If the Mill script path is generally useful, maybe it should be a parameter to the Evaluator. The script could pass it into MillMain, and put it in the Evaluator, where BSP could read it. If the script path isn't useful to anything other than BSP, maybe it is better to just close this issue as wontfix, since it's just a minor inconvenience during development of Mill. The workaround is to use ci/publish-local.sh instead of dev.launcher for local testing.

[lefou]

Mill's BSP server isn't designed to be run via mill dev.launcher.