Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Windows launcher #791

Merged
merged 8 commits into from
Apr 13, 2020
Merged

Conversation

sake92
Copy link
Contributor

@sake92 sake92 commented Mar 15, 2020

There are 2 problems this PR tries to fix:

  1. production version of Mill, built by assembly task
  2. development version of Mill, built by dev.launcher task

These problems manifest with java 1.8.
In Hotspots 9+ there exists an option -XX:VMOptionsFile...

Production version (assembly)

Problem

  • Download latest Mill (0.6.1-25-360107-assembly) and rename it to mill.bat
  • unzip "example-1.zip" project from docs
  • run it with downloaded Mill as mill foo.compile
    First time you run it you'll get this message:

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized VM option 'VMOptionsFile=C:\projects\TODO\mill\winfix\example-1\out\mill-worker-+9h7erhXUQxlj6SW7zBIF3jvnFg=-1\vmoptions'

If you kill it and try again, it just hangs...
I couldnt attach VisualVM for some reason to debug further.
I guess it cant start a server and pipe between client-server breaks!?

Fix

Remove -XX:VMOptionsFile stuff.
This is enough to get working Mill for Windows users! (even on java 1.8)

Testing

Build Mill with publish-local.sh commands.
Then use built Mill version on projects. See that it works. 馃槃

Development version (dev.launcher)

Here is where it gets interesting.

Problem

  • Clone Mill repository and build its dev version with mill -i dev.launcher
  • try to run some command with it, you'll get:

dev.launcher java.lang.Error: dev.launcher in Windows is only supported using Java 9 or above
ammonite.$file.build$dev$.windowsVmOptions(build.sc:628)
(this was using the "mill.vmoptions" file passed via -XX:VMOptionsFile)

Why? Because -XX:VMOptionsFile doesnt exist in java 1.8...
Why is it needed in first place? Because Windows has command-line limitation of only ~8000 characters!
Most lengthy operations are MILL_STUFF and classpath, of course (lot of JARs from Coursier local repo...)

Fix

  1. introduce MILL_OPTIONS_PATH=$path/mill.properties to bypass long -DMILL_.. props
  2. introduce a Pathing JAR (see here for more details) to bypass long -cp parameter

Testing

Build dev.launcher and see it works in client-server mode.
Only thing that isnt working on Windows is -i mode of dev.launcher:

scala.reflect.internal.MissingRequirementError: object scala in compiler mirror not found...


Minor NOTE: Once you do -i it stays during that shell session (SETLOCAL wasn't used), so if you run mill after that it will still use interactive mode... This PR fixes that also.

@sake92 sake92 marked this pull request as ready for review March 15, 2020 16:15
PathRef(millPath)
}

def prependShellScript = T{
val (millArgs, otherArgs) = forkArgs().partition(arg => arg.startsWith("-DMILL") && !arg.startsWith("-DMILL_VERSION"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We pass the MILL_SOMETHING properties via file, so that Windows doesn't complain!
Everything except MILL_VERSION because it's used in lots of places.

classpath,
classpath
Agg(pathingJar().path.toString) // TODO not working yet on Windows! see #791
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows, the classpath is only the "pathingJar", it contains all the dependencies.
It works in batch mode, interactive needs bit more debugging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lihaoyi this only relates to the dev.assembly version..

String millOptionsPath = System.getProperty("MILL_OPTIONS_PATH");
if(millOptionsPath != null) {
// read MILL_CLASSPATH from file MILL_OPTIONS_PATH
Properties millProps = new Properties();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows we read the classpath from the MILL_OPTIONS_PATH file instead of env var..

@lihaoyi
Copy link
Member

lihaoyi commented Apr 12, 2020

@sake92 Is this ready to review? If so I can take a look

@sake92
Copy link
Contributor Author

sake92 commented Apr 12, 2020

@lihaoyi yes please!

@lihaoyi
Copy link
Member

lihaoyi commented Apr 13, 2020

@sake92 this looks good to me I think.

Before I merge, could you flesh out the PR description a bit to summarize (1) exactly which user-facing problems you are fixing, (2) what the root cause of those problems is, (3) how your PR solves those root causes and (4) how you tested it to verify your PR solves the things you think it solves? I'm still a bit fuzzy on the different problems with dev, assembly, dev.assembly, dev.launcher, etc. and which ones I should expect to behave differently due to this PR. This will also help future maintenance.

Once that's done I'm happy to merge!

@sake92
Copy link
Contributor Author

sake92 commented Apr 13, 2020

@lihaoyi I updated the description, hope it's clearer now.
In summary:

  • prod version will see almost no changes, except it'll work on Windows now also
  • dev.launcher will use mill.properties instead of -DMILL_PROP properties.
    And on Windows it'll use "pathingJar" instead of "normal classpath"

@lihaoyi
Copy link
Member

lihaoyi commented Apr 13, 2020

This looks good, thanks for the detailed PR description!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants