Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When .mill-jvm-opts doesn't end with a newline, the last parameter is ignored #2140

Closed
phfroidmont opened this issue Nov 23, 2022 · 7 comments · Fixed by #2162
Closed

When .mill-jvm-opts doesn't end with a newline, the last parameter is ignored #2140

phfroidmont opened this issue Nov 23, 2022 · 7 comments · Fixed by #2162
Milestone

Comments

@phfroidmont
Copy link

phfroidmont commented Nov 23, 2022

This is a follow up from scalameta/metals#4557.

It is easily verified as removing the last line from .mill-jvm-opts in test resources, makes the test fail.
On the other hand, I don't see any obvious issue in the parser code.

lefou added a commit to lefou/mill that referenced this issue Nov 25, 2022
Tried to reprodude com-lihaoyi#2140, which claims, we miss the last option when the opts file does not end with a newline.

Can't reproduce locally.

Here is a hex dump of the file added:

```
$ hexdump -C main/client/test/resources/file-wo-final-newline.txt
00000000  0a 23 20 63 6f 6d 6d 65  6e 74 20 61 66 74 65 72  |.# comment after|
00000010  20 61 6e 20 65 6d 70 74  79 20 6c 69 6e 65 0a 2d  | an empty line.-|
00000020  44 50 52 4f 50 45 52 54  59 5f 50 52 4f 50 45 52  |DPROPERTY_PROPER|
00000030  4c 59 5f 53 45 54 5f 56  49 41 5f 4a 56 4d 5f 4f  |LY_SET_VIA_JVM_O|
00000040  50 54 53 3d 76 61 6c 75  65 2d 66 72 6f 6d 2d 66  |PTS=value-from-f|
00000050  69 6c 65 0a 2d 58 73 73  31 32 30 6d              |ile.-Xss120m|
0000005c
```
@lefou
Copy link
Member

lefou commented Nov 25, 2022

I added a test case which only shows that parsing that file works.

Can you confirm, that it properly works for you, when you add a final newline to that file?

@phfroidmont
Copy link
Author

I confirm it works in your test with or without the newline. Actually I did something similar when trying to fix the issue myself.
Where it doesn't work, is when I run the integration tests. If I remove the last line in .mill-jvm-opts and run ./mill integration.forked.test, the relevant test fails.
That is very odd to me, there has to be some magic happening that I don't know about.

@lefou
Copy link
Member

lefou commented Nov 26, 2022

If I remove the last line in .mill-jvm-opts and run ./mill integration.forked.test, the relevant test fails.

I'm reading "remove the last line" but I think you meant the "remove the last newline", don't you?

Can you override the compile target and print out the system properties? Then, place a sys property on the last line with and without a final newline and see if it is included or not.

lefou added a commit that referenced this issue Nov 26, 2022
Tried to reproduce #2140, which claims, we miss the last option when the
opts file does not end with a newline.

Can't reproduce locally.

Pull request: #2145
@phfroidmont
Copy link
Author

Yes, I meant the last newline.
I still have the issue but I found a precision on the context.

In an empty directory, I created 2 files:

.mill-jvm-opts (No newline at the end, hexdump is exactly as in #2145)


# comment after an empty line
-DPROPERTY_PROPERLY_SET_VIA_JVM_OPTS=value-from-file
-Xss120m

build.sc

import mill._
import java.lang.management.ManagementFactory
import scala.jdk.CollectionConverters._


def checkJvmOpts() = T.command {
  val runtime = ManagementFactory.getRuntimeMXBean()
  val args = runtime.getInputArguments().asScala.toSet
  println("###### Properties ####")
  println(System.getProperties.keySet().asScala.toSet.mkString("\n"))
  println("###### JVM args ######")
  println(args.mkString("\n"))
  ()
}

And then, I ran:

$ mill version
[1/1] version
0.10.8

$ mill -i checkJvmOpts
[1/1] checkJvmOpts
###### Properties ####
<...>
PROPERTY_PROPERLY_SET_VIA_JVM_OPTS
<...>
###### JVM args ######
-Dmill.jvm_opts_applied=true
-DMILL_VERSION=0.10.8
-DPROPERTY_PROPERLY_SET_VIA_JVM_OPTS=value-from-file
-Djna.nosys=true
-DMILL_CLASSPATH=/nix/store/cajzcgzv8n2bikadg90kfnplqglp12yq-mill-0.10.8/bin/.mill-wrapped

$ mill checkJvmOpts
[1/1] checkJvmOpts
###### Properties ####
<...>
PROPERTY_PROPERLY_SET_VIA_JVM_OPTS
<...>
###### JVM args ######
-DMILL_VERSION=0.10.8
-DPROPERTY_PROPERLY_SET_VIA_JVM_OPTS=value-from-file
-Xss120m

So the problem only occurs when using the -i flag.

@lefou
Copy link
Member

lefou commented Nov 29, 2022

That's it! There is a different place where we read that file:

mill/build.sc

Line 1164 in f03c3c3

| while IFS= read line

This is probably the culprit. Nice detective work!

lefou added a commit to lefou/mill that referenced this issue Nov 30, 2022
I tried to keep POSIX shell compatibility, hence it's a bit lenghty.

Fix com-lihaoyi#2140
@lefou
Copy link
Member

lefou commented Dec 1, 2022

@phfroidmont I implemented a fix in #2162. But I didn't include a test case, because of effort/complexity to refactor this script for test-ability. Do you mind to test it for your case?

@phfroidmont
Copy link
Author

I confirm it works.
Thanks for all your work!

lefou added a commit that referenced this issue Dec 2, 2022
…ne (#2162)

Reworked `.mill-jvm-opts` reader in shell script prepended to the mill assembly jar.

I tried to keep POSIX shell compatibility, hence it's a bit lenghty.

Fix #2140

Pull request: #2162
@lefou lefou added this to the 0.10.10 milestone Dec 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants