Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make mill -i to default under Windows OS #781

Closed
lefou opened this issue Mar 2, 2020 · 13 comments
Closed

Make mill -i to default under Windows OS #781

lefou opened this issue Mar 2, 2020 · 13 comments

Comments

@lefou
Copy link
Member

lefou commented Mar 2, 2020 •
edited

AFAIK, mill under Windows will only work when started with -i option. Until someone comes with proper fix it makes much sense to disable mill server mode under Windows.

See discussion https://gitter.im/lihaoyi/mill?at=5e5c146fd045e2582509fba4
This was referenced Mar 2, 2020
Closed
Closed
@sake92
Copy link
Contributor

sake92 commented Apr 14, 2020

Mostly fixed in #791 😃
Works correctly but pipe returns Failed to connect to server.. (with exit code 1)
I'll try to find more time to debug that also.

@DamianReeves
Copy link

Is there any chance of getting some more background as to what the issue here is?

@lihaoyi
Copy link
Member

lihaoyi commented Jul 5, 2020

#874

@DamianReeves
Copy link

#874

What if you are not running on WSL for windows. I'm also trying to get this working at a powershell or cmd prompt. Things work with -i , but why is it needed?

@lihaoyi
Copy link
Member

lihaoyi commented Jul 5, 2020

You'll have to investigate and tell us if it's needed or not, and why.

@DamianReeves
Copy link

If I could understand a bit more what -i is supposed to enable maybe I can understand what fails on windows. At worst though we should probably just make -i the default for windows. I'm also not sure how we discover WSL vs normal windows, and if that distinction is needed.

Also it should be noted that WSL2 has a different kernel from WSL1... I can take a look at how it executes in WSL2 as I have a machine with that installed.

@lefou
Copy link
Member Author

lefou commented Jul 6, 2020

Mill -i forces mill to run in a single Java process (entry point: MillMain). By default, it will run two processes, a client (entry point: MillClientMain) and a server process, which is started by the client in case there is no free running one. The communication between these two processes is the culprit on Windows. Also, because of the unclear implications of the name --interactive, we deprecated -i and introduced two new options, see #927.

@lefou lefou mentioned this issue Feb 6, 2021
Merged
lefou added a commit that referenced this issue Feb 6, 2021
@lefou
Various Fixed for Windows (#1146)
84021ec 
Various fixes only shown on Windows, probably also elsewhere.

* Fixed string stream writer/reader used for mill client-server communication, by using an explicit charset for byte conversion
* Avoid `java.nio.file.Paths.get(url.getFile())` in favour of `java.nio.file.Paths.get(url.toURI())` as the latter handles windows paths with leading drive letters well
* Improved CI setup for Windows
* Better run/spawn of Java sub-processes by selecting the same Java executable which is used by mill

We still have some CI failures on Windows, as we need some of these fixed in mill before we can benefit from it. Expect more PRs.

Probably related issues:
* #781
* #874
* #646
* #1098

Commits:

* CI: Added test on Windows

* Try a better handing of windows absolute paths
I hope to avoid a error like:
  Illegal char <:> at index 2: /D:/a/mill/mill/

* Improve string assertion message?

* More details in assertion error

* Use explicit charset for string-byte conversion

* Search java exe in same JAVA_HOME as mill is using

* CI: job deps

* check for Win

* Debug run java process

Pull request: #1146
@sake92
Copy link
Contributor

sake92 commented Mar 26, 2021

@DamianReeves I did some fixing in #1230
Can you please try to build latest mill and test if it works?
Instructions here:
https://gitter.im/lihaoyi/mill?at=605dbef03b9278255bd52443

@james-s-w-clark
Copy link
Contributor

james-s-w-clark commented Jun 10, 2022 •
edited

  • Installed latest mill with scoop (0.10.4)
  • Downloaded sample project
  • Ran a few sample commands (mill foo.compile, mill foo.run) and checked project/out/mill-worker-.....=-1/run, saw the command updating in there (and the commands completed sucessfully)
    • So, it looks like client-server mode is working correctly by default

Just doing a little more testing (running mill at the same time on two different projects). I didn't look into the "locked pipe" stuff, but I thought each server/worker gets its own directory. Package structure of the workers looks the same in MacOS and Windows anyway.... so I'd guesss each package could have 5 independent servers/workers.
Will update comment and tag when I know if there's a problem/if ticket can be closed

@james-s-w-clark
Copy link
Contributor

  • 1 project, Thread.sleep(10_000) , 2x mill foo.run and 1x mill foo.compile → 3 workers, all fine
  • copy project with tiny changes in the clone
    • do 3x mill moduleName.run in each project
      • each project/out directory gets 3 worker/server subfolders. The worker names all have the same prefix mill-worker-y4i2xmtm5Hv8lzskMkPKIWcZtwM=-<index>. I'm not sure what decides this naming (I thought it was a hash of the code/module names or so), but they all seem like independent workers.

@sake92 Thanks for your merge! I don't see the error you mentioned 🎉
@lefou I believe you can close this ticket :)

@sake92
Copy link
Contributor

sake92 commented Jun 10, 2022

@IdiosApps now we use junixsocket library which supports native Windows10 implementation of UNIX pipes, so it is much more reliable, thankfully. Glad you confirmed everything works! 🥳

Btw, you get at most 5 servers worker per JVM+project_path (if one is busy, another one will be instantiated).
See https://github.com/com-lihaoyi/mill/blob/main/main/client/src/mill/main/client/MillClientMain.java#L108

@james-s-w-clark
Copy link
Contributor

@sake92 Nice, I'll have a little look into UNIX pipes. I'm not sure what they are in this context - I guess like sockets, from what I've seen.

Ah that makes sense, thank you for the explanation and the link! I was curious about worker folders today when looking into this issue I opened.

@lefou
Copy link
Member Author

lefou commented Jun 12, 2022

Closing this one, as Windows now properly supports client-server-mode. Also Mill in general is able to recover from am failing server connection by running in-process.

@lefou lefou closed this as completed Jun 12, 2022
@james-s-w-clark james-s-w-clark mentioned this issue Jun 29, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@lihaoyi @DamianReeves @lefou @sake92 @james-s-w-clark