Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syscall: external process launching slower than other languages #31895

Open
bobjalex opened this issue May 7, 2019 · 8 comments

Comments

@bobjalex
Copy link

commented May 7, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.5 windows/amd64

Does this issue reproduce with the latest release?

Yes.
I do not have a Linux installation, so this report does not apply to Linux.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\bobja\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\GoLib
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\bobja\AppData\Local\Temp\go-build355190882=/tmp/go-build -gno-record-gcc-switches

What did you do?

Running some test programs written in Go that launch lots of external processes. Noticed that Go was running slower than same operations in Python, Ruby, and Java. So wrote some programs to time just process launching in the mentioned languages. Those programs are attached, and do demonstrate the performance difference.

timing.zip
The attached directory has files to demonstrate the timings. Just unzip it into some directory and run the included Python script. See the README file. All Python programs are Python 3.

What did you expect to see?

Hoped to see Go running faster than (or at least as fast as) the other languages.

What did you see instead?

Go ran slower. All of the other languages showed performance similar to each other. But Go was slower than all of them by a factor of around 1.5. Ouput from a run of my test script with a newly installed Go 1.12.5:

all_timings.py 1000

Hello from "Go"
(using os.StartProcess).
18.2624008s

Hello from "Go"
(using exec.Command).
18.1920394s

Hello from "Python".
12.799870s

Hello from "Ruby".
13.036183s

Hello from "Java".
12.796388101s

@ianlancetaylor ianlancetaylor changed the title External process launching slower than other languages syscall: external process launching slower than other languages May 7, 2019

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

I think issue may be specific to Windows. I ran your script on my GNU/Linux laptop and got

Hello from "Go"
 (using os.StartProcess).
203.121311ms

Hello from "Go"
 (using exec.Command).
167.040798ms

Hello from "Python".
0.200595s

Hello from "Ruby".
0.166577798s

Hello from "Java".
0.253690545s
@bobjalex

This comment has been minimized.

Copy link
Author

commented May 9, 2019

@alexbrainman

This comment has been minimized.

Copy link
Member

commented May 12, 2019

@bobjalex thank you for reporting the problem.

I don't have Java or Python or Ruby installed on my Windows computer. So I could not verify your findings.

But, as far as I can see from your results, each Go process start is about 5ms slower than other languages. And there is a 5ms sleep here

defer time.Sleep(5 * time.Millisecond)
Could you try and remove the sleep and see, if your problem goes away?

Thank you.

Alex

Could you try

@bobjalex

This comment has been minimized.

Copy link
Author

commented May 14, 2019

@bobjalex

This comment has been minimized.

Copy link
Author

commented May 14, 2019

@alexbrainman

This comment has been minimized.

Copy link
Member

commented May 15, 2019

In summary, the sleep does appear to be the culprit.

I am glad we know what your problem is.

Unfortunately we cannot remove sleep. Otherwise we have trouble removing executable file once process has exited. See #25965 for details, for example.

If you have any good suggestions, I am all ears.

Alex

@bobjalex

This comment has been minimized.

Copy link
Author

commented May 18, 2019

One possibility: remove the 5ms, and when deleting the executable retry several times if the delete fails, until it finally succeeds.

I've been piddling around with testing the need for the delay: instrumented my process_launch_timing program with some code that attempts to detect when the executable is not "released" on return from exec.Run. Using Go with the 5ms removed, I added this to the repeated-launching program:

  • use a do-nothing launched-program target that is writable and is not being used by other processes
  • upon return from Run, attempt to open the target executable for writing:
    os.OpenFile(executable, os.O_WRONLY, 0755)
    I confirmed that this fails if the file is being executed, and succeeds if not.
  • If the above test fails, print a message.

Not really sure how well this works, but it did report problems if I provoked it by manually running the executable during a test run. Of course my environment is a home computer with not a whole lot going on besides the usual Windows 10 background stuff. And I'm using a fully up to date, modern version of Windows.

With that program, I can reliably run a million repetitions without any "open executable immediately after wait" errors detected. (If I start 2 instances of that program running at the same time on the same target executable, it's fun to watch the chaos that ensues -- suggesting the that detection technique is working.)

I have left the delay out of my personal Go environments, to see what happens. So far, so good.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented May 19, 2019

One possibility: remove the 5ms, and when deleting the executable retry several times if the delete fails, until it finally succeeds.

Sounds reasonable. Would you like to adjust Go standard library and see what happens? Send your change for review, if it works for you.

Not really sure how well this works, but it did report problems if I provoked it by manually running the executable during a test run.

We have many tests and many different environments https://build.golang.org to test your change. And other Go users will complain, if your change will not work for them.

Of course my environment is a home computer with not a whole lot going on besides the usual Windows 10 background stuff.

Yes, your environment matters. See, for example, my comment #25965 (comment) But I can test your change in my environment.

Alex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.