x/sys/windows/svc: test failures with "The specified service has been marked for deletion."

[bcmills]

--- FAIL: TestExample (0.54s)
    svc_test.go:88: CreateService(myservice) failed: The specified service has been marked for deletion.
FAIL
FAIL	golang.org/x/sys/windows/svc	0.559s
…
--- FAIL: TestMyService (0.52s)
    mgr_test.go:258: UpdateConfig failed: The specified service has been marked for deletion.
FAIL
FAIL	golang.org/x/sys/windows/svc/mgr	0.547s

2020-10-18T23:04:17-eeed37f/windows-amd64-longtest
2020-10-09T02:54:20-dfb3f7c/windows-amd64-longtest
2020-08-19T03:55:08-9a32b3a/windows-amd64-longtest
2020-08-17T15:53:16-9781c65/windows-amd64-longtest
2020-08-17T08:59:35-3ff754b/windows-amd64-longtest
2019-11-20T15:59:48-bd43791/windows-amd64-longtest
2019-11-20T13:05:36-6bfc516/windows-amd64-longtest
2019-11-19T19:55:28-f068ffe/windows-amd64-longtest
2019-11-19T06:07:38-e882bf8/windows-amd64-longtest
2019-11-18T13:31:27-cf1e2d5/windows-amd64-longtest
2019-11-18T07:38:50-2f86c98/windows-amd64-longtest
2019-11-15T15:19:21-52ab431/windows-amd64-longtest
2019-11-05T14:28:33-ac3223d/windows-amd64-longtest
2019-10-29T15:55:21-f43be2a/windows-amd64-longtest
2019-10-25T09:01:51-53bf42e/windows-amd64-longtest
2019-10-25T02:14:31-6c3a3bf/windows-amd64-longtest
2019-10-24T07:30:52-e66fe6e/windows-amd64-longtest
2019-10-23T14:50:28-b69606a/windows-amd64-longtest
2019-10-22T10:09:44-742c48e/windows-amd64-longtest

CC @alexbrainman @zx2c4

[networkimprov]

cc @mattn

[alexbrainman]

@bcmills thank you for creating this issue.

The error message is ERROR_SERVICE_MARKED_FOR_DELETE returned from DeleteService

https://docs.microsoft.com/en-us/windows/win32/api/winsvc/nf-winsvc-deleteservice

Interestingly, when we see a failure, both golang.org/x/sys/windows/svc and golang.org/x/sys/windows/svc/mgr tests fail. To me this sounds like package tests interfere with each other. In fact both test install the same service (same name and source). I wonder, if golang.org/x/sys/windows/svc and golang.org/x/sys/windows/svc/mgr run at the same time. Is this possible? If yes, then this can certainly happen.

I have seen this error before DeleteService was run while service process was still running.

I have also run

for /l %i in (1 1 10) do go test -count=1 golang.org/x/sys/windows/svc/...

and this command runs fine.

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
ok      golang.org/x/sys/windows/svc    3.475s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.057s
?       golang.org/x/sys/windows/svc/example    [no test files]
ok      golang.org/x/sys/windows/svc/mgr        0.108s

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
ok      golang.org/x/sys/windows/svc    3.508s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.061s
?       golang.org/x/sys/windows/svc/example    [no test files]
ok      golang.org/x/sys/windows/svc/mgr        0.056s

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
ok      golang.org/x/sys/windows/svc    3.410s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.112s
?       golang.org/x/sys/windows/svc/example    [no test files]
ok      golang.org/x/sys/windows/svc/mgr        0.069s

But, if I interrupt my test by pressing Ctrl+C, this happens

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
exit status 3221225786
FAIL    golang.org/x/sys/windows/svc    1.339s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.076s

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>

and I can see service process is still running. I used Process Explorer

https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer

and I know service executable is called a.exe

And indeed, if I start my test again, I see the error.

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>for /l %i in (1 1 10) do go test -count=1 golang.org/x/sys/windows/svc/...

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
--- FAIL: TestExample (0.58s)
    svc_test.go:82: Delete failed: The specified service has been marked for deletion.
FAIL
FAIL    golang.org/x/sys/windows/svc    0.633s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.060s
?       golang.org/x/sys/windows/svc/example    [no test files]

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>

But, if I stop service with

sc stop myservice

I can see a.exe process disappears. And I can run my test again.

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>for /l %i in (1 1 10) do go test -count=1 golang.org/x/sys/windows/svc/...

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
ok      golang.org/x/sys/windows/svc    3.500s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.087s
?       golang.org/x/sys/windows/svc/example    [no test files]
ok      golang.org/x/sys/windows/svc/mgr        0.081s

c:\Users\Alex\dev\src\golang.org\x\sys\windows\svc>go test -count=1 golang.org/x/sys/windows/svc/...
ok      golang.org/x/sys/windows/svc    3.583s
?       golang.org/x/sys/windows/svc/debug      [no test files]
ok      golang.org/x/sys/windows/svc/eventlog   0.060s
?       golang.org/x/sys/windows/svc/example    [no test files]
ok      golang.org/x/sys/windows/svc/mgr        0.067s

Alex

[bcmills]

I wonder, if golang.org/x/sys/windows/svc and golang.org/x/sys/windows/svc/mgr run at the same time. Is this possible?

Yes, tests of different packages can and often do run at the same time.

[zx2c4]

Using a random service name might help

[gopherbot]

Change https://golang.org/cl/267603 mentions this issue: windows/svc: stop service before deletion in TestExample

[gopherbot]

Change https://golang.org/cl/267604 mentions this issue: windows/svc/mgr: rename service in TestMyService

[alexbrainman]

Yes, tests of different packages can and often do run at the same time.

I am surprised. Are you talking about builders or go test command in general?

Using a random service name might help

Good half-advice. I will rename service in one of the tests - that should be good enough. I don't like the random bit - it is easier to manage / write code when things have known predefined names.

Thank you.

Alex

[bcmills]

Yes, tests of different packages can and often do run at the same time.

I am surprised. Are you talking about builders or go test command in general?

Both? Note that it is also possible for a user to run go test on a package multiple times simultaneously. For example, I often run go test -short ./... in one terminal and go test -race ./... simultaneously in another. The golang.org/x/... tests should be written such that they reliably pass when run that way.

[bcmills]

(The -p flag controls how many test binaries a single invocation of go test can run in parallel; see https://tip.golang.org/cmd/go/#hdr-Compile_packages_and_dependencies.)

[alexbrainman]

For example, I often run go test -short ./... in one terminal and go test -race ./... simultaneously in another.

I don't do that.

The golang.org/x/... tests should be written such that they reliably pass when run that way.

No argument from me.

The -p flag controls how many test binaries a single invocation of go test can run in parallel;

Sure. I hardly ever use anything more than go test.

Alex