New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error creating Windows Service, encoding issue calling Win32 Service API #3796

Open
WayneKeenan opened this Issue Dec 2, 2017 · 9 comments

Comments

Projects
None yet
4 participants
@WayneKeenan
Contributor

WayneKeenan commented Dec 2, 2017

buildbot: latest from github
Python: 3.6.3
Windows: 10

The error when trying to start the service is :

Error starting service: The service did not respond to the start or control request in a timely fashion.
Debugging service BuildBot - press Ctrl+C to stop.
****************** b'...\\worker_dir'
Error 0xC0000003 - The instance's SvcRun() method failed

Traceback (most recent call last):
  File "...\env\lib\site-packages\win32\lib\win32serviceutil.py", line 835, in SvcRun
    self.SvcDoRun()
  File "..\env\lib\site-packages\buildbot_worker-2017.12.2-py3.6.egg\buildbot_worker\scripts\windows_service.py", line 226, in SvcDoRun
    if not self._checkConfig():
  File "..\env\lib\site-packages\buildbot_worker-2017.12.2-py3.6.egg\buildbot_worker\scripts\windows_service.py", line 211, in _checkConfig
    dir_string)
  File "..\env\lib\site-packages\win32\lib\win32serviceutil.py", line 272, in SetServiceCustomOption
    win32api.RegSetValueEx(key, option, 0, win32con.REG_SZ, value);
TypeError: Objects of type 'bytes' can not be converted to Unicode.

A quick and dirty monkey patch to get past the error:

Change :

dir_string = os.pathsep.join(self.dirs).encode("mbcs")

To

dir_string = os.pathsep.join(self.dirs)
@rodrigc

This comment has been minimized.

Show comment
Hide comment
@rodrigc

rodrigc Dec 3, 2017

Collaborator

@WayneKeenan actually your change is correct. Values supplied to win32api.RegSetValueEx must be unicode not bytes. Is that the only fix you needed to get things working?

Can you submit a pull request with this change?

Collaborator

rodrigc commented Dec 3, 2017

@WayneKeenan actually your change is correct. Values supplied to win32api.RegSetValueEx must be unicode not bytes. Is that the only fix you needed to get things working?

Can you submit a pull request with this change?

@WayneKeenan

This comment has been minimized.

Show comment
Hide comment
@WayneKeenan

WayneKeenan Dec 3, 2017

Contributor

That was the only monkey patch, but I could only get the worker service to start in debug mode, and even then there was another exception exiting debug mode. (less critical)

So I abandoned it for a dirty workaround:

sc create buildbot binPath= "%BUILDBOT_WORKER_EXE% start %WORKER_DIR%" error= ignore obj= %YOUR_DOMAIN_OR_DOT%\%USERNAME% password= %PASSWORD start= auto depend= tcpip DisplayName= "Buildbot Worker"
sc start buildbot

But with that method windows says the service doest start, because it not correctly responding to the Windows management signals, however, the worker is running. hacky I know.

Although it would be better to fix the root cause, my next step would probably be to use https://nssm.cc/

Contributor

WayneKeenan commented Dec 3, 2017

That was the only monkey patch, but I could only get the worker service to start in debug mode, and even then there was another exception exiting debug mode. (less critical)

So I abandoned it for a dirty workaround:

sc create buildbot binPath= "%BUILDBOT_WORKER_EXE% start %WORKER_DIR%" error= ignore obj= %YOUR_DOMAIN_OR_DOT%\%USERNAME% password= %PASSWORD start= auto depend= tcpip DisplayName= "Buildbot Worker"
sc start buildbot

But with that method windows says the service doest start, because it not correctly responding to the Windows management signals, however, the worker is running. hacky I know.

Although it would be better to fix the root cause, my next step would probably be to use https://nssm.cc/

WayneKeenan added a commit to WayneKeenan/buildbot that referenced this issue Dec 3, 2017

@rodrigc

This comment has been minimized.

Show comment
Hide comment
@rodrigc

rodrigc Dec 3, 2017

Collaborator

What were the other exceptions you saw when starting buildbot as a Windows service?
We should give it a shot at trying to fix those problems.

Collaborator

rodrigc commented Dec 3, 2017

What were the other exceptions you saw when starting buildbot as a Windows service?
We should give it a shot at trying to fix those problems.

@WayneKeenan

This comment has been minimized.

Show comment
Hide comment
@WayneKeenan

WayneKeenan Dec 4, 2017

Contributor

When exiting the service with CTRL-c when initiated as a debug start :

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\cygwin64\home\wayne\buildbot_windows_10_env\lib\site-packages\buildbot_worker-2017.12.2-py3.6.egg\buildbot_worker\scripts\windows_service.py", line 408, in redirectCaptureThread
    ec, data = win32file.ReadFile(handle, CHILDCAPTURE_BLOCK_SIZE)
pywintypes.error: (109, 'ReadFile', 'The pipe has been ended.')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\program files\python36\Lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "c:\program files\python36\Lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\cygwin64\home\wayne\buildbot_windows_10_env\lib\site-packages\buildbot_worker-2017.12.2-py3.6.egg\buildbot_worker\scripts\windows_service.py", line 412, in redirectCaptureThread
    if err[0] != winerror.ERROR_BROKEN_PIPE:
TypeError: 'error' object does not support indexing

Info 0x40001004 - The BuildBot service has stopped (BuildBot).
Contributor

WayneKeenan commented Dec 4, 2017

When exiting the service with CTRL-c when initiated as a debug start :

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\cygwin64\home\wayne\buildbot_windows_10_env\lib\site-packages\buildbot_worker-2017.12.2-py3.6.egg\buildbot_worker\scripts\windows_service.py", line 408, in redirectCaptureThread
    ec, data = win32file.ReadFile(handle, CHILDCAPTURE_BLOCK_SIZE)
pywintypes.error: (109, 'ReadFile', 'The pipe has been ended.')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\program files\python36\Lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "c:\program files\python36\Lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\cygwin64\home\wayne\buildbot_windows_10_env\lib\site-packages\buildbot_worker-2017.12.2-py3.6.egg\buildbot_worker\scripts\windows_service.py", line 412, in redirectCaptureThread
    if err[0] != winerror.ERROR_BROKEN_PIPE:
TypeError: 'error' object does not support indexing

Info 0x40001004 - The BuildBot service has stopped (BuildBot).
@tardyp

This comment has been minimized.

Show comment
Hide comment
@tardyp

tardyp Dec 4, 2017

Member

@WayneKeenan
re: nssm : I have to admit that the buidbot-windows community is not as big as I'd like it to be, and thus the quality of the integration is not perfect.

We are looking for windows/python experts to fix the integration, especially with the service stuff. Feel free to propose enhancements. If this is a complete rewrite using more modern stuff, I think this would be fine.

Member

tardyp commented Dec 4, 2017

@WayneKeenan
re: nssm : I have to admit that the buidbot-windows community is not as big as I'd like it to be, and thus the quality of the integration is not perfect.

We are looking for windows/python experts to fix the integration, especially with the service stuff. Feel free to propose enhancements. If this is a complete rewrite using more modern stuff, I think this would be fine.

@rodrigc

This comment has been minimized.

Show comment
Hide comment
@rodrigc

rodrigc Dec 5, 2017

Collaborator

@WayneKeenan thanks for posting that stacktrace. I think we can fix this.
I wrote some notes on OSError exceptions in Windows when working on Twisted:

https://github.com/twisted/twisted/blob/trunk/src/twisted/python/filepath.py#L332

In the line where it throws the exception, can you print out the members of err:

print("DEBUG1: ", err)
print("DEBUG2: ", dir(err))
if err[0] != winerror.ERROR_BROKEN_PIPE

I'm nat at my Windows machine right now, so need to figure out which member of err to
use to get the error.

Collaborator

rodrigc commented Dec 5, 2017

@WayneKeenan thanks for posting that stacktrace. I think we can fix this.
I wrote some notes on OSError exceptions in Windows when working on Twisted:

https://github.com/twisted/twisted/blob/trunk/src/twisted/python/filepath.py#L332

In the line where it throws the exception, can you print out the members of err:

print("DEBUG1: ", err)
print("DEBUG2: ", dir(err))
if err[0] != winerror.ERROR_BROKEN_PIPE

I'm nat at my Windows machine right now, so need to figure out which member of err to
use to get the error.

tardyp added a commit that referenced this issue Dec 8, 2017

@estan

This comment has been minimized.

Show comment
Hide comment
@estan

estan May 28, 2018

@tardyp I think the same change should be made in the worker service right?

estan commented May 28, 2018

@tardyp I think the same change should be made in the worker service right?

@tardyp

This comment has been minimized.

Show comment
Hide comment
@tardyp

tardyp May 28, 2018

Member

probably..

Member

tardyp commented May 28, 2018

probably..

estan pushed a commit to estan/buildbot that referenced this issue May 28, 2018

Elvis Stansvik
Use .winerror accessor instead of indexing
Seems at some point pywin32 silently dropped the support for indexing.
This fixes the error brought up by @rodrigc here:

  buildbot#3796 (comment)
@estan

This comment has been minimized.

Show comment
Hide comment
@estan

estan commented May 28, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment