Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLI]: Wandb fails with ServiceStartProcessError #5765

Closed
EricLBuehler opened this issue Jun 22, 2023 · 3 comments · Fixed by #5738
Closed

[CLI]: Wandb fails with ServiceStartProcessError #5765

EricLBuehler opened this issue Jun 22, 2023 · 3 comments · Fixed by #5738
Labels
a:cli Area: Client

Comments

@EricLBuehler
Copy link

Describe the bug

When calling wandb.login(), ServiceStartProcessError is raised.

!pip install wandb
import wandb
wandb.login()
---------------------------------------------------------------------------
ServiceStartProcessError                  Traceback (most recent call last)
<ipython-input-19-f4bc2b362420> in <module>
----> 1 wandb.login()

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_login.py in login(anonymous, key, relogin, host, force, timeout)
     72     """
     73     _handle_host_wandb_setting(host)
---> 74     if wandb.setup()._settings._noop:
     75         return True
     76     kwargs = dict(locals())

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_setup.py in setup(settings)
    325     settings: Optional[Settings] = None,
    326 ) -> Optional["_WandbSetup"]:
--> 327     ret = _setup(settings=settings)
    328     return ret
    329 

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_setup.py in _setup(settings, _reset)
    318         _WandbSetup._instance = None
    319         return None
--> 320     wl = _WandbSetup(settings=settings)
    321     return wl
    322 

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_setup.py in __init__(self, settings)
    301             _WandbSetup._instance._update(settings=settings)
    302             return
--> 303         _WandbSetup._instance = _WandbSetup__WandbSetup(settings=settings, pid=pid)
    304 
    305     def __getattr__(self, name: str) -> Any:

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_setup.py in __init__(self, pid, settings, environ)
    112 
    113         self._check()
--> 114         self._setup()
    115 
    116         tracelog_mode = self._settings._tracelog

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_setup.py in _setup(self)
    248 
    249     def _setup(self) -> None:
--> 250         self._setup_manager()
    251 
    252         sweep_path = self._settings.sweep_param_path

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_setup.py in _setup_manager(self)
    275         if self._settings._disable_service:
    276             return
--> 277         self._manager = wandb_manager._Manager(settings=self._settings)
    278 
    279     def _teardown_manager(self, exit_code: int) -> None:

~/.local/lib/python3.8/site-packages/wandb/sdk/wandb_manager.py in __init__(self, settings)
    143         token = _ManagerToken.from_environment()
    144         if not token:
--> 145             self._service.start()
    146             host = "localhost"
    147             if use_grpc:

~/.local/lib/python3.8/site-packages/wandb/sdk/service/service.py in start(self)
    197 
    198     def start(self) -> None:
--> 199         self._launch_server()
    200 
    201     @property

~/.local/lib/python3.8/site-packages/wandb/sdk/service/service.py in _launch_server(self)
    191                 self._wait_for_ports(fname, proc=internal_proc)
    192             except Exception as e:
--> 193                 _sentry.reraise(e)
    194             self._startup_debug_print("wait_ports_done")
    195             self._internal_proc = internal_proc

~/.local/lib/python3.8/site-packages/wandb/analytics/sentry.py in reraise(self, exc)
    144         # this will messily add this "reraise" function to the stack trace,
    145         # but hopefully it's not too bad
--> 146         raise exc.with_traceback(sys.exc_info()[2])
    147 
    148     @_noop_if_disabled

~/.local/lib/python3.8/site-packages/wandb/sdk/service/service.py in _launch_server(self)
    189             self._startup_debug_print("wait_ports")
    190             try:
--> 191                 self._wait_for_ports(fname, proc=internal_proc)
    192             except Exception as e:
    193                 _sentry.reraise(e)

~/.local/lib/python3.8/site-packages/wandb/sdk/service/service.py in _wait_for_ports(self, fname, proc)
    114                     proc_err=proc.stderr.read() if proc.stderr else "",
    115                 )
--> 116                 raise ServiceStartProcessError(
    117                     f"The wandb service process exited with {proc.returncode}. "
    118                     "Ensure that `sys.executable` is a valid python interpreter. "

ServiceStartProcessError: The wandb service process exited with 1. Ensure that `sys.executable` is a valid python interpreter. You can override it with the `_executable` setting or with the `WANDB__EXECUTABLE` environment variable.

Additional Files

No response

Environment

WandB version:
0.15.4

OS:
20.04 Ubuntu x86_64 GNU/Linux

Python version:
Python 3.8.10

Versions of relevant libraries:
N/A

Additional Context

This occurs even when I use a Huggingface trainer without wandb. However, the min. reproducable is the above.

@EricLBuehler EricLBuehler added the a:cli Area: Client label Jun 22, 2023
@lambdamatt
Copy link

@EricLBuehler try running pip install --upgrade click or pip install click==7.1.0. The 0.15.4 requirements.txt is satisfied with the base ubuntu 20.04 python3-click package, but they do not seem to be compatible and I have reproduced this same behavior and fix.

@EricLBuehler
Copy link
Author

Thank you, that fixed it!

@JerryDaHeLian
Copy link

Good job!好使了,非常感谢!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:cli Area: Client
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants