New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to run watchman watch <directory_name> #69

Closed
piotrpalek opened this Issue Dec 2, 2014 · 27 comments

Comments

Projects
None yet
9 participants
@piotrpalek

piotrpalek commented Dec 2, 2014

When I try to run watchamn watch some_dir_name I get this error:

/Users/piotrpalek/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
1417520511: tid=2122277632 unable to talk to your watchman!

I am using OSX Yosemite and installed watchman via homebrew.

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Dec 2, 2014

Contributor

Can you run the following commands and paste the output here?

ls -l /Users/piotrpalek/Library /Users/piotrpalek/Library/LaunchAgents /Users/piotrpalek/Library/LaunchAgents/com.github.facebook.watchman.plist
tail $TMPDIR/.watchman.piotrpalek.log
Contributor

wez commented Dec 2, 2014

Can you run the following commands and paste the output here?

ls -l /Users/piotrpalek/Library /Users/piotrpalek/Library/LaunchAgents /Users/piotrpalek/Library/LaunchAgents/com.github.facebook.watchman.plist
tail $TMPDIR/.watchman.piotrpalek.log
@piotrpalek

This comment has been minimized.

Show comment
Hide comment
@piotrpalek

piotrpalek Dec 2, 2014

Hey, I've since tried multiple things, fixing permissions in disk tool, also disabling the service via:
sudo launchctl unload -w ~/Library/LaunchAgents/com.github.facebook.watchman.plist but that didn't help. So I tried the only true solution to all problems, and restarted my macbook. And that fixed my problem :)

piotrpalek commented Dec 2, 2014

Hey, I've since tried multiple things, fixing permissions in disk tool, also disabling the service via:
sudo launchctl unload -w ~/Library/LaunchAgents/com.github.facebook.watchman.plist but that didn't help. So I tried the only true solution to all problems, and restarted my macbook. And that fixed my problem :)

@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

I'm also getting this issue on 10.9, but restarting doesn't seem to help. I'm getting this:

[ixtli@ixtlis-Mac-Pro] ~ $ watchman watch ./Project
Could not open job overrides database at: /private/var/db/launchd.db/com.apple.launchd/overrides.plist: 13: Permission denied
launch_msg(): Socket is not connected
1420648873: tid=2052051728 unable to talk to your watchman!
[ixtli@ixtlis-Mac-Pro] ~ $

Do you have to run watchman as root? If not then the way you're loading launchd jobs is wrong: according to http://launchd.info/ there is a per-user overrides.plist and watchman seems to be trying to access one owned by root :(

ixtli commented Jan 7, 2015

I'm also getting this issue on 10.9, but restarting doesn't seem to help. I'm getting this:

[ixtli@ixtlis-Mac-Pro] ~ $ watchman watch ./Project
Could not open job overrides database at: /private/var/db/launchd.db/com.apple.launchd/overrides.plist: 13: Permission denied
launch_msg(): Socket is not connected
1420648873: tid=2052051728 unable to talk to your watchman!
[ixtli@ixtlis-Mac-Pro] ~ $

Do you have to run watchman as root? If not then the way you're loading launchd jobs is wrong: according to http://launchd.info/ there is a per-user overrides.plist and watchman seems to be trying to access one owned by root :(

@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

oh and:

-rw-r--r--   1 ixtli staff 1117 Jan  7 11:41 /Users/ixtli/Library/LaunchAgents/com.github.facebook.watchman.plist

/Users/ixtli/Library/LaunchAgents:
total 28
[...]
lrwxr-xr-x 1 ixtli staff   46 Oct 20 15:16 homebrew.mxcl.redis.plist -> /usr/local/opt/redis/homebrew.mxcl.redis.plist
[ixtli@ixtlis-Mac-Pro] ~ $ tail $TMPDIR/.watchman.ixtli.log
tail: /var/folders/23/1xzdj7mx26g_wrwlwvs0f1kc0000gn/T//.watchman.ixtli.log: No such file or directory

ixtli commented Jan 7, 2015

oh and:

-rw-r--r--   1 ixtli staff 1117 Jan  7 11:41 /Users/ixtli/Library/LaunchAgents/com.github.facebook.watchman.plist

/Users/ixtli/Library/LaunchAgents:
total 28
[...]
lrwxr-xr-x 1 ixtli staff   46 Oct 20 15:16 homebrew.mxcl.redis.plist -> /usr/local/opt/redis/homebrew.mxcl.redis.plist
[ixtli@ixtlis-Mac-Pro] ~ $ tail $TMPDIR/.watchman.ixtli.log
tail: /var/folders/23/1xzdj7mx26g_wrwlwvs0f1kc0000gn/T//.watchman.ixtli.log: No such file or directory
@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Jan 7, 2015

Contributor

Watchman doesn't contain any code to elevate privilege. You should always run it as yourself unless you are doing something unusual. Did either of you run it as root? Trying to understand what is broken for you and how it got broken.

Contributor

wez commented Jan 7, 2015

Watchman doesn't contain any code to elevate privilege. You should always run it as yourself unless you are doing something unusual. Did either of you run it as root? Trying to understand what is broken for you and how it got broken.

@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

I did not ever try to run it as root :( Is there an easy way I can completely delete all launchd plists and reinstall it? I installed from source originally as per the instructions on the website.

ixtli commented Jan 7, 2015

I did not ever try to run it as root :( Is there an easy way I can completely delete all launchd plists and reinstall it? I installed from source originally as per the instructions on the website.

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Jan 7, 2015

Contributor

The only thing that watchman creates is ~/Library/LaunchAgents/com.github.facebook.watchman.plist.

To remove that, I would do:

launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
rm ~/Library/LaunchAgents/com.github.facebook.watchman.plist

I'm also tracking an issue where the TMPDIR (which is random by default on OSX) gets baked in to the plist file. My recommendation until I can push a fix for that is to export TMPDIR=/tmp in your .bashrc or equivalent before you run watchman.

Contributor

wez commented Jan 7, 2015

The only thing that watchman creates is ~/Library/LaunchAgents/com.github.facebook.watchman.plist.

To remove that, I would do:

launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
rm ~/Library/LaunchAgents/com.github.facebook.watchman.plist

I'm also tracking an issue where the TMPDIR (which is random by default on OSX) gets baked in to the plist file. My recommendation until I can push a fix for that is to export TMPDIR=/tmp in your .bashrc or equivalent before you run watchman.

@piotrpalek

This comment has been minimized.

Show comment
Hide comment
@piotrpalek

piotrpalek Jan 7, 2015

The only thing I did was install it via brew and try to use it via ember-cli. Nothing more nothing less, and a restart fixed my issues.

piotrpalek commented Jan 7, 2015

The only thing I did was install it via brew and try to use it via ember-cli. Nothing more nothing less, and a restart fixed my issues.

@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

@wez Thank you for your help. I will try to completely remove my install of watchman and install the brew build and track what happens.

@piotrpalek This issue has persisted over several restarts for me :(

ixtli commented Jan 7, 2015

@wez Thank you for your help. I will try to completely remove my install of watchman and install the brew build and track what happens.

@piotrpalek This issue has persisted over several restarts for me :(

@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

For reference, when i try to unload that job I get

[ixtli@ixtlis-Mac-Pro] ~ $ launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
Could not open job overrides database at: /private/var/db/launchd.db/com.apple.launchd/overrides.plist: 13: Permission denied
launchctl: Error unloading: com.github.facebook.watchman

Then, as a super user:

[ixtli@ixtlis-Mac-Pro] ~ $ sudo launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
Password:
launchctl: Dubious ownership on file (skipping): /Users/ixtli/Library/LaunchAgents/com.github.facebook.watchman.plist
nothing found to unload

ixtli commented Jan 7, 2015

For reference, when i try to unload that job I get

[ixtli@ixtlis-Mac-Pro] ~ $ launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
Could not open job overrides database at: /private/var/db/launchd.db/com.apple.launchd/overrides.plist: 13: Permission denied
launchctl: Error unloading: com.github.facebook.watchman

Then, as a super user:

[ixtli@ixtlis-Mac-Pro] ~ $ sudo launchctl unload ~/Library/LaunchAgents/com.github.facebook.watchman.plist
Password:
launchctl: Dubious ownership on file (skipping): /Users/ixtli/Library/LaunchAgents/com.github.facebook.watchman.plist
nothing found to unload
@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

Restart and reinstall don't work :( I can confirm that brew install watchman then watchman watch ~ works fine on the machine running 10.10 sitting next to this one.

ixtli commented Jan 7, 2015

Restart and reinstall don't work :( I can confirm that brew install watchman then watchman watch ~ works fine on the machine running 10.10 sitting next to this one.

@ixtli

This comment has been minimized.

Show comment
Hide comment
@ixtli

ixtli Jan 7, 2015

Aaand to bring an end to my comment spam here I deleted /var/db/launchd.db/com.apple.launchd because it only seemed to contain a mapping of what services i'd enabled in the "sharing" preferences pane and then restarted. It appears to have recreated the folder with the EXACT same permissions but now watchman launches just fine.

I have no idea :/

ixtli commented Jan 7, 2015

Aaand to bring an end to my comment spam here I deleted /var/db/launchd.db/com.apple.launchd because it only seemed to contain a mapping of what services i'd enabled in the "sharing" preferences pane and then restarted. It appears to have recreated the folder with the EXACT same permissions but now watchman launches just fine.

I have no idea :/

@piotrpalek

This comment has been minimized.

Show comment
Hide comment
@piotrpalek

piotrpalek Jan 8, 2015

@ixtli I would help you but I just poked around when it wasn't working, and then restarted and it did work after that. So I really don't know what caused the problem or how I (if at all) fixed it.

piotrpalek commented Jan 8, 2015

@ixtli I would help you but I just poked around when it wasn't working, and then restarted and it did work after that. So I really don't know what caused the problem or how I (if at all) fixed it.

wez added a commit that referenced this issue Feb 2, 2015

osx: fixup some launchd warts
Summary:
On OSX, and perhaps some other environments, the TMPDIR
environment is randomized by default.  This means that in the default
install the watchman socket will be in a random location when you first
run it.  Since we capture this path in the launchd plist this can
result in not being able to communicate with watchman in cases where you
log in again and have a new randomized TMPDIR.

To avoid this issue we now default to always setting WATCHMAN_STATE_DIR
to a fixed and known default.  If not set explicitly at configure time,
we'll default to `$prefix/var/run/watchman` so that the default is
generally either `/usr/local/var/run/watchman` or `/var/run/watchman`
depending on whether this is a local or a system install.

The install rule for creating this dir has to make this dir world
writable so that any user can create their socket there.  This should be
no less secure than putting these files in `$TMPDIR`.

We also need to take some extra measures to ensure that launchd can pick
up these fixed paths; when we decide that we need to spawn watchman; we
repair it by first unloading an existing plist, then writing out the new
one with the newly configured settings.  This allows the repaired plist
to be loaded and the service to be started.  Without this, we would
rewrite the plist only, and launchd would punt on loading it since it
was already loaded.

#69

Test Plan:
`make integration` to prove that this didn't impact the integration
environment.  ad-hoc tests locally; `/usr/local/bin/watchman watch-list` to
start a pre-existing version of watchman without these changes, then
`./watchman watch-list` to run a version from this installation.

The local build is always able to replace and re-launch itself.  The old build
was not able to do this; it would get stuck for the timeout before reporting
that it was unable to talk to watchman.

Reviewers: durham, sid0

Reviewed By: sid0

Differential Revision: https://reviews.facebook.net/D32253
@vthommeret

This comment has been minimized.

Show comment
Hide comment
@vthommeret

vthommeret Mar 2, 2015

I'm seeing this issue. @wez Is there anything I can provide to help diagnose this issue?

Permissions for that file:

$ ls -l ~/Library/LaunchAgents/com.github.facebook.watchman.plist                                     
-rw-r--r--  1 vernon  staff  1121 Mar  2 15:27 /Users/vernon/Library/LaunchAgents/com.github.facebook.watchman.plist

When I try to watch a directory:

~/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
1425338952: tid=1974063872 unable to talk to your watchman!

When I restarted once, it started working, then the issue cropped up again. Now even after restarting, deleting the PLIST file, etc... I get that issue.

Running Mac OS X 10.10.3. Installed via Homebrew. Also tried the latest and compiling from source and ran into the same issue. There's nothing in the log file.

vthommeret commented Mar 2, 2015

I'm seeing this issue. @wez Is there anything I can provide to help diagnose this issue?

Permissions for that file:

$ ls -l ~/Library/LaunchAgents/com.github.facebook.watchman.plist                                     
-rw-r--r--  1 vernon  staff  1121 Mar  2 15:27 /Users/vernon/Library/LaunchAgents/com.github.facebook.watchman.plist

When I try to watch a directory:

~/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
1425338952: tid=1974063872 unable to talk to your watchman!

When I restarted once, it started working, then the issue cropped up again. Now even after restarting, deleting the PLIST file, etc... I get that issue.

Running Mac OS X 10.10.3. Installed via Homebrew. Also tried the latest and compiling from source and ran into the same issue. There's nothing in the log file.

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Mar 3, 2015

Contributor

@synotic what happens if you run launchctl load -f ~/Library/LaunchAgents/com.github.facebook.watchman.plist?

It might also be interesting to trace the syscall activity... can you try this in one window:

# trace syscalls for launchd (pid == 1) and its children
sudo dtruss -f -p 1 > /tmp/wat 2>&1

Then try the launchctl load -f ~/Library/LaunchAgents/com.github.facebook.watchman.plist and once it returns to your prompt, CTRL-C the dtruss process in the other window, and then put /tmp/wat in a gist or otherwise find a way to get it to me?

Contributor

wez commented Mar 3, 2015

@synotic what happens if you run launchctl load -f ~/Library/LaunchAgents/com.github.facebook.watchman.plist?

It might also be interesting to trace the syscall activity... can you try this in one window:

# trace syscalls for launchd (pid == 1) and its children
sudo dtruss -f -p 1 > /tmp/wat 2>&1

Then try the launchctl load -f ~/Library/LaunchAgents/com.github.facebook.watchman.plist and once it returns to your prompt, CTRL-C the dtruss process in the other window, and then put /tmp/wat in a gist or otherwise find a way to get it to me?

@vthommeret

This comment has been minimized.

Show comment
Hide comment
@vthommeret

vthommeret Mar 3, 2015

Running that command (I'm assuming it should be -F instead of -f?), I get the same output ("Operation not permitted").

I sent you an email with the trace output.

vthommeret commented Mar 3, 2015

Running that command (I'm assuming it should be -F instead of -f?), I get the same output ("Operation not permitted").

I sent you an email with the trace output.

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Mar 3, 2015

Contributor

http://apple.stackexchange.com/a/88670 describes what lives in /var/db/launchd.db... I don't have such a directory on my system. My guess is that the permissions or contents of yours are broken somehow. You may be able to simply remove them to fix things?

Contributor

wez commented Mar 3, 2015

http://apple.stackexchange.com/a/88670 describes what lives in /var/db/launchd.db... I don't have such a directory on my system. My guess is that the permissions or contents of yours are broken somehow. You may be able to simply remove them to fix things?

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Mar 27, 2015

Contributor

@synotic did you get things working?

Contributor

wez commented Mar 27, 2015

@synotic did you get things working?

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Apr 26, 2015

Contributor

I think this might have the same underlying cause as #90 which turned out to be a umask related problem with the .plist file we create for launchd, so I'm closing out this issue.

Contributor

wez commented Apr 26, 2015

I think this might have the same underlying cause as #90 which turned out to be a umask related problem with the .plist file we create for launchd, so I'm closing out this issue.

@wez wez closed this Apr 26, 2015

@stroborobo

This comment has been minimized.

Show comment
Hide comment
@stroborobo

stroborobo Jun 30, 2015

I can still reproduce this with version 3.3.0.

bo-mbp:bo> watchman watch-project $DIR >/dev/null
/Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
2015-06-30T16:36:20,758: [cli] unable to talk to your watchman on /usr/local/Cellar/watchman/3.3.0/var/run/watchman/bo!
bo-mbp:bo> l -d /Users/bo/Library/LaunchAgents
drwxr-xr-x+ 11 bo  staff   374B Jun 28 14:05 /Users/bo/Library/LaunchAgents
bo-mbp:bo> l /Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist
-rw-r--r--+ 1 bo  staff   1.4K Jun 30 16:36 /Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist
bo-mbp:bo> l -d /usr/local/Cellar/watchman/3.3.0/var/run/watchman
drwxr-xr-x+ 3 bo  admin   102B Jun 28 21:42 /usr/local/Cellar/watchman/3.3.0/var/run/watchman/
bo-mbp:bo> l /usr/local/Cellar/watchman/3.3.0/var/run/watchman
bo-mbp:bo> 

Reboot or can I get some logs for you? dtrace? Or just let Disk Utility repair permissions? Rebooting fixed it for me on 3.2.0. (I just didn't have the time to do the debugging that day.)

stroborobo commented Jun 30, 2015

I can still reproduce this with version 3.3.0.

bo-mbp:bo> watchman watch-project $DIR >/dev/null
/Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist: Operation not permitted
2015-06-30T16:36:20,758: [cli] unable to talk to your watchman on /usr/local/Cellar/watchman/3.3.0/var/run/watchman/bo!
bo-mbp:bo> l -d /Users/bo/Library/LaunchAgents
drwxr-xr-x+ 11 bo  staff   374B Jun 28 14:05 /Users/bo/Library/LaunchAgents
bo-mbp:bo> l /Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist
-rw-r--r--+ 1 bo  staff   1.4K Jun 30 16:36 /Users/bo/Library/LaunchAgents/com.github.facebook.watchman.plist
bo-mbp:bo> l -d /usr/local/Cellar/watchman/3.3.0/var/run/watchman
drwxr-xr-x+ 3 bo  admin   102B Jun 28 21:42 /usr/local/Cellar/watchman/3.3.0/var/run/watchman/
bo-mbp:bo> l /usr/local/Cellar/watchman/3.3.0/var/run/watchman
bo-mbp:bo> 

Reboot or can I get some logs for you? dtrace? Or just let Disk Utility repair permissions? Rebooting fixed it for me on 3.2.0. (I just didn't have the time to do the debugging that day.)

@wez

This comment has been minimized.

Show comment
Hide comment
@wez

wez Jun 30, 2015

Contributor

Try removing the .plist file?

We've seen some cases where launchd won't load a plist after a major OS upgrade or restore from another disk, even though the visible permissions look fine.

The simplest resolution has been to remove the plist file and then let watchman re-create it.

Contributor

wez commented Jun 30, 2015

Try removing the .plist file?

We've seen some cases where launchd won't load a plist after a major OS upgrade or restore from another disk, even though the visible permissions look fine.

The simplest resolution has been to remove the plist file and then let watchman re-create it.

@stroborobo

This comment has been minimized.

Show comment
Hide comment
@stroborobo

stroborobo Jun 30, 2015

Doesn't change a thing.

stroborobo commented Jun 30, 2015

Doesn't change a thing.

@nayrmi

This comment has been minimized.

Show comment
Hide comment
@nayrmi

nayrmi Aug 1, 2015

Knorkebrot, are you running tmux or screen? I hit this same issue until I exited to a non-tmux session, and now I'm working ok again.

Looking around forums, it seems that launchctl and tmux aren't always good friends.

nayrmi commented Aug 1, 2015

Knorkebrot, are you running tmux or screen? I hit this same issue until I exited to a non-tmux session, and now I'm working ok again.

Looking around forums, it seems that launchctl and tmux aren't always good friends.

@llj098

This comment has been minimized.

Show comment
Hide comment
@llj098

llj098 Aug 8, 2015

same issue here, but I was in a tmux session, after I quit the tmux session, it works.

llj098 commented Aug 8, 2015

same issue here, but I was in a tmux session, after I quit the tmux session, it works.

@stroborobo

This comment has been minimized.

Show comment
Hide comment
@stroborobo

stroborobo Aug 8, 2015

@nayrmi I do, but it didn't work outside of tmux as well.

stroborobo commented Aug 8, 2015

@nayrmi I do, but it didn't work outside of tmux as well.

@bhamiltoncx

This comment has been minimized.

Show comment
Hide comment
@bhamiltoncx

bhamiltoncx Aug 28, 2015

Contributor

Have we reported the failure to talk to launchctl from tmux to the tmux team?

I couldn't find any issues related to that (open or closed):

https://github.com/tmux/tmux/issues

Contributor

bhamiltoncx commented Aug 28, 2015

Have we reported the failure to talk to launchctl from tmux to the tmux team?

I couldn't find any issues related to that (open or closed):

https://github.com/tmux/tmux/issues

@paulcavallaro

This comment has been minimized.

Show comment
Hide comment
@paulcavallaro

paulcavallaro Oct 23, 2016

Just ran into this myself, and I'm not sure it was ever raised with the tmux team, so I filed a report here: tmux/tmux#600

paulcavallaro commented Oct 23, 2016

Just ran into this myself, and I'm not sure it was ever raised with the tmux team, so I filed a report here: tmux/tmux#600

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment