Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Events not firing #10

Closed
whatupdave opened this Issue March 30, 2011 · 116 comments
Dave Newman

I'm trying to pin down exactly what's happening here. The specs don't pass, I ran it in debug mode:

creating Makefile
CFLAGS='-isysroot /Developer/SDKs/MacOSX10.6.sdk -mmacosx-version-min=10.6 -mdynamic-no-pic -std=gnu99 -Os -pipe -Wmissing-prototypes -Wreturn-type -Wmissing-braces -Wparentheses -Wswitch -Wunused-function -Wunused-label -Wunused-parameter -Wunused-variable -Wunused-value -Wuninitialized -Wunknown-pragmas -Wshadow -Wfour-char-constants -Wsign-compare -Wnewline-eof -Wconversion -Wshorten-64-to-32 -Wglobal-constructors -pedantic' /usr/bin/gcc -isysroot /Developer/SDKs/MacOSX10.6.sdk -mmacosx-version-min=10.6 -mdynamic-no-pic -std=gnu99 -dead_strip -framework CoreServices -D DEBUG=true -o '/users/dave/code/temp/rb-fsevent/bin/fsevent_watch' fsevent/fsevent_watch.c
fsevent_watch compiled
.
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

config.sinceWhen    18446744073709551615
config.latency      0.300000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

FSEventStreamRef @ 0x1001085c0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path'
   latestEventId = -1
   latency = 300000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

config.sinceWhen    18446744073709551615
config.latency      0.300000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path

FSEventStreamRef @ 0x1001085c0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path'
   latestEventId = -1
   latency = 300000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0


append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0


append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0


append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0

F
append_path called for: /users/dave/code/temp/rb-fsevent/spec/fixtures
  resolved path to: /Users/dave/code/temp/rb-fsevent/spec/fixtures

config.sinceWhen    18446744073709551615
config.latency      0.500000
config.flags        00000000
config.paths
  /Users/dave/code/temp/rb-fsevent/spec/fixtures

FSEventStreamRef @ 0x1001085e0:
   allocator = 0x7fff709faee0
   callback = 0x100001522
   context = {0, 0x0, 0x0, 0x0, 0x0}
   numPathsToWatch = 1
   pathsToWatch = 0x7fff709faee0
        pathsToWatch[0] = '/Users/dave/code/temp/rb-fsevent/spec/fixtures'
   latestEventId = -1
   latency = 500000 (microseconds)
   flags = 0x00000000
   runLoop = 0x0
   runLoopMode = 0x0



Failures:

  1) FSEvent should work with path with an apostrophe
     Failure/Error: @results.should == [custom_path.to_s + '/']
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path/"]
            got: [] (using ==)
       Diff:
       @@ -1,2 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/custom 'path/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:30:in `block (2 levels) in <top (required)>'

  2) FSEvent should catch new file
     Failure/Error: @results.should == [@fixture_path.to_s + '/']
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/"]
            got: [] (using ==)
       Diff:
       @@ -1,2 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:40:in `block (2 levels) in <top (required)>'

  3) FSEvent should catch file update
     Failure/Error: @results.should == [@fixture_path.join("folder1/").to_s]
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/"]
            got: [] (using ==)
       Diff:
       @@ -1,2 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:49:in `block (2 levels) in <top (required)>'

  4) FSEvent should catch files update
     Failure/Error: @results.should == [@fixture_path.join("folder1/").to_s, @fixture_path.join("folder1/folder2/").to_s]
       expected: ["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/", "/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/folder2/"]
            got: [] (using ==)
       Diff:
       @@ -1,3 +1,2 @@
       -["/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/",
       - "/users/dave/code/temp/rb-fsevent/spec/fixtures/folder1/folder2/"]
       +[]
     # ./spec/rb-fsevent/fsevent_spec.rb:61:in `block (2 levels) in <top (required)>'

Finished in 13.4 seconds
5 examples, 4 failures

I'm running ruby-1.9.2-p180
Mac OSX 10.6.6

I'm not sure what other information would be helpful. If i build the debug binary and run it on a directory i'm not seeing any events firing.

Any ideas?

Dave Newman

Ok after filing this I checked apple update and updated to Mac 10.6.7 which has fixed the problem of file system events not firing.

Travis Tilley
Collaborator

CRAZY! I just read the email github sent about the bug and was worried I forgot to double-check everything in xcode3.latest or something silly. ;)

I'm glad that fixed things for you, but it's worrisome that there might be a configuration out there that just flat out doesn't work. shrug

Travis Tilley ttilley closed this March 30, 2011
Dave Newman

I'm not 100% if it was the mac version or something else going on in the system. I rebooted a few times to see if it made a difference. I did install windows boot camp the other day so I guess my beautiful mac will never be the same....

Roland Moriz
rmoriz commented May 03, 2011

having exactly the same issues. running 10.6.7 with all patches + Xcode 4.0.2

events get fired by OSX as http://www.fernlightning.com/doku.php?id=software:fseventer:start shows them

could not find a way to fix this. tried several different rubies (1.8.7, 1.9.2, 1.8.7EE) without success.

Roland Moriz
rmoriz commented May 03, 2011

However v0.3.10 works (at least specs):

(in /private/tmp/rb-fsevent)
/Users/rmoriz/.rvm/rubies/ree-1.8.7-2011.03/bin/ruby -S bundle exec rspec ./spec/rb-fsevent/fsevent_spec.rb
No examples were matched by {:focus=>true}, running all
creating Makefile
fsevent_watch compiled
....

Finished in 13.27 seconds
4 examples, 0 failures
Travis Tilley ttilley reopened this May 06, 2011
Travis Tilley
Collaborator
ttilley commented May 06, 2011

rmoriz: there are two fsevent APIs. one is private, fairly low level, and essentially allows for a userspace application to insert itself in the middle of I/O events in the kernel... which allows not-well-behaved code to cause serious problems. This is the API used by spotlight, and apparently also fseventer.

The public API for FSEvents is based on a daemon that makes use of this much lower level API to log a much less detailed version of events, trimmed down to the directory level, under the /root_of_that_volume/.fseventsd/ directory.

Now... the most confusing detail of what you're seeing is that 0.3.10 works and 0.4 does not. This is different from anyone else I've heard from so far has been seeing: you either have issues that break use of FSEvents by any and all applications (minus spotlight, and cheaters that use undocumented APIs), or issues that break FSEvents when working within a specific volume.

I recently heard from a user who disabled spotlight, re-enabled it, rebooted twice, and POOF... magic. Things worked again. I'm still a bit frustrated I wasn't able to figure out what was going on before his machine just started working again out of nowhere, and thus don't understand the solution to what he was seeing. However, more out of superstition than logic (as spotlight uses /dev/fsevents, not fseventsd), you might want to go to preferences -> spotlight -> privacy, add your root filesystem, reboot, remove it, reboot. You have no idea how painful it is for me to even suggest such a thing.

Before hopping down that rabbit hole, however, it's worth re-trying 0.4 if you're seeing 0.3.10 work. With the changes involved it makes very little sense for you to not be seeing events. I can see there being compounding issues of all kinds that might stop them from reaching you, but not that fsevent_watch itself isn't seeing them. Please re-compile with debugging as well.

export FWDEBUG="true"
gem install rb-fsevent

...Note to self: make that a commandline option.

Roland Moriz
rmoriz commented May 06, 2011

@ttilley that was me on twitter ;)

btw. 0.3.10 didn't work, only the tests were all green, sorry for the confusion. As far as I saw the 0.4 released added more tests.

The strange thing was, that other fsevent based apps worked but rb-fsevent did not. First I thought maybe the noatime mount option (b/c of my SSD) could be a problem. I remembered that I've disabled spotlight indexing on the main partition. After removing the partition form the ignore list the indexer started. I rebooted 2 times because of other things…

Then rb-fsevent (used in guard) started working perfectly out of the blue. The rb-fsevent tests got green!
I've now disabled spotlight indexing again, rb-fsevent still works.

tl;dr

  • re-active spotlight indexing so mds starts
  • reboot => rb-fsevent may work now.
  • disabled the spotlight indexing again

BTW: I've updated my Xcode to the 4.0.x release. That alone did not help but maybe it's part of the solution? (System was already 10.6.7 when I started)

Travis Tilley ttilley closed this May 06, 2011
Travis Tilley
Collaborator
ttilley commented May 06, 2011

oh. well. alrighty then!

Definitely good to know that we have since improved the quality of the test suite so that it doesn't give false passes... Especially since I wasn't aware of them. A few of the modifications between 0.3.10 and 0.4 were actually done to prevent false failures. Both may have had the same cause.

Lachlan Donald
lox commented June 01, 2011

I just had exactly the same issue. Very strange, nothing had changed in terms of ruby, rb-fsevent or any other software on my system, my sync script just stopped getting events. I could see events with fseventer, but nothing was coming through to my script. OSX is 10.6.7.

I tried rebooting twice, no result. I excluded the root partition from indexing, rebooted and everything is working again.

Lachlan Donald
lox commented June 01, 2011

Unfortunately the problem re-surfaces once I remove my root partition from the Spotlight exclusion list. Frustrating.

Myles Byrne

Same issue here. Adding the root partition to the spotlight exclusion list and rebooting fixed the problem

Lachlan Donald
lox commented June 06, 2011

Actually, I was able to add just the directories in question to the exclusion list and it works fine.

Benjamin Quorning

On OSX 10.6.7, I have absolutely no luck making rb-fsevent work. Tried adding my "code" folder to Spotlight’s exclusion list and rebooted the machine. Still doesn’t work.

Travis Tilley ttilley reopened this June 17, 2011
Travis Tilley
Collaborator

can you all check each volume for /.fseventsd/no_log ? Also, make sure each volume has a /.fseventsd/fseventsd-uuid file containing a UUID of the format:

ABC01D2E-F345-6ABC-D7E8-F9AB01C234D5

Since I'm not using the volume specific API, it might also just blindly be using the root filesystem settings... That's certainly possible.

Also check to make sure there's an as-root process running called fseventsd (or, more specifically, /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/Support/fseventsd).

I guess also make sure there's a /dev/fsevents ? Looks like it should be mode 0644 with major,minor of 12,6393236.

Also, grep /private/var/log/system.log for 'fseventsd'. Note that "destroying old logs" messages are harmless, and simply mean that the filesystem was modified without that specific fseventsd knowing about it (like... booting into your lion partition on the same drive and having its' spotlight index your snow leopard partition... or what have you). Example:

./system.log:Jun 17 15:09:38 Travis-Tilleys-MacBook-Pro fseventsd[20767]: event logs in /.fseventsd out of sync with volume.  destroying old logs. (1 170038 170050)

I really really wish I knew what the problem was, specifically, for you guys so that I could write in a check for it and display a helpful warning. Any information is invaluable.

Benjamin Quorning

Just tried it on my home computer: it works fine. Running the “Singular path” example code from the README, I now see fsevent.run happily printing a line when a Dir is changed.

My work computer (which I reported rb-fsevent not working on yesterday) just returned when calling fsevent.run. Maybe pipe.eof? returns true right away? Anyhow, I’ll try debugging per your instructions above when I get back to my work machine on monday.

Robert Williams

I came across this issue the other day and noticed it may be due to case sensitivity of directory names in the home folder.

All my projects were stored in:

/Users/Rob/Projects

I wanted to tab complete without typing the uppercase 'P' so I renamed projects to:

/Users/Rob/projects

Great so now I can tab complete.. Hold on a minute Guard has stopped working.

Renaming it back to Projects fixed it.

I tried creating a spec to catch this error but it passes without issues. :/

  # This only happens in the users home directory from what i can tell
  # so to prove it we test both
  {
    :in_home_directory     => Pathname.new(File.expand_path('~')),
    :in_fixtures_directory => Pathname.new(File.expand_path('../../fixtures/', __FILE__))
  }.each { |directory_description, path|

    it "should work with folder that has been renamed from Uppercase to lowercase #{directory_description}" do

      uppercase_path = path.join("FseventCasetest")
      lowercase_path = path.join("fseventcasetest")
      uppercase_file_path = uppercase_path.join("watch.txt")
      lowercase_file_path = lowercase_path.join("watch.txt")

      # Make sure fixtures are clean
      FileUtils.rm_rf(uppercase_path)
      FileUtils.rm_rf(lowercase_path)

      FileUtils.mkdir(uppercase_path)

      FileUtils.touch(uppercase_file_path)

      # Use OS X mv that is case sensitive and allows renaming of a directory
      # to its uppercase equivalent in the same parent directory
      `mv -v #{uppercase_path} #{lowercase_path}`


      @fsevent.watch lowercase_path.to_s do |paths|
        @results += paths
      end

      run
      FileUtils.touch lowercase_file_path
      stop

      File.delete lowercase_file_path
      # Make sure fixtures are clean
      FileUtils.rm_rf(uppercase_path)
      FileUtils.rm_rf(lowercase_path)

      @results.should == [lowercase_path.to_s + '/']

    end
  }
Travis Tilley
Collaborator

The subset of events involving directory renames (while running) aren't -actually- handled properly, since the communication format between the C subprocess and ruby library doesn't allow for it. This is only an issue when renaming a directory that's explicitly watched (not a subdirectory). I was determined to fix that and a number of other things (see the "coming soon" comments in the readme), but, erm... I got distracted. ^^;

You know, I actually have a fair chunk of time today and throughout this week to devote to the work necessary to check off those TODO items. I guess it's time to get motivated and bang that out...

An aside for @thibaudgg - I intend to use tagged netstrings (probably) as a serialization format. It honestly took an attempt at serializing to JSON in C to make me realize I was being silly creating so much extra work for myself.

Benjamin Quorning

Before I got around to debugging per @ttilley’s request above, I found the following: Using the irbtools gem (version 1.0.4 here, haven’t tested with others) on ruby-1.8.7-p174 (ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]) causes rb-fsevent (calling fsevent.run per the README) to return instantly. Using ruby-1.8.7-p334 [ x86_64 ] or ruby-1.9.2-p180 [ x86_64 ] works just fine with irbtools.

Not requiring irbtools makes rb-fsevent work just fine on ruby-1.8.7-p174 here.

So, that looks like a bug right there. (Though maybe not on this gem…)

Travis Tilley
Collaborator

twitch

Benjamin Quorning

My problems aren’t over yet: Now rb-fsevent does not work on my work computer anymore. Not in 1.8.7 (any patchlevel) or 1.9.2. With or without irbtools, no difference.

  • I have a /.fseventsd folder. (root:admin mode 700)
  • I have no /.fseventsd/no_log file.
  • I do have the /.fseventsd/fseventsd-uuid file with a proper-looking UUID. (root:admin mode 600)
  • The process /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/Support/fseventsd is running
  • There is a device /dev/fsevents (root:wheel mode 644)
  • grep fseventsd /private/var/log/system.log reveals nothing.

This is on OSX 10.6.8, 2.4GHz Intel Core 2 Duo. Let me know if you need more information.

Benjamin Quorning

I just tried running the README sample code from an unsaved TextMate file. Here, Dir.pwd is "/private/var/folders/ds/[something very long]/-Tmp-/" – and rb-fsevent works!

Now, why doesn’t it work when watching my home dir? And how can I help you debug this issue?

Travis Tilley
Collaborator

...wtf? so it works under /private but not outside of /private? can you try other folders that are under /private/?

Benjamin Quorning

Today’s round of debugging leaves me equally baffled…

  1. Tried watching /private (works), /Users/benjamin (works!) and /Users/benjamin/code (doesn’t work).
  2. I tried making a new subfolder /Users/benjamin/kode and watching it as first “kode”, then “Kode.” Both work.
  3. Suspecting that my code folder’s name (“code”) was the problem, I renamed the folder and watched it. Yay, now it works.
  4. Renamed back to “code” to confirm that the problem were indeed the folder’s name. Argh (but yay, I guess), it still works.

Concluding: I don’t know what the problem was, but changing the folder’s name to something else, and back, seems to have solved the problem.

Travis Tilley
Collaborator

That actually says quite a bit. When you rename a folder, that doesn't change the inode that data is kept in... just metadata. So, technically, there wasn't a new 'thing' to watch. It's not the folder itself that's a problem, OR that it's under the specially-handled OSX 'private' folder where exceptions are made in the realpath logic.

[09:13:08][~]$ stat -x code
  File: "code"
  Size: 68           FileType: Directory
  Mode: (0751/drwxr-x--x)         Uid: (  501/ ttilley)  Gid: (   20/   staff)
Device: 14,2   Inode: 5577795    Links: 2
Access: Fri Jul 15 09:12:43 2011
Modify: Fri Jul 15 09:12:43 2011
Change: Fri Jul 15 09:12:43 2011
[09:13:17][~]$ mv code kode
[09:13:24][~]$ stat -x kode
  File: "kode"
  Size: 68           FileType: Directory
  Mode: (0751/drwxr-x--x)         Uid: (  501/ ttilley)  Gid: (   20/   staff)
Device: 14,2   Inode: 5577795    Links: 2
Access: Fri Jul 15 09:12:43 2011
Modify: Fri Jul 15 09:12:43 2011
Change: Fri Jul 15 09:12:43 2011
[09:13:29][~]$ mv kode code
[09:13:40][~]$ stat -x code
  File: "code"
  Size: 68           FileType: Directory
  Mode: (0751/drwxr-x--x)         Uid: (  501/ ttilley)  Gid: (   20/   staff)
Device: 14,2   Inode: 5577795    Links: 2
Access: Fri Jul 15 09:12:43 2011
Modify: Fri Jul 15 09:12:43 2011
Change: Fri Jul 15 09:12:43 2011

Note that the inode (where it's kept on disk, the thing being watched) hasn't changed. It remains 5577795. Also, the access/modify/change times have stayed the same throughout. ALL that has changed is a small chunk of metadata in the inode referring to the directory.

I mean, it's not a solution, but it IS a significant amount of information.

Travis Tilley
Collaborator

It's good to know we're not the only ones seeing fsevents do crazy things of a similar nature -> http://lists.apple.com/archives/filesystem-dev/2011/Jan/msg00000.html

Note that his testing scenario resulted in the file creation event in one scenario, but not another (where the file was deeper in the tree before the multiple-level-ancestor directory was renamed)... and at least for time machine, both cases resulted in an inaccurate backup.

Fun times.

volkanunsal

I'm having this exact same problem. The event are not firing and as a result Guard, which depends on rb-fsevent, does not work on my system. Hopefully a solution can be found for it.

Upgrading to Lion fixed this for me.

Travis Tilley
Collaborator

@tenaciousflea can you perform the checks I mentioned where I re-opened this bug?

I wish I could ssh into one of your machines or something to poke around... in a shared screen session perhaps. As of yet, I have been unable to reliably reproduce this and so I can't dig into what might be causing it in depth. >_<

Travis Tilley
Collaborator

Actually... I live on east coast USA. If any of you still having this problem are ok with letting me shell in and poke around, I'm often on irc.freenode.net as either Aphelion or ttilley (though not as often non-idle). Alternatively, we could set up screen sharing over ichat, which would be nice since we could talk at the same time.

I really really really want to stomp this bug flat and be done with it.

volkanunsal

@ttilley I can offer to help you using my computer. I looked up your post from when you reopened the issue, but I'm not proficient enough with shell programming to do the tests. If you give me brief snippets of code to run, I can run them and tell you what they produce.

Is there a chatroom you go to often? I'm on the East Coast as well, and I can be available over the weekend.

Travis Tilley
Collaborator

I tend to idle in: #ruby-lang #ruby #macruby #rubinius #jruby

Thibaud Guillaume-Gentil
Owner

You can also use #guard (irc.freenode.net).

Travis Tilley
Collaborator

...I just realized it's 7am and I haven't gone to sleep yet. So I hope you're not a morning person.

I have, however, converted fsevent_watch into its own xcode project, replaced the extconf.rb fakeout build system with a rakefile that calls xcodebuild, performed some cleanup, did extensive testing to ensure that this works as expected in 10.6+10.7 using xcode 3.2.6, 4.0.2, 4.1, and 4.2b5 (4.1+ for lion), refactored some C, added a hidden command-line argument to enable file-level events in lion for giggles, and introduced a system of pluggable output formats with the eventual goal of exposing extensive metadata on each event fired.

Graph here: https://github.com/thibaudgg/rb-fsevent/network

Not bad for a sitting. I think.

volkanunsal

Awesome. I'm trying to reproduce the problem at my home computer. Just the day before I was seeing the issue both on my home and my work computer. Then I upgraded to Lion at both places -- which meant upgrading XCode too. Yesterday when I left office, I was still seeing the problem on my work computer, but at my home computer it has...evolved. At least I think it's no longer something to do with rb-fsevent. The problem has moved up to the guard-coffeescript. I am able to see the events are detected on my computer, but guard-coffeescript is not compiling javascript files. Well, at least that's something.

I'm going to have to report back on how the office computer is functioning on Monday. I already posted on guard-coffeescript issue log to let them know about this.

Travis Tilley
Collaborator

crap. there goes another chance to figure out, exactly, what the problem is and write a check for it. just like everyone else who reports this bug you did something and then it magically went away.

volkanunsal

Seems updating to Lion solved it for me as well. It may have been the XCode update, not Lion by itself, though.

Travis Tilley ttilley closed this August 16, 2011
Travis Tilley
Collaborator

...I'm going to just close this issue. I can't reproduce it and everyone who reports it also reports it going away after doing X where X varies. I'm strongly debating a pre-compiled binary approach and am doing some heavy refactoring of fsevent_watch.

Andrey Tarantsov

Guys, I just wanted to chime in to say that LiveReload 2 users are experiencing this exact problem too, most with folders inside Dropbox. I'm 100% sure it's an OS bug. I generally recommend the create-new-folder-and-move-everything approach, and it's yet to fail for anyone (others reported that rebooting solved the issue).

So when you detect a failing folder /foo/bar/boz, you do

cd /foo/bar
mv boz boz.0
mkdir boz
mv boz.0/* boz/*
rmdir boz.0

Ditto for /foo/bar and /foo, if needed. Problem will be solved.

I have no idea what the cause is, but I have plans to include automatic detection of this issue into LiveReload 2.

volkanunsal

@andreyvit

In my case, I figured out the folder name was the cause of the problem. The folders had . and ( characters in their names. Moving to folders without those characters solved my problem. The example you give seems to validate my conclusion.

Andrey Tarantsov

@tenaciousflea Um… which example? /foo/bar/boz was the original name in the example, and it's not even a real name. (This happened to me with perfectly regular folders. In fact, it used to happen on ~/Dropbox/Projects/livereload itself. :)

volkanunsal

@andreyvit

In that case, the mystery continues.

Travis Tilley ttilley reopened this September 14, 2011
Travis Tilley
Collaborator

This bug makes me want to stab myself in the face.

If I can poke at a machine/setup where this problem actually happens that'd be great. Every person who has offered has ended up doing something before I get there (install something, rename folders, upgrade xcode, upgrade macos) and the problem goes away before I can take a look.

Greg Tomei

I am experiencing this on OSX 10.7.1 with rb-event version 0.4.3.1. Its a real pain.

I am finding that autotest-fsevent does work (0.2.5).

Travis Tilley
Collaborator
volkanunsal

@ttilley

This started happening on my computer again after a routine firmware update. I haven't tested it thoroughly to understand its symptoms just yet, but when I moved the folder to the desktop, the issue went away. If you want to sync up today (or tomorrow) to look at my system, that would work for me.

Travis Tilley
Collaborator
Travis Tilley
Collaborator

it was massively helpful to poke around on a machine where this was consistently breaking, and i'm now sure i have a clearer picture of the bug. @andreyvit - ping in case you're interested (or have more info than I do here).

In this particular instance of the bug, the folder 'Xperiments' was returning 'xperiments' via realpath as well as apple's path resolving apis. After renaming the folder to 'xperiments' and then back to 'Xperiments', those apis returned the expected with-caps name. Unfortunately I accidentally fixed the bug on the machine before getting the kind of data I wanted, and I have no idea how to directly parse the fseventsd data logs, but I'm guessing the events were being produced using the with-caps version of the name and thus watching for the without-caps version will miss them entirely.

In short, the bug is that realpath() doesn't give a correct result all the time for case-insensitive-but-preserving HFS+ volumes.

Travis Tilley
Collaborator

i'd also like to note that stat -x was also returning unmodified whatever path it was given. if called on the with-caps path, that's what it gave as a name. same for without-caps.

Travis Tilley
Collaborator

...we are of course back to the scenario where I don't have access to a machine where the bug is occurring, so I can only blindly try things and hope it works. if realpath() isn't returning the correct result, readdir() might not either (though i'm hoping this is unlikely, since ls shows the correct case).

anyways, the idea is to use realpath() or similar and then manually check each directory node for the correct case for that child. slightly painful, but at least it only needs to happen once... when registering a path to watch.

Andrey Tarantsov

@ttilley Great findings. Does actually explain why recreating folders fixes this problem. And btw I have info that 10.7 users also still experience the problem, so it's not fixed in OS X. Will try to play with this new info too.

Travis Tilley
Collaborator

the person who let me poke around was running 10.7.1 actually.

It seems that FSCopyAliasInfo() reliably returns sane results, but again... I no longer have access to a machine where the bug still happens. On the plus side, FSAliasInfo also contains a signature of the containing volume's filesystem type... so checking for this bug as well as the "high latency + macfuse" bug can be done in one go (sshfs can fire when you start writing, but not again when you finish, if the write operation takes more than @latency time).

Travis Tilley
Collaborator

more specifically, calling FSCopyAliasInfo with /users/ttilley/desktop gets us (yes, i named my lion install volume "rawr"):

targetName                 = 'Desktop'
volumeName                 = 'rawr'
pathString                 = '/Users/ttilley/Desktop'
volumeCreateDate           = 0.3394049572.0 (Wednesday, July 20, 2011 7:32:52 PM Eastern Daylight Time)
targetCreateDate           = 0.3394050005.0 (Wednesday, July 20, 2011 7:40:05 PM Eastern Daylight Time)
isDirectory                = YES
parentDirID                = 281309
nodeID                     = 281467
filesystemID               = 0x0000 ('??')
signature                  = 0x482b ('H+')
volumeIsBootVolume         = YES
volumeIsAutomounted        = NO
volumeIsEjectable          = NO
volumeHasPersistentFileIDs = YES
Andrey Tarantsov

Meanwhile, made 2 tools to investigate this:

@ttilley Can you pls look at https://github.com/andreyvit/find-fsevents-bugs/blob/master/find-fsevents-bugs.c, does it look like it would detect the bug you're talking about? I've scanned my Dropbox folder, and so far zero results. Running on the whole disk now.

Andrey Tarantsov

Actually here's a snippet:

realpath(path_buf, real_path_buf);
if (0 != strcmp(path_buf, real_path_buf)) {
    output("Found: '%s' != '%s'\n", path_buf, real_path_buf);
    ++results;
}

Any better ideas?

Travis Tilley
Collaborator

bit too literal:

[17:42:02][find-fsevents-bugs] (master u=)$ ./find-fsevents-bugs /Users/ttilley/Desktop/
Found: '/Users/ttilley/Desktop/' != '/Users/ttilley/Desktop'
Done, 1 result(s).

other than that, I hope so... though checking my ~/.rvm dir takes 30 seconds. :/

I'd like to stress that I haven't a clue whether or not readdir() will give the expected entry if realpath() gets it wrong and I don't have a place to test the theory. If your beta testers could be a resource here that would be glorious.

...I also have never experienced this bug myself on my own machine, so I can't even imagine the conditions with which it is triggered. Dropbox isn't responsible, as not everyone who reported the issue have used it. It does seem to exacerbate the problem though, or at least make it more likely for these conditions to occur.

HEY. Here's an idea: does syncing a mixed-case directory/file between machines result in a broken/inconsistent inode? Make directory 'MixedCase' on machine1, test for issues on machine2? It'd be nice if that ended up being a consistent method of reproducing this bug, but I know not to get my hopes up.

Travis Tilley
Collaborator

You might also enjoy using my updated fsevent_watch binary with debugging enabled, as it's fairly descriptive: https://github.com/ttilley/fsevent_watch/blob/master/fsevent_watch/main.c#L75

...or my MacRuby port, which is significantly easier to read, and feature-filled as fuck (TM): https://github.com/ttilley/mrb-fsevent/blob/master/lib/mrb-fsevent/flags.rb

Andrey Tarantsov

Can you btw give a snippet that calls FSCopyAliasInfo? I'm lost in FSRefs and AliasHandles, and don't really have a burning desire to read all docs. :)

Here's an idea: does syncing a mixed-case directory/file between machines result in a broken/inconsistent in ode?

I can't see why it would be.

I've tried saturating FSEvents with looped renames, but that goes nowhere.

Travis Tilley
Collaborator
FSRef itemRef;
AliasHandle itemAlias;
HFSUniStr255 targetName;
HFSUniStr255 volumeName;
CFStringRef pathString;
FSAliasInfoBitmap returnedInInfo;
FSAliasInfo info;

// get an FSRef here -_-

FSNewAlias(NULL, &itemRef, &itemAlias);
FSCopyAliasInfo(itemAlias, &targetName, &volumeName, &pathString, &returnedInInfo, &info);

I'm working off of the "FSMegaInfo" sample code since it can be a lot less intimidating reading sample code than massive list-every-possible-combination reference docs. ;)

Travis Tilley
Collaborator

oh yeah, anything you don't care about having returned can be NULL instead of a pointer and it will just skip that part. so in theory you can just get the pathString. i think.

AND it requires CoreServices.

...AND you can only get an FSRef for a file that exists, so you can't use it for watching something that has yet to be created. The pattern for that is using an FSRef for the parent directory and a unicode name string, according to the docs. Not very useful if multiple sections of your to-watch path don't exist yet, which should be possible. :/

...simplest path-to-fsref seems to be to create a CFURL for the path and call CFURLGetFSRef

Travis Tilley
Collaborator

really, I don't think we're going to make any progress until after one of your beta testers runs your find-fsevents-bugs app or someone else chimes in here saying they're experiencing the bug. we have no way to test anything.

Travis Tilley
Collaborator

as an aside, my AIM handle is YourTravis and i'm often on irc.freenode.net as either ttilley or Aphelion. if you want to bounce ideas, or if anyone on this bug is still experiencing it (anyone), i'm around and would like nothing better than destroying this bug and never hearing another word about it again.

Latency McLaughlin
Andrey Tarantsov

I don't have anyone who's still having the issue either. The next person to report it will be treated with great care :)

Travis Tilley
Collaborator

@latency - you can remove yourself by clicking "disable notifications for this issue". I accidentally added you by using the ruby instance variable syntax for @latency outside of a code block. I do apologize... and thought that correcting it in an edit might un-do the notification, but this apparently wasn't the case.

The URL is: #10

I unfortunately can accidentally subscribe you, but not intentionally *un*subscribe you.

Travis Tilley
Collaborator

@andreyvit - HFS+ volumes store file names in a funky variant of unicode that attempts to normalize character representations via a different-across-os-releases decomposition algorithm. A subtlety of this algorithm is that besides case-insensitivity, there are also unicode characters that are ignored completely when comparing file names. Case sensitive HFSX volumes, however, do NOT ignore said unicode characters (in addition to being case sensitive). On HFS+, "forwards" and "sdrawrof" (written as printed, not stored) are equivalent filenames.

God I'm glad I'm not a filesystem developer.

In finder, I have a filename that reads !באמת. The filename is 9 bytes. When I ls in iterm, it prints as באמת!. When I ls in Terminal.app, I get the form used in finder. Additionally, when I start IRB in iterm, Dir[*] displays filenames in a completely different order than if I do the same in Terminal:

# iterm
> Dir['*']
 => ["באמת!", "☃"] 

# Terminal
> Dir['*']
 ["☃" ,"!באמת"] <=

However, they both print the same result when realpath is used (with the exclamation point at the end). This is another solid example of multiple APIs returning different results. Perhaps this oddity could be used to simulate the conditions that trigger this bug, even if we don't know the exact conditions that would do so otherwise... and perhaps that's just wishful thinking on my part.

For reference, the actual bytes for that filename are: "\xd7\x91\xd7\x90\xd7\x9e\xd7\xaa\x21"

Travis Tilley
Collaborator

I REALLY wish I got the byte representation of the directory causing this bug when I had the chance, but I didn't think of it. It's certainly possible that renaming it changed this representation.

Travis Tilley
Collaborator

fuck it. I burned a dev support ticket and linked to this issue. Hopefully support can enlighten us (though it might take them a while to do so... I didn't purchase above-basic support).

Travis Tilley
Collaborator

Every simple method of path resolution I know ignores case for the file path part. I have a file named LikeYoMothaFuckaWeee.txt, and if I call realpath() on it in lowercase, the last path component is in lowercase. NSString's fileSystemRepresentation does no better. Neither does stringByStandardizingPath or stringByResolvingSymlinksInPath. Similarly for the methods on NSURL.

BUT... converting a file path NSURL to a file reference path and back results in the correct case for all components every time. Unless someone has a better answer, I think that's what I'm going to use.

[23:20:24][MiXeDcAsE]$ irb
irb(main):001:0> t = Dir['*'][0]
=> "LikeYoMothaFuckaWeee.txt"
irb(main):002:0> t.downcase!
=> "likeyomothafuckaweee.txt"
irb(main):003:0> t = File.realpath(t)
=> "/Users/ttilley/Dropbox/testing/MiXeDcAsE/likeyomothafuckaweee.txt"
irb(main):004:0> u = NSURL.fileURLWithPath(t)
=> #<NSURL:0x400220c20>
irb(main):005:0> u.description
=> "file://localhost/Users/ttilley/Dropbox/testing/MiXeDcAsE/likeyomothafuckaweee.txt"
irb(main):006:0> u.fileReferenceURL.description
=> "file:///.file/id=6571367.13518787"
irb(main):007:0> u.fileReferenceURL.filePathURL.description
=> "file:///Users/ttilley/Dropbox/testing/MiXeDcAsE/LikeYoMothaFuckaWeee.txt"
Andrey Tarantsov

@ttilley To clarify, LiveReload does not call realpath (or anything similar) before starting monitoring. So it was using the same name that Choose Directory dialog returned.

Also, regarding Unicode chars, again I had this problem on latin-only names like ~/Dropbox/Projects/Active/livereload (as far as I remember, the whole ~/Dropbox/Projects was broken back then). And it happened on just one of my computers — I was using two back then, and on the other one all folders were perfectly monitorable (so the problem is not getting replicated by Dropbox).

Aaron Gibralter

I'm having the issue too -- I'm using guard with rspec and rb-fsevent in a app directory located on dropbox. The problem of not detecting changes used to happen on my old MacBookPro running the latest version of 10.6 and rvm+REE; however, it never seemed to happen on my iMac (in the same synched dropbox directory). Now, the problem has appeared all of sudden on my new MacBook Air running the latest version of Lion (in that same dropbox directory) running rvm+ruby1.9.2 and rvm+ruby1.9.3-rc1. I tried running Disk Utility's verify/fix permissions on my Macintosh HD (as I've seen in other posts about this problem) to no avail. I also tried moving the directory (within dropbox) and it didn't seem to help either.

Andrey Tarantsov

@agibralter Your broken directory is a very valuable asset. If you don't mind us looking at your system, please don't try to fix it!

Feel free to say hello either on IRC mentioned above, or on LiveReload Support web chat (Campfire): https://andreytarantsov.campfirenow.com/f2839.

Travis Tilley
Collaborator

I won't be around much of the day... please contact andrey if at all possible. He has as much enthusiasm for seeing this bug die, and might even be more capable at debugging it. ;)

I'd certainly love to see you run his find-fsevents-bugs app to see what it returns.

Travis Tilley
Collaborator

@agibralter - I also want to stress that your broken directory is very helpful for us. We can't reliably reproduce this bug and must depend on users who report it in order to test theories on fixing it.

Travis Tilley
Collaborator

@agibralter - after running andrey's helper tools, i'd love for you to install the pre-release rb-fsevent gem (0.9.0.pre1) to see if the problem persists with that version. I switched to using the AliasInfo method of resolving paths.

Andrey Tarantsov

@ttilley If you are around, please join us in Campfire, and I'll hook you up with remote desktop session too.

Andrey Tarantsov

@agibralter Thanks a lot for letting us in. We found that:

  1. 'Broken' folder is ~/Dropbox/Foo/Bar, latin characters only, nothing fancy.
  2. Readdir and realpath return ~/Dropbox/Foo/Bar, while FSCopyAliasInfo returns ~/Dropbox/Foo/bar (note the lowercase bar).
  3. We've tried fseventsmon on ~/Dropbox/Foo/Bar, ~/Dropbox/Foo/bar, ~/Dropbox/Foo/Bar/subfolder and ~/Dropbox/Foo/bar/subfolder. No changes detected by either of those.

I think the bottom line is:

  • we have a good way to detect this case
  • we have no way to continue monitoring until the problem is fixed
  • we seem to have a way to fix the problem (just rename the folder and then rename it back).

We did not actually try fixing it, and I've asked agibralter to leave it as is for now (and make a copy for further work), so we can try something else if there are any ideas.

I believe the best way forward is to auto-detect and maybe to offer auto-fixing.

Travis Tilley
Collaborator

Sorry I had to duck out in the middle like that.

...I'm also insanely curious to hear what dev support might say when they respond.

Did you get a byte representation of the file name? If not, in ruby 1.9.x:

# where num_in_list is the array index for that file/directory
Dir['*'][num_in_list].bytes.map {|b| '\x' + b.to_s(16)}.join('')
Andrey Tarantsov

@ttilley: No, I did not get byte representation, I'm quite sure it is just a regular English name. And yes, dev support may be a good way forward on this. I think I have 2 support incidents with my Mac dev program, will try to use one. (But everything after this point is probably pure curiosity. I think we have a good plan of detection and recovery.)

I have an idea to set up a VM with OS X, install Dropbox and see if we can get some broken folders after initial sync.

Travis Tilley
Collaborator

I installed dropbox on my 32bit mac running snow leopard just in case there's an oddity in syncing back and forth and I noticed that directory renames don't sync if the only change is in case. renaming 'test' to 'TEST' doesn't sync. you need to rename a file for it to think it's sync-worthy.

Travis Tilley
Collaborator

@andreyvit - True. I think I'll add code that resolves via realpath, copyalias, and for 10.6+ resolving file reference urls (where it just has volume id and inode number and determines the path from that). If they're not all the same... print super helpful error message and quit. for my use case, it doesn't seem like auto-fixing this would be helpful.

Perhaps we should link everyone who has the bug here to add their comments: http://openradar.appspot.com/10207999

Aaron Gibralter

Ok so on my iMac rb-fsevent (0.4.3.1) works just fine at detecting changes in ~/Dropbox/Foo/Bar. When I ran find-fsevents-bugs on the iMac I get this:

15:18 [~/Dropbox/Foo/Bar]  
$ ./find-fsevents-bugs /Users/aarongibralter/Dropbox/Foo/Bar/
Found (realpath): '/Users/aarongibralter/Dropbox/Foo/Bar/' != '/Users/aarongibralter/Dropbox/Foo/Bar'
Found (FSCopyAliasInfo): '/Users/aarongibralter/Dropbox/Foo/Bar/' != '/Users/aarongibralter/Dropbox/Foo/Bar'
Done, 2 result(s).

Whereas when I run it on my macbook air (where rb-fsevents doesn't detect changes) I get thousands of results (Done, 3332 result(s). to be precise).

My iMac is running 10.6.8 and my MBA is running 10.7.1.

Aaron Gibralter

Also, on both computers, Finder and ls show that Foo is capitalized. It may be that when Dropbox creates folders and files on a computer, it does it strangely... the files were all on my iMac to start.

Aaron Gibralter

And one more thing -- it used to work on my macbook air. It just stopped working a couple days ago and I'm really not sure why. I can remember what I could have done to change anything.

(sorry for the spamming!)

Travis Tilley
Collaborator

Response from apple developer support:

Travis

I'm responding to your report of a problem with FSEvents not firing properly.  You wrote:

> While the bug is occuring, realpath()/readdir() and FSCopyAliasInfo()
> return paths with different case.

Yeah, that's most suspicious.  I suspect that this problem has nothing to do with FSEvents per se, but rather that there's case
insensitivity edge case deep within the kernel (either in HFS Plus or in the VFS name lookup cache).

> I have an openradar for this: http://openradar.appspot.com/10207999

Thanks for that.  Given that you only filed this yesterday, I'm going to give it a couple of days to wend its way through
the system, after which I'll see if kernel engineering has any suggestions as to why this might be happening.  I'll
send you another update soon (probably in the second half of next week).
Aaron Gibralter

Interesting. Is there any way to handle the edge case of incorrect capitalization in FSCopyAliasInfo in the mean time?

Travis Tilley
Collaborator

oh yeah, there's a way to detect the scenario in which the problem is likely to occur. one could even auto-rename folders without asking if you're ok with that level of intrusion (might make sense for an application like livereload, but not a generic library like rb-fsevent). there is not, however, a way to simply fix the issue wholesale.

Aaron Gibralter

Well I think renaming is a bit too intrusive... but couldn't rb-fsevent just watch out for that situation, print a warning, and then monitor the corrected directory?

Travis Tilley
Collaborator

@andreyvit - the final response from apple is that this isn't a bug because the fsevents table is case sensitive. i think the point of the bug report, being that path resolution apis weren't necessarily returning the correct case for a path, was completely ignored (so client and fseventsd can still have different paths). re-submitting...

in the meantime, i added file reference url resolution as the default 10.6+ strategy. no idea if that'll be more consistent. sigh.
ttilley/fsevent_watch@e249e0f#L1R49

Travis Tilley
Collaborator

The equivalence of what is happening is:
1. fsevents_tool -files ./test
2. mv ./test ./bar
3. echo "" > bar/test

Nobody should or would expect events to arrive on test/test.

The user workaround is as follows:

  1. Do not rename watched paths. (mv /User/foo/bar /Users/foo/BAR)
  2. Watch the parent directory instead of the directory you wish to rename.

/me slams head into desk repeatedly

Andrey Tarantsov
Aaron Gibralter

Would you still like to keep my directory in science experiment mode? Or can I move things around? :)

Andrey Tarantsov

Well, personally, I would be super-happy if you could download LiveReload 2 app (http://livereload.com/), add your directory there and see if it triggers the error. If you choose to do so, you can report the result to support@livereload.com to avoid spamming people here.

Other than that, there hasn't been any development on this topic, so I guess no — feel free to delete/move it. Thanks!

Travis Tilley
Collaborator

@agibralter - my bug report has stalled. 100%. after clarifying the issue, any questions about the status of the bug go ignored. the utility they mention in their bug reproduction example (for the incorrect bug) doesn't actually exist, at least publicly. sending the raw contents of /.fseventsd/ would be a potential risk for you, and painful for us, as it would include binary representations of all events, coalesced to a degree. I don't know of an easy way to look at the raw HFS+ inode data alongside fork data to determine potential on-disk issues.

The only thing I can think of is something that using the underlying internal-only fsevent device API that fseventsd itself uses, and censoring that yourself to only include data you want to share. Something like: http://www.osxbook.com/software/fslogger/

As far as I can guess, there should STILL be an fsevent notification somewhere... it's just incorrect in regards to case. If you can include ls output of the directory, stat -x of the directory, and a snippet of output from something like fslogger while editing files under the broken directory, that might be enough info to push forward the bug report with apple.

Travis Tilley
Collaborator

...not that it'd particularly help us much, but i'd really like to see this fixed in macos.

Travis Tilley
Collaborator

First bug report was March 30, 2011. Last bug report was October 25, 2011.

The majority of reporters had a problem with a path underneath a dropbox sync point. Dropbox doesn't have a UI for performing updates. Growl 1.3 was released on the mac app store around November 9, breaking dropbox notification support. An updated version of dropbox was available not long after, but that may have been the first time a lot of people had ever updated. It's a bit suspicious to me that bug reports stopped coming in right when a bunch of people had a reason to update dropbox... which doesn't use the public fsevents api, instead opting to use the private api.

Given the details and timeframe I'm happy to consider this bug unresolvable and dead. Unresolvable because it's looking likely that the issue isn't with macos or my own software, and dead because I haven't had any recent reports and I'd really rather stab myself in the face than spend more time on this.

...Since I was desperately waiting to resolve this issue before making another release, expect one in the hopefully not too distant future that also resolves the issue with installing without xcode.

Travis Tilley ttilley closed this November 19, 2011
Andrey Tarantsov

Hehe, I've recently had two users encounter this bug (for whom LiveReload has detected it and referred them to me). Both on Dropbox. Of course, I have no idea if they have updated Dropbox recently or not. (Does it even require manual updating? I don't think it ever asked me about updates.)

Michael Stalker

I know this is closed, but I recently ran into this issue in my project folder. I have Dropbox installed, but my project folder is not in Drobox. I renamed my project folder from "sites" to "Sites" and everything started to work fine. It still works after renaming it back to "sites".

Travis Tilley
Collaborator

@mstalker - frustrating, isn't it? i originally thought this was completely unrelated to dropbox because the problem wasn't always under the dropbox folder, but it does appear that dropbox (specifically /Library/DropboxHelperTools/Dropbox_VERSION/dbfseventsd) is still involved in triggering the state where this problem occurs. I'd be lying if I said I fully understood the scenario, but upgrading dropbox has improved the situation dramatically. If you don't mind, can you look and see what version is being run? Locally I have u501.

I recently discovered that dropbox for linux can also cause problems on that platform. How it manages to do that, I have no idea. Compass.app has a bug report that FS notifications (via inotify) simply don't work on ubuntu under the dropbox folder.

Dheeraj Kumar

I just faced this issue (LiveReload warned me) and I don't even have Dropbox installed :|

I don't use it, and have never on my mac (latest Lion)

What do you say about that?

Travis Tilley
Collaborator
Dheeraj Kumar

Sure, you can inspect it. Is it okay to move the files and just keep the empty folder? I'll contact you on IRC.

Tom Locke

Wow, reading this thread is painful. I do hope ttilley hasn't stabbed himself in the face yet.

Another poor soul having this problem (Lion). Renaming /Users to /users fixed it, yesterday. Today it is broken again.

Anything I can do to help? I've compiled and run find-fsevents-bugs, but not sure how much light that can shed.

Oh! Update: Renamed every folder in my path to all lower and things seem to be working again. Who needs caps anyway?

Andrey Tarantsov

@tablatom The case does not matter by itself, it's the act of renaming that (sometimes) fixes the problem, so you probably want to rename everything back (esp /Users)

By the way, here is a list of steps that, so far, has always helped everyone with this problem. (LiveReload can detect the specific offensive folder; feel free to ask me for a free version if you encounter the problem.)

Andrey Tarantsov

@ttilley BTW, it's about time we create a Wiki page about this. Forcing people to find this obscure (and closed!) ticket, and then to read one year's worth of comment is unnecessarily cruel. Feel free to copy text from my support page and/or to link to it.

Tom Locke
Karl-Petter Åkesson

I just started using Guard but never got it to react to file changes. After several hours of research I found out about this issue and it seemed to be what I was experiencing. Followed the thread for a bit, checking that I had no_log file, no system.log entries etc. Then I saw that people had solved it by renaming folders and scrolled down to the end of it.

So after checking my paths I saw that when starting Guard it reports Guard is now watching at '/users/kalle/coding/rails/community/trunk' but my actual path is Guard is now watching at '/Users/kalle/coding/rails/community/trunk', i.e. the Users folder was reported with a downcased U.

I started thinking if I could have detected this more easily because reading through the thread I started trying to rename different folders. I looked back into my shell history and I saw that I had made pwd before I fixed the issue and it reported the correct path!(I think I did this before the fix, not 100% sure). So to make it easier for anyone stumbling into this again and just want to make a quick test I wrote this short ruby script that I think shall detect it and it will report where it sees a difference.

current_dir_ruby = Dir.pwd
current_dir_system = `pwd`.strip # strip to remove any carriage returns at the end
if(current_dir_ruby != current_dir_system)
  puts "*******************************************************************************"
  puts "Seems you are affected by the bug odd Mac OS bug described here"
  puts "https://github.com/thibaudgg/rb-fsevent/issues/10"
  ruby_paths = current_dir_ruby.split("/")
  system_paths = current_dir_system.split("/")
  path = ""
  system_paths.zip(ruby_paths).each do |sp, rp|
    path = path<<sp<<"/"
    if(rp!=sp)
      puts "Ruby gives wrong folder name for #{path}"
      puts "Rename that folder will probably fix your problems"
      puts ""
    end
  end
  puts "*******************************************************************************"
end

Hope it can help anyone!

Btw, wouldn't it make sense to open this issue again? Even if its not an issue with rb-fsevent it is very likely users will stumble into it again.

Thibaud Guillaume-Gentil
Owner

@karl-petter I think it's more something that could be handle by Listen.
What do you think of that @ttilley @Maher4Ever ?

Karl-Petter Åkesson

@thibaudgg I do not really understand what you mean? As I have understood it this is an OSX bug and has nothing to do either with Listen or rb-fsevent but users of rb-fsevent are of course affected. Doesnt it affect Listen too?

The solution so far seems to be to rename the affected directory but the problem is to find which one. So I just wrote that little script to more easily see that(if I'm correct that pwd reports the correct path while Dir.pwd the faulty one) Since I fixed my problem before writing the script I have not been able to test it:(

Andrey Tarantsov

@ttilley @thibaudgg Let me ask it this way: why doesn't rb-fsevent have code to detect the problem and point people to a URL with an explanation? (Does Listen have it?)

Andrey Tarantsov

Also note that if you run LiveReload 2 and add an affected (sub)folder to it, LR will tell you the exact name of the buggy folder. I think I already said as much in this thread. Beta versions are free to run; there's no downside (like a trial period or whatever), and you can simply trash the app afterwards.

The relevant code is in OldFSTree.m and Project.m, btw, feel free to reuse it under the MIT license.

Travis Tilley
Collaborator
Karl-Petter Åkesson

I was again affected by this bug and got the opportunity to test my script, it works. So its a good alternative if people do not want to install a piece of software to see if they are affected or not. I'll see if I can figure out what is causing this, right now I suspect I get affected each time I restart my machine, which is not very often though. Guard worked fine yesterday but not this morning and I made a shutdown of the laptop when I left work yesterday.

pvdev
pvdev commented June 12, 2012

Wow. I searched all over the place for issues with directories magically changing to all caps and nothing. Then while looking to see why I should have installed rb-fsevent with guard, I come across this thread. Many hours have been spent trying to figure out why a guard 'watch' regex would suddenly stop working. I finally saw the reason with the Dir.pwd in a rib session. Thought it was because I was using SugarSync on OSX, but I see dropbox user's have seen it also. Tried moving the whole project directory to normal disk space e.g., /tmp, but still had the all caps in check with irb. Interestingly, it only breaks guard and running rspec manually continues to work. With me it first happen at the root of my SugarSync branch, but since I've seen it in project level directories. I've also notice when the problem is occurring I get the same Dir.pwd output with the rvm/ruby or system ruby.

Enough of a rant.
--Perry

Andrey Tarantsov

I wish I could be at WWDC right now and point some OS folks to this thread. Maybe next year.

If anyone can reliably reproduce the problem or has ideas on how to reproduce it, please speak up, by the way.

(Funnily, though, I haven't heard about this bug from LiveReload users lately. I used to get a couple of reports per month, so I have an impression that it's getting better, for an unknown reason.)

molfar
molfar commented July 01, 2013

Is there any result of this issue? How can be it fixed? I have this bug now.

molfar
molfar commented July 01, 2013

I have solved path case problem in my path. First of you need to check difference between ruby's Dir.pwd and bash pwd output in project's dir. In my case, I had difference:
Dir.pwd => /User/Myname/Documents/project
pwd => /User/myname/Documents/project
In this case the problem is in Myname dir's name, not in project's dir name. Renaming project dir has no effect. But renaming my user's folder to Myname and back to myname - has effect.
So, I think readme should bu updated, to not only trying renaming project's dir, but at first comparing output of these commands and finding the problematic dirname.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.