New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for file watching `~` #7

Closed
jvican opened this Issue Nov 11, 2017 · 13 comments

Comments

Projects
None yet
6 participants
@jvican
Member

jvican commented Nov 11, 2017

We should probably add a really fast source file watching for ~compile and ~test. We may want to add this directly to Zinc rather than experiment with it directly in blossom. I prefer this approach since it's more generic.

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Nov 11, 2017

Member

I would go with something native for every OS like https://facebook.github.io/watchman/

Member

olafurpg commented Nov 11, 2017

I would go with something native for every OS like https://facebook.github.io/watchman/

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Nov 17, 2017

Member

It has just occurred to me that source file watching should be supported by our tool with a cross-platform solution, while still allowing users to run a native solution that executes a compile command with bloop.

Member

jvican commented Nov 17, 2017

It has just occurred to me that source file watching should be supported by our tool with a cross-platform solution, while still allowing users to run a native solution that executes a compile command with bloop.

@Duhemm Duhemm added the feature label Nov 21, 2017

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Nov 22, 2017

Member

I've been looking into watchman and use it locally. I've liked a lot how it works. The fact that it works on all OSes makes me lean on it for now, for our first prototype.

Member

jvican commented Nov 22, 2017

I've been looking into watchman and use it locally. I've liked a lot how it works. The fact that it works on all OSes makes me lean on it for now, for our first prototype.

@jvican jvican added this to the Bloop 0.1 milestone Nov 23, 2017

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Nov 27, 2017

Member

I think support for file watching inside bloop should come in Bloop 0.2. For now, we should just have high-quality docs on how to use watchman with bloop + nailgun. What do you think @Duhemm? If you agree, I propose we move this to milestone 0.2.

Member

jvican commented Nov 27, 2017

I think support for file watching inside bloop should come in Bloop 0.2. For now, we should just have high-quality docs on how to use watchman with bloop + nailgun. What do you think @Duhemm? If you agree, I propose we move this to milestone 0.2.

@olafurpg

This comment has been minimized.

Show comment
Hide comment
@olafurpg

olafurpg Nov 27, 2017

Member

I agree this can be postponed , esp. since it's possible to use watchman/entr with bloop + nailgun. I'm also eager to explore integrating lsp with bloop to run compile/test on file save. In that case the editor provides the event hook.

Member

olafurpg commented Nov 27, 2017

I agree this can be postponed , esp. since it's possible to use watchman/entr with bloop + nailgun. I'm also eager to explore integrating lsp with bloop to run compile/test on file save. In that case the editor provides the event hook.

@Duhemm

This comment has been minimized.

Show comment
Hide comment
@Duhemm

Duhemm Nov 27, 2017

Member

That sounds very reasonable 👍

Member

Duhemm commented Nov 27, 2017

That sounds very reasonable 👍

@Duhemm Duhemm modified the milestones: Bloop 0.1, Bloop 0.2 Nov 27, 2017

@jvican jvican self-assigned this Nov 29, 2017

@Jasper-M

This comment has been minimized.

Show comment
Hide comment
@Jasper-M

Jasper-M Dec 1, 2017

Is there something inherently wrong with just using java.nio.file.WatchService for this? Not that it has the most beautiful interface...

Jasper-M commented Dec 1, 2017

Is there something inherently wrong with just using java.nio.file.WatchService for this? Not that it has the most beautiful interface...

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Dec 1, 2017

Member

We're using directory-watcher because it claims speedups in OSX and does not do polling. I have a prototype with it I'll PR any time soon.

Member

jvican commented Dec 1, 2017

We're using directory-watcher because it claims speedups in OSX and does not do polling. I have a prototype with it I'll PR any time soon.

@fsvehla

This comment has been minimized.

Show comment
Hide comment
@fsvehla

fsvehla Dec 1, 2017

Is there something inherently wrong with just using java.nio.file.WatchService for this? Not that it has the most beautiful interface...

It uses polling under macOS and Linux, which leads to for example SBT using 40% of CPU constantly watching a medium size code base.

fsvehla commented Dec 1, 2017

Is there something inherently wrong with just using java.nio.file.WatchService for this? Not that it has the most beautiful interface...

It uses polling under macOS and Linux, which leads to for example SBT using 40% of CPU constantly watching a medium size code base.

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Dec 1, 2017

Member

Does it under Linux? I think under linux it uses fsevents.

Member

jvican commented Dec 1, 2017

Does it under Linux? I think under linux it uses fsevents.

@fsvehla

This comment has been minimized.

Show comment
Hide comment
@fsvehla

fsvehla Dec 1, 2017

My information re: Linux might be outdated, I am not using it for development.

fsvehla commented Dec 1, 2017

My information re: Linux might be outdated, I am not using it for development.

@gmethvin

This comment has been minimized.

Show comment
Hide comment
@gmethvin

gmethvin Dec 1, 2017

The OS differences are basically:

  • Windows JDK WatchService does async watching and recursive watching (ExtendedWatchEventModifier.FILE_TREE).
  • Linux JDK WatchService does async watching but not recursive watching. In practice you can handle the recursive case you can register a new WatchKey for directories when they are created.
  • macOS JDK WatchService uses polling and at best can be configured to poll every two seconds. To get a decent experience you need to use a separate native program or JNI/JNA. directory-watcher provides/uses a JNA implementation that also supports recursive watching.

I think it makes sense to use a higher-level API, perhaps something like File.Monitor from better-files. (directory-watcher already implements that: https://github.com/gmethvin/directory-watcher/#better-files-integration-scala). Besides the fact that the WatchService API is not very nice, it's also useful to customize the recursive aspect so it uses the most efficient mechanism for each OS.

gmethvin commented Dec 1, 2017

The OS differences are basically:

  • Windows JDK WatchService does async watching and recursive watching (ExtendedWatchEventModifier.FILE_TREE).
  • Linux JDK WatchService does async watching but not recursive watching. In practice you can handle the recursive case you can register a new WatchKey for directories when they are created.
  • macOS JDK WatchService uses polling and at best can be configured to poll every two seconds. To get a decent experience you need to use a separate native program or JNI/JNA. directory-watcher provides/uses a JNA implementation that also supports recursive watching.

I think it makes sense to use a higher-level API, perhaps something like File.Monitor from better-files. (directory-watcher already implements that: https://github.com/gmethvin/directory-watcher/#better-files-integration-scala). Besides the fact that the WatchService API is not very nice, it's also useful to customize the recursive aspect so it uses the most efficient mechanism for each OS.

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Dec 11, 2017

Member

#125 successfully implements this feature thanks to your work, great library 👍 @gmethvin. In the next days we'll experiment with file watching in bloop in larger projects and we'll see how it fares in comparison with the default file watching algorithms in sbt/sbt.

Member

jvican commented Dec 11, 2017

#125 successfully implements this feature thanks to your work, great library 👍 @gmethvin. In the next days we'll experiment with file watching in bloop in larger projects and we'll see how it fares in comparison with the default file watching algorithms in sbt/sbt.

@jvican jvican closed this Dec 13, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment