IO::Notifications for changes to file contents (2019.03 vs 2019.07.1)

[bduggan]

The Problem

.IO.watch and watch-path do not provide notifications when there is a change to the contents of a file -- this is a change between 2019.03 and 2019.07.1.

Expected Behavior

The supply $file.IO.watch will emit IO::Notification objects when the contents of the file changes.

Actual Behavior

It doesn't

Steps to Reproduce

Run this:

(while true; do date; sleep 1; done) > watchme &

Then run this:

react whenever 'watchme'.IO.watch -> $e {
  say "got an event: " ~ $e
}

Under 2019.03 we see:

got an event: IO::Notification::Change<140210566333952>
got an event: IO::Notification::Change<140210566334912>
got an event: IO::Notification::Change<140210566334960>

But under 2019.07.1, we see

(i.e. nothing)

Environment

• Operating system: OS/X 10.14

• Compiler version (perl6 -v): see above

[vrurg]

Not reproducible on macOS 10.14.6, rakudo 2019.07.1 and HEAD work equally well:

got an event: IO::Notification::Change<140566742368256>
got an event: IO::Notification::Change<140566742369216>
got an event: IO::Notification::Change<140566742369264>
got an event: IO::Notification::Change<140566742369312>

I remove BLOCKER for now. Ticket could be closed if no additional information provided.

[bduggan]

What information would be helpful? I'm still seeing this issue. (macOS 10.14 (18A391))
Also tail -f watchme works fine

[vrurg]

Ok, I've reproduced the problem. Due to different shell (I use fish) I rewrote the command line to a slightly different variant and that was the difference. I.e. things work fine for:

while true; date >>watchme ; sleep 1; end

but fail if:

while true; date; sleep 1; end >watchme

The breaking change really happened somewhere between 2019.03 and 2019.07.

[bduggan]

Yes -- I noticed this too -- adding one line at a time works.

[jonathanstowe]

I've checked with FreeBSD (which has on occasion shown similar regressions,) and that is fine @ his is Rakudo version 2019.07.1-296-g66e0561f9 built on MoarVM version 2019.07.1-187-gcadf3a793

Was there a libuv bump somewhere in there? If there was I'd start there.

[jnthn]

Was there a libuv bump somewhere in there?

I think MoarVM/MoarVM@a515819b6031f2 is the only one between 2019.03 and 2019.07.1.

[vrurg]

@bduggan I have opened a MoarVM ticket MoarVM/MoarVM#1178. To have better diagnosis @jnthn reasonably proposes trying a version of MoarVM prior to MoarVM/MoarVM@a515819. Unfortunately, I'm in the middle of a big task and won't have time for a while for this. Maybe you could give it a try and report back?

[bduggan]

So -- I downgraded to the commit right before that one, and rebuilt (also downgrading nqp and rakudo to versions such that the deps were satisfied) -- and can report that it did not fix the issue -- i.e. this version still failed :

This is Rakudo version 2019.03.1-410-ge0076423c built on MoarVM version 2019.05-26-gc63e7eb7a
implementing Perl 6.d.

The moarvm,nqp, and rakudo commits to which I downgraded were respectively -- MoarVM/MoarVM@c63e7eb , Raku/nqp@6372077 and e007642

Here's the output, of running 2019.03.1 (which works), and the above (which doesn't) --

$ cat runme.p6
react whenever 'watchme'.IO.watch -> $e {
  say "got an event: " ~ $e
}
$ perl6 -v
This is Rakudo version 2019.03.1 built on MoarVM version 2019.03
implementing Perl 6.d.
$ perl6 runme.p6
got an event: IO::Notification::Change<140480293651968>
got an event: IO::Notification::Change<140480293652928>
got an event: IO::Notification::Change<140480293652976>
^C
$ ~/p6-dev/rakudo/perl6 -v
This is Rakudo version 2019.03.1-410-ge0076423c built on MoarVM version 2019.05-26-gc63e7eb7a
implementing Perl 6.d.
$ ~/p6-dev/rakudo/perl6 ./runme.p6
^C
$

[vrurg]

@bduggan Thank you!

May I also ask you to report on MoarVM ticket?

[bduggan]

Sure -- done :-)

[vrurg]

The problem is in unmistakably tracked down to MoarVM/MoarVM@62b58164a

[ugexe]

Unmistakably? The issue is not present in 2019.03 released in June/July, and you’re linking to a commit made 7 months ago.

[vrurg]

@ugexe I provided full methodology in MoarVM ticket. And yes, I'm 100% positive that it was the commit.

[vrurg]

Ah, I see why: the NQP bump which brought in that commit happened on Mar 16.

[vrurg]

@ugexe with git checkout 2019.03:

⇒ cd nqp/MoarVM/3rdparty/libuv/ && git log | head -10                                                     23-09-19 22:33:01
commit 7ebb26225f2eaae6db22f4ef34ce76fa16ff89ec
Author: cjihrig <cjihrig@gmail.com>
Date:   Thu Aug 16 22:05:34 2018 -0400

    2018.08.18, Version 1.23.0 (Stable)

[ugexe]

Ah then indeed there is no reason to doubt it is libuv 1.26

…

On Mon, Sep 23, 2019 at 9:33 PM Vadim Belman ***@***.***> wrote: @ugexe <https://github.com/ugexe> with git checkout 2019.03: ⇒ cd nqp/MoarVM/3rdparty/libuv/ && git log | head -10 23-09-19 22:33:01 commit 7ebb26225f2eaae6db22f4ef34ce76fa16ff89ec Author: cjihrig ***@***.***> Date: Thu Aug 16 22:05:34 2018 -0400 2018.08.18, Version 1.23.0 (Stable) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3100?email_source=notifications&email_token=AAB2RBETI6KZAPARNY2JDADQLF37DA5CNFSM4IJMSRI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7M3O3A#issuecomment-534361964>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAB2RBG4NK3VDJUCJABRMW3QLF37DANCNFSM4IJMSRIQ> .

[vrurg]

Since the problem has been tracked down to libuv changes and resulted in libuv/libuv#2488, I remove blocker from this ticket because it's nothing to do with changes in Rakudo. Moreover, people from libuv concluded that the problem stems even deeper down the stack and it's a bug of Darwin's implementation of kernel events.

As long as it's out of our control there is no reason for the blocking status.

[jnthn]

As long as it's out of our control there is no reason for the blocking status.

Arguably, we could consider reverting the libuv version bump, however.

[vrurg]

There're dependencies on later versions of libuv. uname, in particular. I'm afraid, revert would require a chain of other reverts.

[jnthn]

@vrurg Ah, then than argues more for waiting, given it seems there's PRs already submitted in libuv. Thanks for golfing it down to report, by the way.

[vrurg]

libuv/libuv@97b85e8 workarounds the bug. Waiting for next libuv release to be included into MoarVM.

[vrurg]

Closing via MoarVM/MoarVM@0f05efb

Thanks @bduggan!

[bduggan]

Great, thanks!