DA-992: Pair windows' delete + create event to generate a rename event #1

parzed · 2020-01-17T21:46:23Z

For each rename action on a file, two syscall event is generated:
1- FILE_ACTION_RENAMED_OLD_NAME and
2- FILE_ACTION_RENAMED_NEW_NAME.

Here I assumed that the first one is followed by the second one.
Then one fsnorify RENAME event is generated.
RENAME event has the file's oldName set up.

the OldName attribute is added to the Event struct of fsnotify.
The system call of FILE_ACTION_RENAMED_OLD_NAME holds the file's old name. Upon receiving this syscall event, watcher.rename is set up with the file's old name which serves as a state variable. No fsnotify event is created.

The system call of FILE_ACTION_RENAMED_NEW_NAME holds the file's new name. A RENAME fsnotify event is created with file's old name, watcher.rename, as OldName attribute .

Cases:
receive FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has a value, perfect case, a RENAME event is generated.

receive any syscall rather than FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has a value, first a RENAME event is created with empty name and the OldName is watcher.rename value, then the syscall event is handled.

receive FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has no value, a RENAME event is generated with an empty oldName.

Tests:
in imtegration_test.go

TestFsnotifyMultipleRenames():
generates 1000 rename action and checks if the RENAME attribute is set up correctly and all of the events are received.

TestFsnotifyRenameEventAttributes():
Checks if the OldName attribute is set up correctly for a rename event

hu13 · 2020-01-20T22:54:50Z

receive any syscall rather than FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has a value, first a RENAME event is created with empty name and the OldName is watcher.rename value, then the syscall event is handled.

receive FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has no value, a RENAME event is generated with an empty oldName.

In these imperfect cases, how would drive handle it?
how can these cases happen? on what frequency?

parzed · 2020-01-21T16:23:51Z

receive any syscall rather than FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has a value, first a RENAME event is created with empty name and the OldName is watcher.rename value, then the syscall event is handled.

receive FILE_ACTION_RENAMED_NEW_NAME and watcher.rename has no value, a RENAME event is generated with an empty oldName.
In these imperfect cases, how would drive handle it?
This is up to the server I think.
The Mac watcher (the one that has buffer), if it has a rename event of 'old name' or 'new name' and can not find its match in the buffer it creates an event based on the case. it creates a DELETE event for the old name case and CREATE event for the new name case. I think differentiating a faulty rename case from actual CREATE and DELETE is not a bad idea.
Anyways, in such a cases server can consider these approach or discarding the events.

how can these cases happen? on what frequency?
I have not seen any statistics for that and did not encounter the case in the tests. I can do a search to see in what cases it can happen. I considered them in case of system failures.

ianyh

Just stylistic comments I already had, but holding off on the rest of the review as we discussed.

fsnotify.go

windows.go

hu13

Some stylist comments.
I still want to discuss the main logic interactively since I don't fully understand some of it.

fsnotify.go

fsnotify_test.go

integration_test.go

windows.go

fsnotify_test.go

windows.go

integration_test.go

windows.go

ianyh · 2020-07-01T14:21:30Z

windows.go

@@ -468,47 +476,57 @@ func (w *Watcher) readEvents() {

 			name := syscall.UTF16ToString(buf)
 			fullname := filepath.Join(watch.path, name)
-
 			var mask uint64
 			switch raw.Action {
 			case syscall.FILE_ACTION_REMOVED:
 				mask = sysFSDELETESELF
 			case syscall.FILE_ACTION_MODIFIED:
 				mask = sysFSMODIFY
 			case syscall.FILE_ACTION_RENAMED_OLD_NAME:


It looks like in this case you don't set mask.

Oh, I guess this is just relying on the variable initializing by default to 0 so the ands do the right thing.

windows.go

ianyh · 2020-07-01T17:30:24Z

windows.go

+					if watch.rename != "" {
+						if w.sendEvent("", watch.names[name]&mask, watch.rename) {
+							if watch.names[name]&sysFSONESHOT != 0 {
+								delete(watch.names, name)


Do you need to return in this case? Otherwise we send two events.

* Let's begin * Add getpath * Init test workflow * 1.40.0 exits? * Linter fix * asdf * 100 Co-authored-by: hu13 <hangkun@preveil.com>

#1) * made the changes related to recursive directory check * made changes in window.go for buffer size * added the oldname attribute * old name added to rename event, one event is generated for rename * added oldname in printing rename events * tests for checking the oldName attr for rename added * create fsnotify event added * input to create event changed * create fsnotify event function modified * ID added * logs added * added create fsnotify event in inotify * logs added * prints added * reviews * reviews addressed * Hangkun/da 992/window rename event (#2) * Let's begin * Add getpath * Init test workflow * 1.40.0 exits? * Linter fix * asdf * 100 Co-authored-by: hu13 <hangkun@preveil.com> * Clean up unused * Badge Co-authored-by: Hangkun Ung <hangkun.ung@gmail.com> Co-authored-by: hu13 <hangkun@preveil.com>

* introduce GitHub Actions * Add lint+vet+old versions to GitHub Action * Remove Travis CI and references * Drop support/testing for Go 1.11 and earlier (fsnotify#381) * Update x/sys to latest (fsnotify#379) * add //go:build lines + add 1.17.0-rc2 to test matrix (fsnotify#377) * Update test matrix for go 1.17 stable release (fsnotify#385) * Add AddRaw to not follow symlinks + Fix link folloing on Windows (fsnotify#289) * v1.5.0 preparation (fsnotify#380) * revise pull request template * Revert "Add AddRaw to not follow symlinks + Fix link folloing on Windows (fsnotify#289)" This reverts commit e2e9517. * prepare 1.5.1, retract 1.5.0 * Removed dead link * Update issue templates (fsnotify#410) * Update issue templates * remove old issue template * Test on Go 1.18 and two most recent versions (fsnotify#411) * Test on Go 1.18 and two most recent versions * on push * ci * update readme * revise contributing * maintainers wanted * Final Notice: Maintainers Wanted * fix go vet warnings: call to (*T).Fatalf from a non-test goroutine (fsnotify#416) * made the changes related to recursive directory check * made changes in window.go for buffer size * DA-992: Pair windows' delete + create event to generate a rename event (#1) * made the changes related to recursive directory check * made changes in window.go for buffer size * added the oldname attribute * old name added to rename event, one event is generated for rename * added oldname in printing rename events * tests for checking the oldName attr for rename added * create fsnotify event added * input to create event changed * create fsnotify event function modified * ID added * logs added * added create fsnotify event in inotify * logs added * prints added * reviews * reviews addressed * Hangkun/da 992/window rename event (#2) * Let's begin * Add getpath * Init test workflow * 1.40.0 exits? * Linter fix * asdf * 100 Co-authored-by: hu13 <hangkun@preveil.com> * Clean up unused * Badge Co-authored-by: Hangkun Ung <hangkun.ung@gmail.com> Co-authored-by: hu13 <hangkun@preveil.com> * Rebase upstream fixes * Rename bundle only works on windows * Update CI golang Co-authored-by: Ichinose Shogo <shogo82148@gmail.com> Co-authored-by: Oliver Bristow <evilumbrella+github@gmail.com> Co-authored-by: Nahum Shalman <nahamu@gmail.com> Co-authored-by: Nathan Youngman <4566+nathany@users.noreply.github.com> Co-authored-by: Loïc Vernet <vernet.loic@gmail.com> Co-authored-by: Nathan Youngman <git@nathany.com> Co-authored-by: paris <pariya.b@gmail.com> Co-authored-by: hu13 <hangkun@preveil.com>

parzed added 6 commits November 7, 2019 14:41

made the changes related to recursive directory check

86cd899

made changes in window.go for buffer size

7019d65

added the oldname attribute

16adc20

old name added to rename event, one event is generated for rename

239aaee

added oldname in printing rename events

da31f03

tests for checking the oldName attr for rename added

645570a

parzed requested review from hu13 and ianyh January 17, 2020 21:47

ianyh requested changes Feb 11, 2020

View reviewed changes

fsnotify.go Outdated Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

windows.go Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

parzed added 3 commits February 20, 2020 16:05

create fsnotify event added

70af172

input to create event changed

7445823

create fsnotify event function modified

9e1651b

parzed force-pushed the da-992/window-rename-event branch from 6610dff to 9e1651b Compare February 21, 2020 00:53

parzed added 6 commits March 19, 2020 11:45

ID added

df4716e

ID added to fsnotify event

a6a6bff

logs added

230bef0

added create fsnotify event in inotify

80b5916

logs added

b36c8d5

prints added

c87c26a

ianyh changed the title ~~Da 992/window rename event~~ Window rename event Apr 8, 2020

ianyh self-requested a review April 8, 2020 19:16

ianyh changed the title ~~Window rename event~~ DA-992: Window rename event Apr 9, 2020

hu13 requested changes Apr 22, 2020

View reviewed changes

ianyh requested changes Jun 11, 2020

View reviewed changes

fsnotify_test.go Outdated Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

windows.go Outdated Show resolved Hide resolved

parzed added 2 commits June 12, 2020 12:31

reviews

582bcdb

reviews addressed

7a2e928

ianyh requested changes Jul 1, 2020

View reviewed changes

hu13 force-pushed the preveil branch from 7019d65 to 57f1a48 Compare May 9, 2021 02:09

hu13 changed the title ~~DA-992: Window rename event~~ DA-992: Pair windows' delete + create event to generate a rename event Jun 9, 2021

hu13 and others added 3 commits June 9, 2021 16:01

Hangkun/da 992/window rename event (#2)

d4127b0

* Let's begin * Add getpath * Init test workflow * 1.40.0 exits? * Linter fix * asdf * 100 Co-authored-by: hu13 <hangkun@preveil.com>

Merge branch 'preveil' into da-992/window-rename-event

bf267b8

Clean up unused

f3e5d13

hu13 requested a review from ianyh June 9, 2021 20:12

Badge

41977b8

hu13 merged commit 580c3c9 into preveil Jul 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DA-992: Pair windows' delete + create event to generate a rename event #1

DA-992: Pair windows' delete + create event to generate a rename event #1

parzed commented Jan 17, 2020 •

edited by jira bot

Loading

hu13 commented Jan 20, 2020

parzed commented Jan 21, 2020

ianyh left a comment

hu13 left a comment

ianyh Jul 1, 2020

ianyh Jul 1, 2020

ianyh Jul 1, 2020

DA-992: Pair windows' delete + create event to generate a rename event #1

DA-992: Pair windows' delete + create event to generate a rename event #1

Conversation

parzed commented Jan 17, 2020 • edited by jira bot Loading

hu13 commented Jan 20, 2020

parzed commented Jan 21, 2020

ianyh left a comment

Choose a reason for hiding this comment

hu13 left a comment

Choose a reason for hiding this comment

ianyh Jul 1, 2020

Choose a reason for hiding this comment

ianyh Jul 1, 2020

Choose a reason for hiding this comment

ianyh Jul 1, 2020

Choose a reason for hiding this comment

parzed commented Jan 17, 2020 •

edited by jira bot

Loading