New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve docs on watchers, zero-duration events and "flooding" #124

Open
ErikBjare opened this Issue Oct 25, 2017 · 4 comments

Comments

Projects
None yet
2 participants
@ErikBjare
Copy link
Member

ErikBjare commented Oct 25, 2017

Me and @johan-bjareholt just had a conversation about this, I think it's something worthwhile writing good docs about since it's a source of important details about how to handle uncertainty in data.

Zero-duration events

When watchers do polling there can be uncertainty about what happened in between polls. An example is when the window watcher first sees a terminal window, and then 5 seconds later on the second poll sees a browser window. How long was the terminal window active? 5 seconds or shorter?

This leads to an imprecision that is inherent in most forms of polling (but notably not in aw-watcher-afk since it knows how long it was since the last input activity occurred) which we deal with by not making assumptions in the collection stage. The assumptions are postponed to the analysis stage where we apply heuristics like flooding to get cleaner (but less exact) data.

Flooding

Flooding is a technique to fill the empty time interval that occur around zero-duration events.

An example of a flooding rule can be something like this:

  • If a zero-duration event A is followed by a zero-duration event B then A gets its duration extended until the start of B.
  • Else if a zero-duration event A is followed by a longer event C then the longer-duration event gets extended backward until the end of A.
@ErikBjare

This comment has been minimized.

Copy link
Member

ErikBjare commented Oct 30, 2017

I implemented a simple flooding algorithm here: ActivityWatch/aw-research#4

@ErikBjare ErikBjare added the backlog label Oct 31, 2017

@ErikBjare ErikBjare modified the milestones: v0.8, v0.9 Feb 13, 2018

@ErikBjare

This comment has been minimized.

Copy link
Member

ErikBjare commented May 28, 2018

Moved flood into aw_transforms and added it as function in query2: ActivityWatch/aw-core#58

@ErikBjare

This comment has been minimized.

Copy link
Member

ErikBjare commented May 28, 2018

Here's an example query of all the ActivityWatch-related activity on my desktop, with and without flooding.

image

As can be seen it makes quite the difference.

@ErikBjare

This comment has been minimized.

Copy link
Member

ErikBjare commented May 28, 2018

Moved the flooding call earlier in the query (before filter_period_intersect, right after the query of the window events), causing a slightly larger difference.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment