Skip to content

Commit

Permalink
Merge pull request #21 from IamMaxim/glossary
Browse files Browse the repository at this point in the history 
Key terms glossary
  • Loading branch information
@alkaitagi
alkaitagi committed Mar 8, 2021
2 parents 31198be + 1f96ca8 commit 89a91d0
Show file tree
Hide file tree
Showing 2 changed files with 82 additions and 7 deletions.
34 changes: 27 additions & 7 deletions documents/cli.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ $ vpt check

This will record the data for 5 seconds and then create 3 files in your current working directory:

* `vpt-audio.wav`: a sample of audio recorded through your audio source
- `vpt-audio.wav`: a sample of audio recorded through your audio source

* `vpt-keyboard.txt`: a list of recorded keyboard actions
- `vpt-keyboard.txt`: a list of recorded keyboard actions

* `vpt-mouse.txt`: a list of mouse actions (moves & clicks)
- `vpt-mouse.txt`: a list of mouse actions (moves & clicks)

Feel free to use your keyboard and mouse during the 5 seconds of the check to ensure that the data from them is
correctly recorded.
Expand Down Expand Up @@ -56,24 +56,44 @@ Video source 2
To select an audio source for recording, use `--audio <NAME>` for specifying the audio source and `--video <NAME>` for
specifying the video source:

```
```sh
$ vpt --audio AudioSource2 --video "Video source 2"
```


## Obtaining raw data for analysis

If you wish to get a dataset of keyboard/mouse activity along with the predicted work state engagement levels from
audio/video, use the `dump` option:

```
```sh
$ vpt dump
```

This will create a `vpt-data.sql` SQLite database in the current working directory with the collected data.

To dump the data into a different file, use the `-o` parameter:

```
```sh
$ vpt dump -o for-analysis.sql
```

## Clearing old data

To clear the data collected by the program, use the `clear` option:

```sh
$ vpt clear
Warning! You are about to delete the data from 2021-01-01 13:37:00 to 2021-02-01 13:38:00.
Are you sure (y/N)?
```

Type `y` and hit `Enter` to confirm deletion.

If you only wish to delete a part of the data, you may constrain the range of deletion with `--start <TIME>`
and `--end <TIME>` options with a time point in the format `YYYY.MM.DD HH:MM:SS`:

```sh
$ vpt clear --start "2021-01-01 13:37:00" --end "2021-01-02 13:37:00"
Warning! You are about to delete the data from 2021-01-01 13:37:00 to 2021-01-02 13:37:00.
Are you sure (y/N)?
```
55 changes: 55 additions & 0 deletions documents/glossary.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Glossary

The keys terms used in the project.

## Conceptual terms

| Term | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------------------- |
| **Gaze code** | Representation of user's gaze, encoded as 3 states (workspace, elsewhere, absent) |
| **Voice code** | Representation of user's voice, encoded as 2 states (present, absent). |
| **Interaction state** | Estimation of user's activity at any given time based on their gaze and speech inputs. |
| **Engagement level** | Scale of user's involment into their working process. Consists of 5 levels mapping to the interaction states. |
| **User presence** | State of active engagement. Corresponds to the engagement levels 0-1. |
| **User absence** | State of lack of engagement. Corresponds to the engagement levels 2-4. |
| **Tracking sequence** | Present/absent timeline used for representing user's activities over a period of time. |
| **Storage format** | Data fields and their types used for storing recoding sequence. There are 2 such formats - for SQL and File stores. |

## Technical terms

### Base classes

| Term | Description |
| ------------------ | ------------------------------------------------------------------------ |
| **Source node** | Base class for data sources, e.g. `DeviceVideoSource`, `KeyboardSource` |
| **Processor node** | Base class for data processors, e.g. `SpeechDetector`, `MouseCompressor` |
| **Sink node** | Base class for data sinks, e.g. `SQLiteStore`, `GraphView` |

### Source nodes

| Term | Description |
| --------------------- | ------------------------------------------------------------------------ |
| **DeviceVideoSource** | Provides a video stream coming from a physical device (e.g. webcamera) |
| **MP4VideoSource** | Provides a video stream coming from an MP4 video file. |
| **DeviceAudioSource** | Provides an audio stream coming from a physical device (e.g. microphone) |
| **WAVAudioSource** | Provides an audio stream coming from a WAV audio file. |
| **KeyboardSource** | Provides a stream of keystrokes coming to the system. |
| **MouseSource** | Provides a stream of mouse events (move, click) coming to the system. |

### Processor nodes

| Term | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------ |
| **GazeDetector** | Converts a video stream to a stream of gaze codes. |
| **SpeechDetector** | Detects speech in an audio stream and outputs a signal stream for that. |
| **EngagementEstimator** | Converts the gaze code and the speech presence signal into a prediction of the engagement level. |
| **MouseCompressor** | Compresses a stream of mouse events to reduce redundancy. |

### Sink nodes

| Term | Description |
| ---------------- | ---------------------------------------------------------------------------------------- |
| **VideoDisplay** | Displays a video stream in a window. |
| **SQLiteStore** | Persistently store the engagement level and keyboard/mouse events in an SQLite database. |
| **FileStore** | Persistently store the audio and keyboard/mouse events in the filesystem. |
| **GraphView** | Displays an ongoing graph of presence of the user. |

0 comments on commit 89a91d0

Please sign in to comment.