Detect stream correlations in a limited number of streams
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
1-generate
2-process
3-visualize
scripts
static
.eslintrc.json
.gitignore
README.md
initall.sh
package-lock.json
package.json
runall.sh

README.md

stream-correlation

Detect stream correlations in a limited number of streams

Thread correlated streams together and display in UI


What and Why

Slide Deck

slides.com/evandana/stream-correlation

Scenario: What Data is Correlated?

  • Given a continuous stream of data for multiple series
  • When I focus on one stream
  • Then I want to see which streams are correlated

While the data processing for determining correlated streams is a simple use of a correlation library, performing the full implementation of this solution for streaming data raises many important questions. These technology agnostic implementation considerations comprise the most meaningful and lasting benefit from this exercise.

Implementation Questions

  1. How much data should be analyzed to determine correlation? (Min and max data set sizes?)
  2. How does the data structure translate from initial series to multiple threaded series?
  3. How do you handle two series that are intermittently correlated?

Dev

Folder Structure

  1. Generate data from a data source
  2. Process data from the source
  3. Visualize the processed data

Install & Run

  1. npm run install Runs install scripts for all three sub-directories, each in a separate terminal
  2. npm run start Runs start scripts for each sub-directory, each in a separate terminal
  3. Use Chrome to navigate to http://127.0.0.1:8000

Preview

Preview Step 1 - Raw Data

Preview Step 2 - Correlations

Preview Step 3 - Threads

Basic Process Sketch

Stream Correlation Process Sketch