Standardize EEG Data Model for Streaming #1

alexcastillo · 2017-06-17T04:25:37Z

Hi everyone,

I've created this repo with the purpose of discussing a data model standard for streaming EEG.
Please review the README.md file and feel free to submit PRs.

cc @aj-ptw @marionleborgne @urish @teonbrooks @andrewheusser

Best,

Alex

andrewjaykeller · 2017-06-17T05:01:59Z

There is a lot of redundancy with this model. The only things that should be in a sample are dynamic variables i.e.:

{
  "buffer": [ // List of samples
    {
      "timestamp": Date, 
      "data": [ // List of channels
         float, 
         float, 
         float, 
         float
      ]
    }
  ]
}

Having to send these over the wire means we need to be lean as possible. I find the best results when i minimize the amount of redundant data between sample.

I think a header or start byte define what the metrics are:

{
  "metric": String // E.g. EEG, ACCL, etc (optional)
  "source": String // E.g. Muse, OpenBCI, etc (optional)
  "sampleRate": Number // E.g. 250. always in hz (optional)
  "mock": Boolean // E.g. false (optional),
  "topology": [ 'Pz', 'Oz', 'Cz', 'C1']
}

especially with hierarchal topics we can inject and split on these exact metrics.

marionleborgne · 2017-06-17T07:52:41Z

Hey guys. I'm in the mountains now. I will reply on Sunday with a more detailed answer but in short my recommendation for the data model would be to map it to the LSL model. 1. Send only one time the rich StreamInfo, serialized to JSON. This contains sampling rate, channel positions, channel data format etc. 2. Open a stream and the only data that is sent is Steam samples serialized to JSON. In LSL parlance it's : `{timestamp :..., data : [..., ...]}` For efficiency concerns when transmitting over the network we can bundle the samples in chunks and so the JSON transmitted is: ``` [ {timestamp :..., data : [..., ...]}, ..., {timestamp :..., data : [..., ...]} ] ``` Add the quotes to make it real json (I'm writing from my phone) but that's the gist of it. I can add more details when I'm back. I don't think we should send anything related to the stream info (like electrode positions, sampling rate etc) with the stream samples. It's just inefficient. The stream info should be query-able on demand.

…

On Fri, Jun 16, 2017, 10:02 PM AJ Keller ***@***.***> wrote: There is a lot of redundancy with this model. The only things that should be in a sample are dynamic variables i.e.: { "buffer": [ // List of samples { "timestamp": Date, "data": [ // List of channels { "id": String // E.g. FP1, FP2, F3, F4, C3, C4, etc (optional) "value": Number }, ... ] } ] } Having to send these over the wire means we need to be lean as possible. I find the best results when i minimize the amount of redundant data between sample. I think a header or start byte define what the metrics are: { "metric": String // E.g. EEG, ACCL, etc (optional) "source": String // E.g. Muse, OpenBCI, etc (optional) "sampleRate": Number // E.g. 250. always in hz (optional) "mock": Boolean // E.g. false (optional) } especially with hierarchal topics we can inject and split on these exact metrics. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACX3uy72L3IFgrIY7iCi5pBEs81_H0mBks5sE13HgaJpZM4N9H41> .

marionleborgne · 2017-06-17T07:56:02Z

Basically what AJ says. Except the streaminfo (what he calls the headers) should respect the LSL convention (really, it's the XDF format they're using)

alexandrebarachant · 2017-06-17T16:52:13Z

Out of curiosity, why not using LSL ? There is no js implementation but I might be easier to make one rather than creating a new standard from scratch.

alexcastillo · 2017-06-17T18:03:43Z

Thanks everyone for the feedback!

Sounds like we want to go leaner and use LSL.
I'm not familiar with LSL but definitely don't want to reinvent the wheel.

@alexandrebarachant can you point me to an LSL implementation in other languages? Maybe a Python implementation.

Maybe we can port it to JavaScript to make the communication between server/client more efficient.

Best,

Alex

alexcastillo · 2017-06-17T18:08:53Z

Have anyone used this one?

https://github.com/sccn/labstreaminglayer/tree/master/LSL

alexandrebarachant · 2017-06-17T18:16:15Z

That's the one. I'm using it extensively.

Now, the typical usecase is streaming on a local network, so that is why I'm asking the question.

marionleborgne · 2017-06-17T19:24:02Z

I've extended LSL to work outside of a lab network. It's in a private repo right now. I will try to release it next week.

…

On Sat, Jun 17, 2017, 11:16 AM alexandre barachant ***@***.***> wrote: That's the one. I'm using it extensively. Now, the topical usecase is streaming on a local network, so that is why I'm asking the question. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACX3u8GxU-9dakYi55EClqynikHfzZAVks5sFBfvgaJpZM4N9H41> .

urish · 2017-06-17T22:16:11Z

I went ahead and created a small node program to stream Muse 2016 data to LSL:

https://github.com/urish/muse-lsl

Basically, it leverages muse-js over noble (BLE bindings for Node.js) using bleat (emulates Web Bluetooth for muse-js), and then uses ffi to interface with liblsl.

So now I got the data streaming to LSL (I can see it in their Lab Recorder program).

Do you have any good pointers for LSL compatible software that I can use to visualize the EEG data, etc.?

alexcastillo · 2017-06-17T22:31:01Z

This is awesome, Uri!

Could we separate the lsl part into its own lib that can accept an EEG Observable?
That way we could use it for the Muse, OpenBCI, etc.

And of course, let me know how I can help.

andrewjaykeller · 2017-06-17T22:33:07Z

How do I stream to LSL from OpenBCI shield? Arduino C/C++ compatible.

urish · 2017-06-17T22:53:26Z

thank Alex!

Basically, the LSL wrapper sits in https://github.com/urish/muse-lsl/blob/master/lsl.js, so it can be quite easily separated. I don't have much experience with LSL though - I first heard about it today :)

alexandrebarachant · 2017-06-17T23:02:16Z

I also have a muse lsl repo : https://github.com/alexandrebarachant/muse-lsl It's in Python. There is a lsl viewer to plot live EEG signal. There is also something to record P300 data. More detail here : http://alexandre.barachant.org/blog/2017/02/05/P300-with-muse.html Le 18 juin 2017 00:53, "Uri Shaked" <notifications@github.com> a écrit :

…

thank Alex! Basically, the LSL wrapper sits in https://github.com/urish/muse- lsl/blob/master/lsl.js, so it can be quite easily separated. I don't have much experience with LSL though - I first heard about it today :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACC9thhbny9hXW2ieMZ5uw49GF8qVMqRks5sFFjmgaJpZM4N9H41> .

urish · 2017-06-18T12:40:49Z

Thank @alexandrebarachant , going to try to feed my muse-lsl into the viewer in yours!

What's the best way to communicate with you? Got some questions regarding how to analyze muse data, seems like you have some practical experience with this

alexandrebarachant · 2017-06-18T12:57:42Z

@urish you can contact me by email. I'm also hanging around in the NeurotechX slack

urish · 2017-06-18T13:21:50Z

cool, I just joined it! Ping you there

teonbrooks · 2017-06-18T22:00:49Z

hey everyone, I'm just now catching up. I'm in Cape Town, which is GMT+2. I'm flying tomorrow evening and landing Tuesday afternoon. I will also poke my head into the slack. So far, I like what I am seeing :)

urish · 2017-06-18T22:18:21Z

Quick update: node js implementation of muse-lsl can now stream values successfully, tested with lsl-viewer. Thanks for @alexandrebarachant a few bugs were fixed :-)

marionleborgne · 2017-06-19T17:48:21Z

Hi everyone,

This is the data structure of the cloudbrain pub/sub lib. It extends LSL beyond the lab network but is still using the LSL data structure for ease of use in other applications.

LSL refresher

In the LSL model, you have 3 main objects:

StreamInfo to describe the structure of the samples. (The stream info can be used for XDF file headers)
StreamInlet to consume samples.
StreamOutlet to publish samples.

I the following section, I explain the data model that we have been using in Cloudbrain to extend LSL beyond the lab network. It's basically the same as LSL, except that we serialize objects to JSON so that we can have a uniform data contract between distributed applications.

Data model

You have two main objects that are being passed:

Samples: they contain time-stamped channel values. Used by StreamOutlet/ StreamInlet to send/receive data.
StreamInfo: Describes the sample with attributes like sampling_rate, channel_count
, etc..

These 2 objects (Samples and StreamInfo) are being serialized to JSON strings to be passed between different applications.

Samples

StreamInlets and StreamOutlets send and receive samples. A sample has the following data format: {"data": [<float>, ..., <float>], "timestamp": <float> }

You can send and receive chunks of samples. In this case:

{
 "chunk": [
    {"data": [<float>, ..., <float>],  "timestamp": <float> },
    ...
    {"data": [<float>, ..., <float>],  "timestamp": <float> }
  ]
}

StreamInfo

In LSL, StreamInfo can be serialized to XML. In the cloudbrain lib that extends LSL, I added the ability to serialize StreamInfo to JSON since this is how other types of data are being serialized and shared between distributed apps.

{
  "info": {
    "name": <string>,
    "type": <string>,
    "channel_count": <int>,
    "nominal_srate": <float>,
    "channel_format": <string>,
    "source_id": <string>,
    "version": <string>,
    "created_at": <timestamp>,
    "uid": <string>,
    "session_id": <string>,
    "hostname": <string>,
    "v4address": <string>,
    "v4data_port": <int>,
    "v4service_port": <int>,
    "v6address": <string>,
    "v6data_port": <string>,
    "v6service_port": <int>,
    "desc": {
      "channels": {
        "channel": [
          {
            "label": <string>,
            "unit": <string>,
            "type": <string>
          },
           ...
          {
            "label": <string>,
            "unit": <string>,
            "type": <string>
          }
        ]
      },
      "cap": {
        "name": <string>,
        "size": <string>
        "labelscheme": <string>
      },
      "manufacturer": <string>
    }
  }
}

Examples

Buffer of 5 samples. Each sample has 2 channels:

{ 
  "chunk": [
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0}
  ]
}

StreamInfo

{
  "info": {
    "name": "muse",
    "type": "EEG",
    "channel_count": 8,
    "nominal_srate": 0,
    "channel_format": "float32",
    "source_id": "some_id",
    "version": "1.1",
    "created_at": "0",
    "uid": null,
    "session_id": null,
    "hostname": null,
    "v4address": null,
    "v4data_port": 0,
    "v4service_port": 0,
    "v6address": null,
    "v6data_port": 0,
    "v6service_port": 0,
    "desc": {
      "channels": {
        "channel": [
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          }
        ]
      },
      "cap": {
        "name": "undefined",
        "size": "undefined",
        "labelscheme": "undefined"
      },
      "manufacturer": "Interaxon"
    }
  }
}

alexcastillo · 2017-06-20T01:16:25Z

@marionleborgne
This is perfect, exactly what I was looking for. I'll create some charts that will take this data format as an input.

@urish What do you think? We could update our work to use this data format.

Thanks, everyone.

marionleborgne · 2017-06-20T03:30:04Z

@alexcastillo glad you like it. @aj-ptw is also using this data contract when pushing samples to cloudbrain from his wifi shield.

marionleborgne · 2017-06-20T03:31:04Z

@alexcastillo looking forward to the charts! 👍

urish · 2017-06-20T13:22:05Z

Thank you so much @marionleborgne !

I just updated muse-js to emit the sample data in this format, and muse-lsl + angular-muse to consume it.

@alexcastillo - also looking forward to the charts :)

andrewjaykeller · 2017-06-20T16:27:05Z

How am I supposed to stream from Arduino to LSL? Any ideas here?

alexandrebarachant · 2017-06-20T16:32:13Z

@aj-ptw : your best chance is to ask LSL people directly. there is a C implementation, but i have no idea if you can compile it in a Arduino library.

marionleborgne · 2017-06-20T16:39:47Z

@aj-ptw another option, since we are in the process of adding cloudbrain connectivity to the wifi shield, is to publish data over MQTT to cloudbrain and then just start the cloudbrain module that publishes data to LSL.

marionleborgne · 2017-06-20T16:40:55Z

Haha, first thing that comes up when googling this question :p sccn/labstreaminglayer#52 @aj-ptw beats me to it

marionleborgne · 2017-06-21T23:32:04Z

@alexcastillo @alexandrebarachant @urish @teonbrooks

Question from @aj-ptw and I: anyone has an objections to sending voltage values as nano volts. The channels format will be int.

See OpenBCI/OpenBCI_WIFI#13 (comment) for background.

alexcastillo · 2017-06-22T01:27:29Z

That's ok with me. Any chance we can add that detail to the StreamInfo?

…

On Wed, Jun 21, 2017 at 4:32 PM Marion Le Borgne ***@***.***> wrote: @alexcastillo <https://github.com/alexcastillo> @alexandrebarachant <https://github.com/alexandrebarachant> @urish <https://github.com/urish> @teonbrooks <https://github.com/teonbrooks> Question from @aj-ptw <https://github.com/aj-ptw> and I: anyone has an objections to sending voltage values as nano volts. The channels format will be int. See OpenBCI/OpenBCI_WIFI#13 (comment) <OpenBCI/OpenBCI_WIFI#13 (comment)> for background. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABIH4mNcXmtVGmGIliMtHvgzbiGzQ8yvks5sGaf1gaJpZM4N9H41> .

marionleborgne · 2017-06-22T02:00:10Z

Actually, it's already in the stream info ("desc" property). See JSON example.

…

alexcastillo · 2017-06-22T02:02:52Z

Perfect.

…

On Wed, Jun 21, 2017 at 7:00 PM Marion Le Borgne ***@***.***> wrote: Actually, it's already in the stream info ("desc" property). See JSON example. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABIH4mCppcCwDdjTJ42LEsRV0tb2A0Soks5sGcqqgaJpZM4N9H41> .

marionleborgne · 2017-06-23T00:11:54Z

@urish i just looked at your muse-lsl repo. It's awesome! Thanks 👍

marionleborgne · 2017-06-23T00:15:36Z

@urish it occurred to me that you could actually abstract / pull out the LSL part out of the muse-lsl repo and this would be a generic LSL lib for JS - doesn't have to be tied to muse. For example, OpenEXP (@teonbrooks ) can use it publish event markers to LSL. His app is in JS.

You just created the missing JS LSL lib 😄

alexcastillo · 2017-06-23T01:21:36Z

That's exactly what I told him :)

…

On Thu, Jun 22, 2017 at 5:15 PM Marion Le Borgne ***@***.***> wrote: @urish <https://github.com/urish> it occurred to me that you could actually abstract / pull out the LSL part out of the muse-lsl repo and this would be a generic LSL lib for JS - doesn't have to be tied to muse. For example, OpenEXP ***@***.*** <https://github.com/teonbrooks> ) can use it publish event markers to LSL. His app in is JS. You just created the missing JS LSL lib 😄 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABIH4qJ8Iydwkx0zBEMnfzC-vXzbsfJdks5sGwOpgaJpZM4N9H41> .

urish · 2017-06-26T15:09:14Z

here we go: https://www.npmjs.com/package/node-lsl

marionleborgne · 2017-06-26T18:08:33Z

This is great @urish. Thanks!

alexcastillo mentioned this issue Jun 18, 2017

liblsl wrapper for JavaScript sccn/lsl_archived#168

Open

marionleborgne mentioned this issue Jun 21, 2017

Send JSON over MQTT with Cloudbrain data format OpenBCI/OpenBCI_WIFI#13

Closed

marionleborgne mentioned this issue Jun 23, 2017

Ship LSL lib w/ repo? urish/muse-lsl#1

Closed

teonbrooks mentioned this issue Jun 23, 2017

callback function from screen presentations jspsych/jsPsych#402

Closed

jdpigeon mentioned this issue Jan 27, 2018

Proposed Data Structures neurosity/eeg-pipes#4

Closed

Standardize EEG Data Model for Streaming #1

Standardize EEG Data Model for Streaming #1

Comments

alexcastillo commented Jun 17, 2017

andrewjaykeller commented Jun 17, 2017 • edited Loading

marionleborgne commented Jun 17, 2017 via email • edited Loading

marionleborgne commented Jun 17, 2017

alexandrebarachant commented Jun 17, 2017

alexcastillo commented Jun 17, 2017

alexcastillo commented Jun 17, 2017

alexandrebarachant commented Jun 17, 2017 • edited Loading

marionleborgne commented Jun 17, 2017 via email

urish commented Jun 17, 2017

alexcastillo commented Jun 17, 2017 • edited Loading

andrewjaykeller commented Jun 17, 2017

urish commented Jun 17, 2017

alexandrebarachant commented Jun 17, 2017 via email

urish commented Jun 18, 2017

alexandrebarachant commented Jun 18, 2017

urish commented Jun 18, 2017

teonbrooks commented Jun 18, 2017

urish commented Jun 18, 2017

marionleborgne commented Jun 19, 2017 • edited Loading

LSL refresher

Data model

Samples

StreamInfo

Examples

alexcastillo commented Jun 20, 2017

marionleborgne commented Jun 20, 2017

marionleborgne commented Jun 20, 2017

urish commented Jun 20, 2017

andrewjaykeller commented Jun 20, 2017

alexandrebarachant commented Jun 20, 2017

marionleborgne commented Jun 20, 2017

marionleborgne commented Jun 20, 2017

marionleborgne commented Jun 21, 2017

alexcastillo commented Jun 22, 2017 via email

marionleborgne commented Jun 22, 2017 via email

alexcastillo commented Jun 22, 2017 via email

marionleborgne commented Jun 23, 2017

marionleborgne commented Jun 23, 2017 • edited Loading

alexcastillo commented Jun 23, 2017 via email

urish commented Jun 26, 2017

marionleborgne commented Jun 26, 2017

andrewjaykeller commented Jun 17, 2017 •

edited

Loading

marionleborgne commented Jun 17, 2017 via email •

edited

Loading

alexandrebarachant commented Jun 17, 2017 •

edited

Loading

alexcastillo commented Jun 17, 2017 •

edited

Loading

marionleborgne commented Jun 19, 2017 •

edited

Loading

marionleborgne commented Jun 23, 2017 •

edited

Loading