Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 62 additions & 107 deletions README.md

Large diffs are not rendered by default.

Binary file added design/proton-architecture.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 0 additions & 45 deletions docker-compose.yml

This file was deleted.

40 changes: 30 additions & 10 deletions examples/awesome-sensor-logger/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,35 @@
# awesome-sensor-logger
# Demo to visualize the sensor data from your phone

To install dependencies:
[Sensor Logger](https://github.com/tszheichoi/awesome-sensor-logger) is a free, easy-to-use, cross-platform data logger that logs readings from common motion-related sensors on smartphones. Recordings can be exported as a zipped CSV file, or streamed to Proton via HTTP API.

```bash
bun install
```
Demo video: https://youtu.be/vi4Yl6L4_Dw?t=1049

To run:
## Setup Guide

### Step 1: start docker compose
Install [Docker Desktop](https://docs.docker.com/desktop/) and fork this repo or download the [docker-compose.yml](docker-compose.yml) to a local folder. Start everything via `docker compose up`. The following containers will be started:
* latest version of Proton, to receive the live data and apply stream processing
* latest version of Grafana, to visualize the live data with streaming SQL
* a lightweight proxy server to convert JSON from sensor loggers to the format for Proton Ingest API
* a short-live container to start proton client and create the stream `phone`

Wait for about half a minute to have all containers up running.

### Step 2: install mobile app and push data to Proton
Download Sensor Logger at www.tszheichoi.com/sensorlogger.

| Android | iOS |
|:-:|:-:|
| [<img src="https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png" height="50">](https://play.google.com/store/apps/details?id=com.kelvin.sensorapp&pcampaignid=pcampaignidMKT-Other-global-all-co-prtnr-py-PartBadge-Mar2515-1) | [<img src="https://developer.apple.com/app-store/marketing/guidelines/images/badge-example-preferred_2x.png" height="50">](https://apps.apple.com/app/id1531582925) |

Open the app, and go to the settings page, by clicking the button at the bottom.
1. Click the `Data Streaming` menu.
2. Turn on `Enable HTTP Push`.
3. Get your IP for your server (on Mac, you can hold Option key and click the WiFi icon to get the IP address), and set the `Push URL` as `http://<ip>:8000`
4. Optionally, you can turn on `Skip Writing`.

### Step 3: view the live dashboard in Grafana

In your laptop/server, access `http://localhost:3000` and open the `Phone Sensor`

```bash
bun run index.ts
```

This project was created using `bun init` in bun v1.0.15. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime.
33 changes: 32 additions & 1 deletion examples/awesome-sensor-logger/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
version: '3.7'
services:
proton:
image: ghcr.io/timeplus-io/proton:latest
Expand All @@ -7,6 +6,23 @@ services:
- 3218:3218 #http port for JDBC driver, default streaming mode
- 8123:8123 #http port for JDBC driver, default batch mode
- 8463:8463 #tcp port for go driver or grafana plugin
healthcheck:
test: ["CMD", "curl", "http://localhost:3218/proton/ping"]
interval: 2s
timeout: 10s
retries: 3
start_period: 10s

init-proton:
image: ghcr.io/timeplus-io/proton:latest
command:
- sh
- -c
- |
proton-client -h proton -n --query "CREATE STREAM IF NOT EXISTS phone(raw string)"
depends_on:
proton:
condition: service_healthy

proxy:
image: timeplus/sensor-logger-proxy
Expand All @@ -17,3 +33,18 @@ services:
STREAM: phone
depends_on:
- proton

grafana:
image: grafana/grafana:latest
pull_policy: always
ports:
- 3000:3000
environment:
GF_AUTH_ANONYMOUS_ENABLED: 1
GF_AUTH_ANONYMOUS_ORG_ROLE: Admin
GF_INSTALL_PLUGINS: timeplus-proton-datasource
volumes:
- ./grafana_provisioning:/etc/grafana/provisioning
depends_on:
init-proton:
condition: service_completed_successfully
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may like to put another empty line after

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested all docker compose files. Without the empty line in the bottom, it can work, but I am not sure whether having the empty last line is a best practice or not. Maybe nice to have

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saw some discussions https://stackoverflow.com/questions/2287967/why-is-it-recommended-to-have-empty-line-in-the-end-of-a-source-file More like to avoid unexpected errors for old tools.

Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 2,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "timeplus-proton-datasource",
"uid": "c24e0faf-1490-4321-a373-7b2b07ca2e38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "continuous-GrYlRd"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"fillOpacity": 80,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineWidth": 1,
"scaleDistribution": {
"type": "linear"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 0
},
"id": 1,
"options": {
"barRadius": 0,
"barWidth": 0.97,
"fullHighlight": false,
"groupWidth": 0.7,
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"orientation": "auto",
"showValue": "auto",
"stacking": "none",
"tooltip": {
"mode": "single",
"sort": "none"
},
"xTickLabelRotation": 0,
"xTickLabelSpacing": 0
},
"targets": [
{
"addNow": false,
"datasource": {
"type": "timeplus-proton-datasource",
"uid": "c24e0faf-1490-4321-a373-7b2b07ca2e38"
},
"isStreaming": true,
"queryText": "select array_join(xyz) as v, multi_if(v[1]=1,'x',v[1]=2,'y','z') as axis, v[2] as value, time from(\nselect to_datetime64((p:time::int64)/1000000000,3) as time,[[1,to_float32_or_zero(p:values.x)],[2,to_float32_or_zero(p:values.y)],[3,to_float32_or_zero(p:values.z)]] as xyz from (select array_join(json_extract_array(raw,'payload')) as p from phone where p:name='gyroscope')\n)",
"refId": "A"
}
],
"title": "Bar Chart",
"type": "barchart"
},
{
"datasource": {
"type": "timeplus-proton-datasource",
"uid": "c24e0faf-1490-4321-a373-7b2b07ca2e38"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"pointSize": {
"fixed": 5
},
"scaleDistribution": {
"type": "linear"
},
"show": "points"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 0
},
"id": 2,
"options": {
"dims": {
"exclude": [
"z"
]
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"series": [],
"seriesMapping": "auto",
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"addNow": false,
"datasource": {
"type": "timeplus-proton-datasource",
"uid": "c24e0faf-1490-4321-a373-7b2b07ca2e38"
},
"isStreaming": true,
"queryText": "select to_datetime64((p:time::int64)/1000000000,3) as time,to_float32_or_zero(p:values.x) as x,to_float32_or_zero(p:values.y) as y,to_float32_or_zero(p:values.z) as z from (select array_join(json_extract_array(raw,'payload')) as p from phone where p:name='gyroscope')\n",
"refId": "A"
}
],
"title": "XY Chart",
"type": "xychart"
}
],
"refresh": "",
"schemaVersion": 39,
"tags": [],
"templating": {
"list": []
},
"time": {
"from": "now-3m",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Phone Sensor",
"uid": "a3406569-2c08-454c-8791-64064366e107",
"version": 3,
"weekStart": ""
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: 1

providers:
# <string> an unique provider name. Required
- name: 'a unique provider name'
# <int> Org id. Default to 1
orgId: 1
# <string> name of the dashboard folder.
folder: ''
# <string> folder UID. will be automatically generated if not specified
folderUid: ''
# <string> provider type. Default to 'file'
type: file
# <bool> disable dashboard deletion
disableDeletion: false
# <int> how often Grafana will scan for changed dashboards
updateIntervalSeconds: 10
# <bool> allow updating provisioned dashboards from the UI
allowUiUpdates: true
options:
# <string, required> path to dashboard files on disk. Required when using the 'file' type
path: /etc/grafana/provisioning/dashboards
# <bool> use folder names from filesystem to create folders in Grafana
foldersFromFilesStructure: true
Loading