Quick demo using the Elastic stack to visualize sensor information provided by Santander Town Hall and available through datos.gob.es. This dataset contains sensor measurements for light, noise, temperature and battery gathered from different locations across the city of Santander.
- docker engine (v1.12.0+)
- docker-compose
- Elastic stack v5.4 (installed and managed by Docker()
Data can be accessed here. For this example we will use the JSON format to load data into Elasticsearch using Logstash.
In order to parse the data with Logstash, we will remove the EOLs in the file. For this I will use jq JSON processor but you can use anything that you like, even sed ;)
wget http://datos.santander.es/api/rest/datasets/sensores_smart_env_monitoring.json
cat sensores_smart_env_monitoring.json | jq -c . > data_no_eol.json
To parse, load and visualize the data we only need to execute the containers defined in docker-compose.yml:
docker-compose up
Note that Elastic containers will also install X-Pack, commercial extensions provided by Elastic, to get access control and better mapping capabilities.
Once data is loaded into Elasticsearch we will use Kibana to visualize it. To access Kibana the default username and password is elastic:changeme
The first step is to configure a Kibana index pattern for the index 'sensors'. In the Management > Kibana > Index Patterns section we should have:
After this step, we can import the pre-built visualizations and dashboard. For that we should go to Management > Kibana > Saved Objects and import kibana/objects.json file.
Finally, to open the pre-built dashboard we go to Dashboard and we open the dashboard called 'Santander'. You can also create your own visualization and dashboards depending on your needs. Enjoy!