As of SFM 1.12, sfm-elk is deprecated.
A proof-of-concept analytics dashboard for social media content harvested by Social Feed Manager using the ELK (Elasticsearch, Logstash, Kibana) stack.
sfm-elk works by listening for warc_created messages produced by harvesters. Upon receiving a message it:
- Iterates over social media API calls records recorded in the WARC using a platform-specific WARC iterator (e.g., twitter_rest_warc_iter), outputting line-oriented json records to STDOUT.
- Pipes the json records to jq to filter out unnecessary metadata.
- Pipes the filtered json records to Logstash.
- Logstash loads the json records to Elasticsearch, which makes them available from Kibana.
sfm-elk currently supports loading WARCs from the Twitter REST API, Twitter Stream API, and Weibo API. A Twitter dashboard and Weibo dashboard are provided.
For more information on deploying and using SFM ELK, see the Exploring Social Media Data with ELK.