Research: Investigate Runing `ELK` on Fly.io

[nelsonic]

Context

At present the Fly.io logging is very basic. e.g: https://fly.io/apps/authprod/monitoring

This is "OK" for basic debugging/diagnosing but it's useless for trend analysis as there's no clarity on log retention and there's no ability to search, etc.

If you read the flyctl logs docs: https://fly.io/docs/flyctl/logs they are basic - at the time of writing ...

Presumably not many people are needing anything advanced.
or the people that need something more advanced are using a 3^rd party service. 🤷‍♂️

There's nothing like CloudWatch aws.amazon.com/cloudwatch

Todo

Confirm resource requirements, this is a JAVA beast after all ... https://github.com/elastic 🙄

Can we run a Single Docker Container?

https://elk-docker.readthedocs.io/

Read: https://medium.com/@svetob/sending-elixir-logs-to-logstash-as-json-1396605f9b7b
and https://github.com/svetob/logstash-json

[nelsonic]

https://fly.io/laravel-bytes/integrating-the-elastic-stack-elk-into-a-laravel-app-on-fly/

[nelsonic]

This container could work: https://hub.docker.com/r/sebp/elk/ 💭

[nelsonic]

Pricing: https://fly.io/docs/about/pricing/

[nelsonic]

https://community.fly.io/t/installing-elastic-search-server/4628

Doesn't look promising. 😢

[nelsonic]

But wait ...
https://fly.io/laravel-bytes/integrating-the-elastic-stack-elk-into-a-laravel-app-on-fly/#setting-up-elasticsearch

[nelsonic]

Going to need to re-read this with fresh eyes. But my first impression is that it's possible. 🤞

[nelsonic]

For reference, this is the cost of using the Elastic hosting service:
https://www.elastic.co/pricing/

[SimonLab]

From what I saw quickly on the Fly community forum, the recommended tool to use with an application deployed with Fly is fly-log-shipper
From what I understand you can send the log information to other tools for them to save/search... it:

[nelsonic]

@SimonLab good find. 🔍 👍
Of these options, I'm inclined to try https://github.com/grafana/loki 💭
Did you look at any other Open Source logging tools with hosted options?