[PROPOSAL][CCI] Add sample for logs and metrics collection and processing

[Refffy]

What are you proposing?

Add a service or may be a script for collecting and analyzing logs, crashes, etc. like sentry, loki or graphana

What problems are you trying to solve?

It'll help to calculate and store statistics of errors appearance ratio. Also, it'll be very helpfull for opensearch newbies (like me) to create new issues

[wbeckler]

Do you have any ideas for a good python framework for this? It probably makes sense to limit the options to open source, so that the library can remain open source.

[dtaivpp]

@Refffy I did a presentation for collecting logs from Python into OpenSearch. I wonder if it would make sense to integrate something like that here.

https://gist.github.com/dtaivpp/a9b00957aa7d9cfe33e92aff8d50c835

Above is a gist and below is the video of this.

https://youtu.be/-8CNa4T2Jaw

Is something like this what you were thinking of?

[Refffy]

Hello, @dtaivpp! Yes, I think that your code snippet on gist is makes sense to this issue. But I also think, that it'll great not only to collect logs, but also to show common project issues in realtime like sentry, loki or graphana or even use elastic search for collecting and representing logs metrics (for example, crashes or something like that), but I suppose that it'll be hard to implement.

[dtaivpp]

I see so more of like an APM type workflow so that you get more context. Might be worth exploring adding traces to this. Here is another good place to read:

https://opentelemetry-python.readthedocs.io/en/latest/api/trace.html

[saimedhi]

@Refffy, @dtaivpp Please feel free to take up this issue and raise a PR with your proposed solution. Thanks :)

[Djcarrillo6]

Hi @saimedhi! This sounds like a fun proposal to work on! Can I request assignment on this issue?

[dblock]

Do I understand this issue correctly that we're talking about adding a sample for logs and metrics collection and processing?

[Djcarrillo6]

@dblock that was my interpretation as well. Something similar to was @dtaivpp mentioned.

[Djcarrillo6]

@dblock @saimedhi What do you think is best way to address this issue? I was thinking of integrating the log collection script that David mentioned.. Does that make sense to you?

[dblock]

Closed via #331.