Logging & Monitoring

[nesimtunc]

We need a logging and monitoring mechanism. We can use Winston

We can discuss here about our options: evrimagaci/periodum#76

This task can be done after #1 completed

[NovusEdge]

I would like to take this issue if no one's assigned, please let me know of the specifics

[nesimtunc]

Hi @NovusEdge,

Thank you for your interest! Of course! We'd be happy if you can contribute.

Specs are in the discussion, as we're open to discuss, feel free to suggest or recommend anything related.

You may create your branch from mine as a code base.

Before start to work on it maybe you may write some description (kindda spec but not much detail - don't wanna take much of time) about the implementation, if we're agree on it, then you may start on it.

How that sounds?

[NovusEdge]

My apologies, but it seems I wouldn't be able to work on this issue. 😞

[nesimtunc]

It’s okay, no worries :)

[sonerpyci]

Hi there,
If there is no one working on it I would like to get started with this issue.

Also,
You stated that you do not want to deal with maintenance and DevOps workload for logging and monitoring, but additional costs can be prevented by configuring grafana and prometheus in docker-compose.yml. Since it is a public and high-scaling project, I think it is more beneficial to use more powerful tools with large communities behind it, and I can provide this.

[nesimtunc]

Hi @sonerpyci,

Thank you for stopping by. You got a point! As a matter of fact, we're already awaiting a reply from AppSignal for that support. Meanwhile can you tell us more what can you exactly provide to us, please?

Thanks!

[sonerpyci]

Hi @nesimtunc, When I left a comment on this issue, I've already prepared a working example of periodum-api with grafana and prometheus support on the same docker network. As prometheus is an open-sourced tool that enables collection of custom metrics and totally free, I believe it will be more suitable for this project. Let me attach an example image of my local version,

[nesimtunc]

Hi @sonerpyci,

Sounds good! Why don't send us a PR and we'll look into it. Btw, do you have DevOPS experience?

[sonerpyci]

Hi @nesimtunc,
I've just created a PR to review changes on periodum-api repository.
Also, I have some DevOps experience as I have worked with Docker before, but I am not very assertive about it, I'm experienced mainly on backend services.

[nesimtunc]

Hi @sonerpyci,

Thank you for the PR, I have reviewed. Since, this is only for monitoring, I wonder do you have a recommended solution for logging? We want to log un/handled exceptions, and some other logs with severity, also needs to aggregate them. With your solution, we can aggregate HTTP status codes via Grafana but we also need a logging system instead of stdout (console) log.

[sonerpyci]

Hi @nesimtunc,
Since the periodum is a project that accessible to everyone, In my opinion, there should be a request and response logger that works as a middleware.
I think Logger middleware basically should log only request and response information.

(such as client IP, request body, response body, route, headers, HTTP Method, HTTP Status Code, etc)

In case of any handled or unhandled exception, the logger middleware should log an additional field that contains exception details.

In order to better examine the errors that occur, logs should be saved to a database or elastic search.

I think there is no need to use a package to provide what I wrote above, I have designed a logger middleware before that works the same as above.

[sonerpyci]

Hi @nesimtunc,
I've just created a new PR to add Logging capability to the project.