Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Periodic connection loss #320

Closed
asimpson opened this issue Mar 9, 2023 · 8 comments
Closed

Periodic connection loss #320

asimpson opened this issue Mar 9, 2023 · 8 comments
Labels
datasource/ClickHouse type/bug Something isn't working

Comments

@asimpson
Copy link
Contributor

asimpson commented Mar 9, 2023

What happened:
Every day the datasource throws a 500 error that is resolved by re-saving data source settings in Grafana. An error is logged as rpc error: code = Unavailable desc = error reading from server: EOF.

@asimpson asimpson added type/bug Something isn't working datasource/ClickHouse labels Mar 9, 2023
@asimpson
Copy link
Contributor Author

asimpson commented Mar 9, 2023

One suggestion was to toggle the protocol selection in the config editor. We're waiting to see if that gets rid of the periodic error.

@Ryado
Copy link
Contributor

Ryado commented Mar 9, 2023

This could be related to ideling in ClickHouse Cloud, wdyt @gingerwizard ?

@jkaflik
Copy link
Collaborator

jkaflik commented Mar 13, 2023

@asimpson it's version v2.0.7?

@Ryado @asimpson

The main question is if this issue is reproduced on ClickHouse Cloud only or happens for ClickHouse in general.
For cloud, extending dial timeout would do the trick if service goes idle. It's 10s default on backend:

https://github.com/jkaflik/clickhouse-datasource/blob/13a95dd951983246c856b617fb8165fbb9b6ca1b/pkg/plugin/settings.go#L120

In UI we suggest 60s, but user can skip it easily:
https://github.com/jkaflik/clickhouse-datasource/blob/13a95dd951983246c856b617fb8165fbb9b6ca1b/src/selectors.ts#L64

@jkaflik
Copy link
Collaborator

jkaflik commented Mar 13, 2023

@asimpson, this error message rpc error: code = Unavailable desc = error reading from server: EOF - isn't it an GRPC error, so Grafana could not talk to plugin process? I think we need to check what happens inside plugin. Is it something you can reproduce?

@jkaflik
Copy link
Collaborator

jkaflik commented Apr 12, 2023

@asimpson do you have any updates on this topic?

@asimpson
Copy link
Contributor Author

@jkaflik Unfortunately I was not able to reproduce the issue 😞

@aangelisc
Copy link
Contributor

Is this still an issue @jkaflik?

@mshustov
Copy link
Collaborator

mshustov commented Sep 7, 2023

let's close until we have a report with more details

@mshustov mshustov closed this as completed Sep 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasource/ClickHouse type/bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

5 participants