Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat: clickhouse query logs receiver #316

Merged
merged 28 commits into from
Apr 25, 2024
Merged

Feat: clickhouse query logs receiver #316

merged 28 commits into from
Apr 25, 2024

Conversation

raj-k-singh
Copy link
Contributor

@raj-k-singh raj-k-singh commented Apr 22, 2024 •
edited

Adds a receiver for scraping system.query_log table in clickhouse as otel logs

contributes to SigNoz/signoz#4830

@raj-k-singh raj-k-singh changed the title Feat/signoz clickhouse system logs receiver Feat: clickhouse system logs receiver Apr 22, 2024
@raj-k-singh raj-k-singh changed the title Feat: clickhouse system logs receiver Feat: clickhouse query logs receiver Apr 22, 2024
@raj-k-singh raj-k-singh force-pushed the feat/signoz-clickhouse-system-logs-receiver branch from 7db0a7a to 0888b89 Compare April 22, 2024 03:40
@raj-k-singh
feat: get signozclickhousesystemlogsreceiver started
6f914c8
@raj-k-singh
chore: put together receiver scaffolding for clickhousesystemtablesre…
db3fda3 
…ceiver
@raj-k-singh
chore: add clickhousesystemstablereceiver to list of available otel c…
b14152b 
…ollector receivers
@raj-k-singh
feat: add impl for unixTsNowAtClickhouse
4b4b038
@raj-k-singh
feat: get query_log querying logic started
e0b068e
@raj-k-singh
feat: flesh out query_log scraping loop
91f038d
@raj-k-singh
feat: finish up with e2e tracer for query log receiver
b671986
@raj-k-singh
chore: add example config for clickhousesystemtablesreceiver
3956ea4
@raj-k-singh
chore: get receiver_test started
646867d
@raj-k-singh
chore: refactor clickhouse calls behind an interface for easier mocki…
6443a52 
…ng in tests
@raj-k-singh
chore: flesh out receiver happy path test
9201f22
@raj-k-singh
chore: wrap up receiver lifecycle test
43760f7
@raj-k-singh
chore: add test for receiver start and shutdown
37f6bc0
@raj-k-singh
chore: add config validation tests
70200a3
@raj-k-singh
chore: some cleanup
52068df
@raj-k-singh
chore: hook log emitted telemetry on receiver
b32e7dc
@raj-k-singh
chore: some more cleanup
4899944
@raj-k-singh raj-k-singh force-pushed the feat/signoz-clickhouse-system-logs-receiver branch from ac3d090 to 4899944 Compare April 23, 2024 05:27
@raj-k-singh raj-k-singh marked this pull request as ready for review April 23, 2024 05:27
srikanthccv
srikanthccv reviewed Apr 23, 2024
View reviewed changes
Copy link
Member

@srikanthccv srikanthccv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial suggestions. I am still trying to understand the timestamp adjustment logic.

receiver/clickhousesystemtablesreceiver/utils.go Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/query_log.go Outdated Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/query_log.go Outdated Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/receiver.go Outdated Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/receiver.go Outdated Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/receiver.go Outdated Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/receiver.go Outdated Show resolved Hide resolved
receiver/clickhousesystemtablesreceiver/receiver.go Show resolved Hide resolved
srikanthccv
srikanthccv approved these changes Apr 24, 2024
View reviewed changes
Copy link
Member

@srikanthccv srikanthccv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks for addressing the comments.

@raj-k-singh raj-k-singh merged commit 24ec1a7 into main Apr 25, 2024
3 checks passed
@raj-k-singh raj-k-singh deleted the feat/signoz-clickhouse-system-logs-receiver branch April 25, 2024 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srikanthccv srikanthccv srikanthccv approved these changes

@ankitnayan ankitnayan Awaiting requested review from ankitnayan ankitnayan is a code owner

@makeavish makeavish Awaiting requested review from makeavish makeavish is a code owner

@nityanandagohain nityanandagohain Awaiting requested review from nityanandagohain nityanandagohain is a code owner

@prashant-shahi prashant-shahi Awaiting requested review from prashant-shahi prashant-shahi is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@raj-k-singh @srikanthccv