Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ClickHouse Client #3196

Merged
merged 4 commits into from
Mar 25, 2022
Merged

Add ClickHouse Client #3196

merged 4 commits into from
Mar 25, 2022

Conversation

wsquan171
Copy link
Contributor

Signed-off-by: Shawn Wang wshaoquan@vmware.com

@wsquan171 wsquan171 force-pushed the ch-client branch 4 times, most recently from 1cdd123 to f654906 Compare January 13, 2022 01:39
@codecov-commenter
Copy link

codecov-commenter commented Jan 13, 2022

Codecov Report

Merging #3196 (666800c) into main (2c7d486) will decrease coverage by 2.67%.
The diff coverage is 55.61%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3196      +/-   ##
==========================================
- Coverage   65.58%   62.91%   -2.68%     
==========================================
  Files         277      278       +1     
  Lines       27264    37801   +10537     
==========================================
+ Hits        17881    23782    +5901     
- Misses       7483    12054    +4571     
- Partials     1900     1965      +65     
Flag Coverage Δ
e2e-tests 53.74% <3.17%> (?)
kind-e2e-tests 55.48% <3.48%> (-0.46%) ⬇️
unit-tests 43.42% <71.70%> (+0.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/flowaggregator/flowaggregator.go 64.28% <50.00%> (-4.83%) ⬇️
...lowaggregator/clickhouseclient/clickhouseclient.go 56.00% <56.00%> (ø)
pkg/agent/cniserver/pod_configuration_linux.go 26.31% <0.00%> (-40.36%) ⬇️
pkg/controller/ipam/antrea_ipam_controller.go 48.71% <0.00%> (-31.57%) ⬇️
pkg/controller/networkpolicy/endpoint_querier.go 61.46% <0.00%> (-29.97%) ⬇️
pkg/controller/egress/controller.go 62.19% <0.00%> (-26.26%) ⬇️
pkg/controller/ipam/validate.go 57.95% <0.00%> (-24.31%) ⬇️
pkg/agent/cniserver/ipam/antrea_ipam_controller.go 56.66% <0.00%> (-23.34%) ⬇️
pkg/agent/util/iptables/lock.go 60.00% <0.00%> (-21.82%) ⬇️
pkg/controller/egress/store/egressgroup.go 37.93% <0.00%> (-21.80%) ⬇️
... and 239 more

@wsquan171 wsquan171 force-pushed the ch-client branch 7 times, most recently from b3adb20 to ef23a7d Compare January 24, 2022 08:57
build/yamls/flow-aggregator.yml Outdated Show resolved Hide resolved
build/yamls/flow-aggregator.yml Outdated Show resolved Hide resolved
pkg/flowaggregator/clickhouseclient/clickhouseclient.go Outdated Show resolved Hide resolved
Copy link
Contributor

@heanlan heanlan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Shawn for working on the e2e test.

build/yamls/flow-aggregator.yml Outdated Show resolved Hide resolved
build/yamls/flow-visibility/flow-visibility-clickhouse.yml Outdated Show resolved Hide resolved
test/e2e/framework.go Outdated Show resolved Hide resolved
test/e2e/framework.go Outdated Show resolved Hide resolved
test/e2e/flowaggregator_test.go Outdated Show resolved Hide resolved
build/yamls/flow-visibility/flow-visibility-clickhouse.yml Outdated Show resolved Hide resolved
build/yamls/flow-visibility/flow-visibility-clickhouse.yml Outdated Show resolved Hide resolved
build/yamls/flow-visibility/flow-visibility-clickhouse.yml Outdated Show resolved Hide resolved
test/e2e/framework.go Outdated Show resolved Hide resolved
test/e2e/framework.go Outdated Show resolved Hide resolved
@heanlan
Copy link
Contributor

heanlan commented Feb 15, 2022

Hi @tnqn , could you please approve running workflows for this PR? Thanks.

@tnqn
Copy link
Member

tnqn commented Feb 15, 2022

Hi @tnqn , could you please approve running workflows for this PR? Thanks.

Done, and invited @wsquan171 to the organization.

test/e2e/framework.go Outdated Show resolved Hide resolved
@heanlan heanlan added this to the Antrea v1.6 release milestone Feb 22, 2022
build/yamls/flow-aggregator.yml Outdated Show resolved Hide resolved
build/yamls/flow-aggregator.yml Outdated Show resolved Hide resolved
build/yamls/flow-aggregator.yml Outdated Show resolved Hide resolved
pkg/config/flowaggregator/config.go Outdated Show resolved Hide resolved
pkg/flowaggregator/clickhouseclient/clickhouseclient.go Outdated Show resolved Hide resolved
@wsquan171 wsquan171 force-pushed the ch-client branch 2 times, most recently from a62b9e3 to 33f440f Compare March 23, 2022 21:52
antoninbas
antoninbas previously approved these changes Mar 23, 2022
Copy link
Contributor

@antoninbas antoninbas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

wsquan171 and others added 4 commits March 24, 2022 11:44
Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Signed-off-by: heanlan <hanlan@vmware.com>
This change adds the following changes to CH client options:
1. adds new option "commitInterval" for customizing batch commit
   interval from CH client
2. sets default values for database, db url, commit interval when
   clickHouse config exists in FA configMap.
3. makes externalFlowCollectorAddr optional when CH is configured
4. adds an explicit switch for turning on CH exporting, for more
   intuit conf editing.
5. moves previous IPFIX and JSON collector options to a dedicated
   section.

Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
@wsquan171
Copy link
Contributor Author

/test-all

@wsquan171
Copy link
Contributor Author

/test-conformance

@wsquan171
Copy link
Contributor Author

/test-e2e

@wsquan171
Copy link
Contributor Author

/test-networkpolicy

@antoninbas
Copy link
Contributor

/test-integration

8 similar comments
@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@wsquan171
Copy link
Contributor Author

/test-integration

@antoninbas antoninbas merged commit 10afc0d into antrea-io:main Mar 25, 2022
@wsquan171 wsquan171 deleted the ch-client branch March 25, 2022 22:12
@dreamtalen dreamtalen mentioned this pull request Mar 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants