Add params for secure ClickHouse connections. #2018
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
/gcbrun |
Codecov Report
@@ Coverage Diff @@
## master #2018 +/- ##
==========================================
+ Coverage 90.95% 90.99% +0.04%
==========================================
Files 499 499
Lines 21560 21572 +12
==========================================
+ Hits 19609 19629 +20
+ Misses 1951 1943 -8
Continue to review full report at Codecov.
|
konstantin-popov
force-pushed
the
ch_secure_connection
branch
from
86a81f5 to
3c429c2
Compare
Member
|
/gcbrun |
Member
|
/gcbrun |
Member
|
/gcbrun |
evanh
approved these changes
Jul 29, 2021
Member
evanh
left a comment
evanh
left a comment
There was a problem hiding this comment.
Sorry this took so long to merge.
Member
|
/gcbrun |
evanh
added a commit
that referenced
this pull request
Jul 29, 2021
This was referenced Oct 27, 2024
untitaker
added a commit
that referenced
this pull request
Jan 14, 2025
### Description
This pull request introduces SSL/TLS support for ClickHouse connections
in the Snuba project. The changes include new CLI options for enabling
secure connections, updates to the ClickhousePool and HTTPBatchWriter
classes, and corresponding configuration options in settings and tests.
### Changes Overview
1. **CLI Options**:
- Introduced new CLI options for enabling secure connections to
ClickHouse:
- `--clickhouse-secure`: If true, an encrypted connection will be used.
- `--clickhouse-ca-certs`: An optional path to certificates directory.
- `--clickhouse-verify`: Verify ClickHouse SSL cert.
2. **Class Updates**:
- Modified `ClickhousePool`, `HTTPBatchWriter`, and other relevant
classes to support SSL/TLS connections.
- Updated constructors and methods to accept and handle SSL/TLS
parameters.
3. **Configuration**:
- Added SSL/TLS configuration options in settings and tests.
- Updated environment variables to support SSL/TLS settings.
4. **Testing**:
- Included SSL/TLS configuration in test cases.
- Updated existing tests to ensure compatibility with SSL/TLS options.
### Detailed Changes
- **snuba/cli/cleanup.py**: Added new CLI options for secure ClickHouse
connections.
- **snuba/cli/migrations.py**: Added new CLI options for secure
ClickHouse connections.
- **snuba/cli/optimize.py**: Added new CLI options for secure ClickHouse
connections.
- **snuba/clickhouse/http.py**: Modified `HTTPBatchWriter` to support
SSL/TLS connections.
- **snuba/clickhouse/native.py**: Updated `ClickhousePool` to handle
SSL/TLS parameters.
- **snuba/clusters/cluster.py**: Updated `ClickhouseCluster` to include
SSL/TLS configuration.
- **snuba/migrations/runner.py**: Added SSL/TLS parameters to migration
runner.
- **snuba/settings/__init__.py**: Added SSL/TLS configuration options.
- **snuba/settings/settings_distributed.py**: Added SSL/TLS
configuration options.
- **tests/clusters/fake_cluster.py**: Updated `FakeClickhouseCluster` to
include SSL/TLS parameters.
- **tests/clusters/test_cluster.py**: Updated tests to include SSL/TLS
configuration.
- **tests/conftest.py**: Updated test setup to include SSL/TLS
configuration.
- **tests/migrations/test_connect.py**: Updated tests to include SSL/TLS
configuration.
- **tests/migrations/test_table_engines.py**: Updated tests to include
SSL/TLS configuration.
- **tests/replacer/test_cluster_replacements.py**: Updated tests to
include SSL/TLS configuration.
## Related Issues
- #6458
## Related Pull Requests:
- #2018
- #2033
### Additional Notes
- This change is backward compatible and does not require any additional
setup for users who do not wish to enable SSL/TLS.
- Please review the changes carefully to ensure that the SSL/TLS
implementation is secure and efficient.
FYI @konstantin-popov
Thank you for reviewing this pull request!
### Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated
in the State of Delaware in 2015 as Functional Software, Inc. and is
gonna need some rights from me in order to utilize my contributions in
this here PR. So here's the deal: I retain all rights, title and
interest in and to my contributions, and by keeping this boilerplate
intact I confirm that Sentry can use, modify, copy, and redistribute my
contributions, under Sentry's choice of terms.
---------
Co-authored-by: Markus Unterwaditzer <markus-honeypot@unterwaditzer.net>
Co-authored-by: Markus Unterwaditzer <markus-tarpit+git@unterwaditzer.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support secure connections to ClickHouse (both native and http).