Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: kafka over ssh #3007

Merged
merged 6 commits into from
Mar 15, 2023
Merged

feat: kafka over ssh #3007

merged 6 commits into from
Mar 15, 2023

Conversation

fracasula
Copy link
Collaborator

@fracasula fracasula commented Feb 20, 2023

Description

This PR is to add support for connecting to Kafka via SSH. It is required by a specific customer and control plane prefers not to have SSH keys on their side so for this one we're opting to configure via environment variables only.

The PR was tested manually with a Kafka over SSH on AWS (EC2).

Related Slack conversation.

Sidenote

In order to make the test work I had to make changes to the way Kafka was provisioned so that the advertised listeners were set correctly.

Notion Ticket

< Notion Link >

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@fracasula fracasula force-pushed the feat.kafkaSsh branch 2 times, most recently from 495d215 to 815c764 Compare February 20, 2023 14:38
@codecov
Copy link

codecov bot commented Feb 20, 2023

Codecov Report

Patch coverage: 73.61% and project coverage change: +0.04 🎉

Comparison is base (ea3bbd5) 53.79% compared to head (e30a9cb) 53.83%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3007      +/-   ##
==========================================
+ Coverage   53.79%   53.83%   +0.04%     
==========================================
  Files         348      350       +2     
  Lines       53846    54003     +157     
==========================================
+ Hits        28965    29074     +109     
- Misses      23240    23281      +41     
- Partials     1641     1648       +7     
Impacted Files Coverage Δ
services/streammanager/kafka/client/config.go 60.27% <ø> (ø)
testhelper/destination/common.go 0.00% <0.00%> (ø)
testhelper/destination/sshserver/sshserver.go 65.78% <65.78%> (ø)
services/streammanager/kafka/client/client.go 83.52% <68.00%> (-6.48%) ⬇️
testhelper/destination/kafka.go 80.66% <83.78%> (+2.19%) ⬆️
services/streammanager/kafka/kafkamanager.go 76.77% <90.62%> (+1.02%) ⬆️
services/streammanager/kafka/client/producer.go 79.81% <100.00%> (-0.55%) ⬇️
...rvices/streammanager/kafka/client/testutil/util.go 66.21% <100.00%> (-4.66%) ⬇️
testhelper/destination/minio.go 70.00% <100.00%> (ø)
testhelper/destination/postgres.go 73.33% <100.00%> (ø)
... and 2 more

... and 6 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

}
return conn, nil
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set the net Dialer with Timeout if not defined?

Suggested change
}
}
dialer.DialFunc = net.Dialer{
Timeout: c.config.DialTimeout,
}.DialContext

Copy link
Collaborator Author

@fracasula fracasula Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a timeout in the SSH config. It works as expected when dialing. If not defined then the timeout in line 47 should apply. Wdyt?

@fracasula fracasula changed the base branch from master to release/1.6.x March 10, 2023 12:10
@fracasula fracasula changed the base branch from release/1.6.x to master March 10, 2023 12:10
@fracasula fracasula merged commit 99262c3 into master Mar 15, 2023
@fracasula fracasula deleted the feat.kafkaSsh branch March 15, 2023 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants