New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to configure TLS origination with postgres #29761
Comments
@samos123 Can you dump the cluster config? And make sure the certificate is valid |
I did double-check by getting a shell to the istio-proxy and doing cat /etc/certs/aiven-ca.crt. The cert is valid and I can use it to connect over TLS using pgadmin when istio isn't configured to do TLS origination. I did find some more interesting logs while trying with postgresql cli client instead of pgadmin:
Here is the cluster config:
Does that help? |
I also saw the following log which seems to indicate the cert was found and picked up after destinationrule was updated:
|
I mean adding |
Did you mean this? If so here is the output of that: https://pastebin.com/sf5wEcqL |
it looks good |
Any ideas on what to try next? |
It seems others are having similar issues with postgres [1][2]. Note that I was able to get it working with Redis succesfully so something special is happening with postgres. Here is how I did Redis TLS origination successfully: https://samos-it.com/posts/securing-redis-istio-tls-origniation-termination.html It's quite easy to reproduce for me. Let me know if you want access to the Postgres TLS database so you can reproduce it too. [1] https://discuss.istio.io/t/egress-gateways-with-tls-origination-and-tls-passthrough-for-egress-chokepoint/6536/2 |
https://istio.io/v1.8/docs/ops/deployment/requirements/#server-first-protocols Just to make sure this is searchable. |
I already have the port defined as TCP in the service entry see my final manifests below: Final desitionationrule:
ServiceEntry:
The pod that's trying to connect:
However it doesn't solve the issue. Any other recommendations? |
This was discussed on chat with @howardjohn, @mandarjog and others . Summary of our discussion, postgres uses starttls and this is causing issues with TLS origination. Istio would need a postgres filter for TLS origination to work with postgres. In addition, there are no known benefits of doing TLS origination for postgres today. There is no improved security or improved visibility. As a result, it's recommended to simply start TLS from the application container directly. This way Istio at least sees the SNI. |
How to re-open? The wiki page doesn't have clear instructions. |
Can this be re-open? |
@samos123 is it possible that postgres disable starttls and only accept mTLS? Then the client only launch plain text and sidecar help to do TLS origination? |
@hobbytp This is not possible. Before TLS negotiation takes place, there is postgres specific protocol message exchange which determines whether to continue in cleartext or start TLS handshake. All this happens within one TCP session, so it is not vanilla TLS and requires starttls as upstream transport socket and support from Envoy postgres filter to drive that transport socket. |
Any ETA on a fix? |
The issue need to be addressed as this issue is linked to #33345 |
🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2021-12-24. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions. Created by the issue and PR lifecycle manager. |
is there any new activity or solution to this problem? |
Bug description
I'm trying to use Aiven Postgres which provides managed postgres that uses TLS only. I want Istio to do the TLS origination using the side car proxy instead of using egress gateway. Note that Aiven is outside of the mesh.
[X] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[X] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
Ability to connect to postgres using plain TCP from the application pod and have the sidecar do the TLS to postgres itself.
Steps to reproduce the bug
DestinationRule:
ServiceEntry:
Pgadmin deployment for testing:
Version (include the output of
istioctl version --remote
andkubectl version --short
andhelm version --short
if you used Helm)Logs on istio-proxy:
How was Istio installed?
ASM install using install_asm script
Environment where the bug was observed (cloud vendor, OS, etc)
GKE on GCP
Additionally, please consider running
istioctl bug-report
and attach the generated cluster-state tarball to this issue.Refer cluster state archive for more details.
The text was updated successfully, but these errors were encountered: