TLS auth cannot be used between proxy and brokers #1991
Comments
|
My original hunch on the advertising being wrong was incorrect. It does in fact go to the correct address if you have the right set of options configured. my proxy.conf tls bits my broker.conf tls bits tlsAllowInsecureConnection doesn't exist in the config file as is. It needs to be documented. This works for the brokerService port. I have another role "ivan". However, if "ivan" uses they admin client against the proxy, he can grant himself any permissions he wants. This is because the proxy is authenticated against the broker as a superuser, anyone authenticated against the proxy gains those superuser powers. There's no forwarding of principal from the proxy for admin like there is for data. |
So that they can be configured to run with TLS. The certificate authority used to generate the certs is also included in tests/ with instructions on how to generate new user and server certs. These certs will be used for an integration test once apache#1991 is solved.
So that they can be configured to run with TLS. The certificate authority used to generate the certs is also included in tests/ with instructions on how to generate new user and server certs. These certs will be used for an integration test once #1991 is solved.
|
@ivankelly @sijie Is this not an issue? Is this tested in K8s along with nodeport config? |
|
@pckeyan This is an issue, yes. Working on a fix now, though I'm testing directly in docker locally. Will add a task for testing with nodeport in k8s also. |
|
@ivankelly I tested with the identified parameter, SSL is working is fine between Proxy and Broker in K8s. Thanks for your help. Appreciated. |
|
@pckeyan do you have authorization enabled? The problem with the parameters I put is that any client that is authorized with the proxy will have full superuser access for admin operations, which obviously isn't ideal. |
|
Yes I have authorization enabled, I tested across LOBs of my use case and is rightly declined. Let me retest. Let me know if you want me to test some thing different. I have billing and order as two lobs. Billing owned topic cannot be used by Order to produce/consume. |
|
produce/consume isn't the problem. The problem is that someone authorized as bank can use the admin api to grant permissions to themselves to produce/consume on a card owned topic. |
|
But still they need Certificates to pass it on to get authenticated; Please correct my understanding |
|
@pckeyan yes, it's not wide open, but it's more open than it should be. |
|
@ivankelly Are you planning for a fix there? What would be ideal fix? |
|
@pckeyan I'd like to talk to @merlimat & @jai1 since they've worked on similar stuff, but the planned fix is to send the original authenticated rolename with the proxied requests, similiar to how we do for the data protocol. Then we'd need to document all this stuff, because there's no documentation right now. Hopefully I can knock out the fix tomorrow. |
|
Hi Ivan,
Since proxy was meant to be on a public IP we have added extra
authentication checks to prevent MITM attacks and safeguard against
compromised proxies. - but all these extra checks are configurable and are
turned off by default.
a. You can set disable authentication and authorization and proxy to
disable all checks. (Not recommended for production)
b. If authentication and authorization is enabled on Proxy and Broker, and
if a consume or produce action is via a proxy the
- originalPrincipal - will contain Client role name (aka principal)
- authRole - will contain Proxy role name, extracted from
authenticationData and authMethod sent by the proxy
- Both the Proxy and the end Client *should* have the appropriate
consume or produce permissions or else a compromised proxy can mimic any
client by just sending a stolen originalPrincipal (string). Or in other
words a compromised proxy should not be able to access more data than a
compromised client.
b. OriginalPrincipal is just a string and can easily be remanufactured by a
compromised proxy, so in order to safeguard the service you can enable "
*forwardAuthorizationCredentials*" on the proxy and enable "
*authenticateOriginalAuthData*" on the broker.
- The proxy will forward the client authenticationData (digital
signature with TTL) and authenticationData instead of a string and the
broker will authenticate the end client and extract originalPrincipal from
client authenticationData.
c. Another scenario could be that the proxy mimics the client by setting
originalPrincipal as NULL
- In order to prevent that you can set "*proxyRoles*" in the broker
configuration.
Below are some links to the configurations you can set. This weekend I will
go through the documentation and add any missing pieces (if any).
https://github.com/apache/incubator-pulsar/blob/master/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java#L247
https://github.com/apache/incubator-pulsar/blob/master/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java#L251
https://github.com/apache/incubator-pulsar/blob/master/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/ProxyConfiguration.java#L79
Regards,
Jai
…On Mon, Jun 25, 2018 at 12:42 PM, Ivan Kelly ***@***.***> wrote:
@pckeyan <https://github.com/pckeyan> I'd like to talk to @merlimat
<https://github.com/merlimat> & @jai1 <https://github.com/jai1> since
they've worked on similar stuff, but the planned fix is to send the
original authenticated rolename with the proxied requests, similiar to how
we do for the data protocol. Then we'd need to document all this stuff,
because there's no documentation right now. Hopefully I can knock out the
fix tomorrow.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1991 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIQh_P5TUcQOTSF6L1RUVVOac5QyD1ILks5uAT1DgaJpZM4UtYek>
.
|
I assume this only applies to Athenz, as TLS doesn't use the auth data AFAICS.
There's quite a bit missing, in particular for the TLS auth context, which most people will be using. Once I've solved this issue, I'll take a pass over the docs too. In terms of this issue, my plan is to make the admin api respect originalPrincipal and proxyRoles also, carrying it in a header ("X-Original-Principal"). This does mean that proxy would have to be added as admin role for each tenant that wants to use admin api through the proxy. In some usecases though, I think proxy will have to be a superuser at the broker. |
The broker admin checks if the authenticated user is listed as a proxy user. If so, it checks looks for the header X-Original-Principal, and validates that the role is authorized to access the resource in question. There are two use cases: 1. The proxy role is a normal role. In this case, if a resource is to be used via the proxy, the proxy user must be explicitly granted permission on the resource. So, to use the admin api for a tenant, the proxy role must be listed as a tenant admin. 2. The proxy role is a superuser role. In this case, any resource can be used via the proxy as long as the user authorized with the proxy is authorized to use the resource. However, if the proxy is compromised, a bad actor has full access to the cluster. This patch is the first part of a fix for apache#1991.
The broker admin checks if the authenticated user is listed as a proxy user. If so, it checks looks for the header X-Original-Principal, and validates that the role is authorized to access the resource in question. There are two use cases: 1. The proxy role is a normal role. In this case, if a resource is to be used via the proxy, the proxy user must be explicitly granted permission on the resource. So, to use the admin api for a tenant, the proxy role must be listed as a tenant admin. 2. The proxy role is a superuser role. In this case, any resource can be used via the proxy as long as the user authorized with the proxy is authorized to use the resource. However, if the proxy is compromised, a bad actor has full access to the cluster. This patch is the first part of a fix for apache#1991.
It turns out that the admin rest api proxy wasn't doing authentication at all, so if you can connect, you have access to the api at the same level as the configured user of the proxy, which could well be superuser access. Master Issue: apache#1991
So that the same authentication service can be used for the webserver also. Master issue: apache#1991
So that it can be used by the proxy also. Master issue: apache#1991
So that it can be used by the proxy also. Master issue: #1991
So that the same authentication service can be used for the webserver also. Master issue: #1991
* Original principal authorization for admin API The broker admin checks if the authenticated user is listed as a proxy user. If so, it checks looks for the header X-Original-Principal, and validates that the role is authorized to access the resource in question. There are two use cases: 1. The proxy role is a normal role. In this case, if a resource is to be used via the proxy, the proxy user must be explicitly granted permission on the resource. So, to use the admin api for a tenant, the proxy role must be listed as a tenant admin. 2. The proxy role is a superuser role. In this case, any resource can be used via the proxy as long as the user authorized with the proxy is authorized to use the resource. However, if the proxy is compromised, a bad actor has full access to the cluster. This patch is the first part of a fix for #1991. * Factor out some validation for original principal Also added a check that if original principal is set, it must have come from a role contained in proxyRoles.
So that it can be used by the proxy also. Master issue: #1991
It turns out that the admin rest api proxy wasn't doing authentication at all, so if you can connect, you have access to the api at the same level as the configured user of the proxy, which could well be superuser access. Master Issue: apache#1991
It turns out that the admin rest api proxy wasn't doing authentication at all, so if you can connect, you have access to the api at the same level as the configured user of the proxy, which could well be superuser access. Master Issue: #1991
|
This has been fixed by multiple pull requests linked above. |
Brokers are configured to use TLS authentication
Proxies cannot connect to the TLS authentication required brokers, because the brokers are advertising their address as :8080, not :8443
The text was updated successfully, but these errors were encountered: