Skip to content
This repository has been archived by the owner on Nov 16, 2022. It is now read-only.

KEYCLOAK-12929 external database support with uri #137

Merged
merged 1 commit into from
Feb 25, 2020

Conversation

pb82
Copy link
Contributor

@pb82 pb82 commented Feb 9, 2020

JIRA ID

https://issues.redhat.com/browse/KEYCLOAK-12929

ping @davidffrench

Additional Information

The Keycloak Operator allows connecting to external (outside of the cluster) databases by providing a pre-populated database secret. This only worked when specifying the IP Address of the external database. This is a problem because the IP address can change. We want to support both, IP and hostname.

Unfortunately IP adresses and hostnames have to be treated in different ways: Hostnames are supported by externalName type services. But this type of service does not support IP addresses and so we have to rely on default ClusterIP type services and an Endpoints object to support that use case. This PR checks if the pre-existing database secret contains an IP or a hostname and chooses the reconciliation strategy accordingly.

Verification Steps

A number of scenarios have to be verified. For every scenario, make sure that the deployment succeeds and that there are no errors in the log. Feel free to ask me for the connection details / db secret of an external database if you need one.

  1. Deploy Keycloak without an external database.
  2. Deploy Keycloak with an external database using an IP address
  3. Deploy Keycloak with an external database using a hostname
  4. Deploy RHSSO without an external database
  5. Deploy RHSSO with an external database using an IP address
  6. Deploy RHSSO with an external database using a hostname

Checklist:

  • Verified by team member
  • Comments where necessary
  • Automated Tests
  • Documentation changes if necessary

@pb82 pb82 requested a review from slaskawi February 9, 2020 13:57
@coveralls
Copy link

coveralls commented Feb 9, 2020

Coverage Status

Coverage decreased (-0.9%) to 40.831% when pulling d65c4b9 on pb82:KEYCLOAK-12929 into f6bdebd on keycloak:master.

Copy link
Contributor

@davidffrench davidffrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pb82 This is fantastic work, thank you. Two small changes from my review. Will verify once these are made.

pkg/model/keycloak_deployment.go Outdated Show resolved Hide resolved
@pb82 pb82 force-pushed the KEYCLOAK-12929 branch 2 times, most recently from a2151ab to 78aed12 Compare February 10, 2020 11:58
@slaskawi
Copy link
Contributor

@pb82 Kubernetes Service manual mentions this:

Note: ExternalName accepts an IPv4 address string, but as a DNS names comprised of digits, not as an IP address. ExternalNames that resemble IPv4 addresses are not resolved by CoreDNS or ingress-nginx because ExternalName is intended to specify a canonical DNS name. To hardcode an IP address, consider using headless Services.

So if my understanding is correct - they say if this is a DNS name, we should use spec.externalName and if it's an IP address - using a Headless Service is a better option.

In the same manual I also found this:

If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those externalIPs. Traffic that ingresses into the cluster with the external IP (as destination IP), on the Service port, will be routed to one of the Service endpoints. externalIPs are not managed by Kubernetes and are the responsibility of the cluster administrator.

So it seems for the IP addresses, we could use spec.externalIPs

Did you try this options? I'm asking because the solution you proposed seems pretty complicated with lots of corner cases. PostgresqlService now takes 3 arguments, which lights a yellow bulb in my head. Also generating Env vars for Keycloak/RHSSO is not so clean anymore and I'm afraid it will be harder and harder to read this code.

If we could get rid of the Endpoints object and implement everything in a single Service (just decide in the runtime, whether it needs a Selector, a spec.externalIPs or spec.externalName - that would be great!

@davidffrench
Copy link
Contributor

I have verified this PR works with an external AWS RDS Postgres instance

@pb82
Copy link
Contributor Author

pb82 commented Feb 10, 2020

@slaskawi I tried using externalIPs but unfortunately this does not work. It looks like they are only meant for node IPs: If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those externalIPs. Looks like we are stuck using two different strategies for IPs vs. Hostnames.

Copy link
Contributor

@slaskawi slaskawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work Peter!

I added a few comments and there two things I would like to you to change:

  • Always add SERVICE_PORT and SERVICE_HOST variables if possible. Generating datastores for RHSSO is very fragile (and overly too complicated). Adding something conditionally may lead to problems with reproducing bugs. I would like to avoid it, if possible.
  • Both PostgresqlService and getPostgresqlServiceDesiredState take bool as the third parameter. In both of the cases, this look awkward. Is it possible to reduce this number of parameters and get rid of it? Maybe it should be embedded into those functions?

pkg/model/rhsso_deployment.go Show resolved Hide resolved
if clusterState.DatabaseSecret == nil {
return
}
if model.IsIP(clusterState.DatabaseSecret.Data[model.DatabaseSecretExternalAddressProperty]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could use methods from util.go here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IsIP is actually just a wrapper around a Go stdlib function. Or do you mean we should move this into the util package?

// set up an endpoints object for the service to send traffic. An externalName
// type service won't work in this case. For more details, see https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-mapping-external-services
desired.AddAction(i.getPostgresqlServiceEndpointsDesiredState(clusterState, cr))
desired.AddAction(i.getPostgresqlServiceDesiredState(clusterState, cr, false))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like passing this true or false (3rd argument) to getPostgresqlServiceDesiredState. It looks a bit awkward - could we somehow infer this argument from the CR inside getPostgresqlServiceDesiredState?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is, true or false here do not enable/disable external database mode but service type. We could infer them by checking for IP again in the service model. Or we could have two models, one for the default clusterIP service and one for the externalName service.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I would implement 2 models. I think it would be a bit cleaner this way.

But I'll leave the final decision to you. I'm fine either way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having two models would complicate it even more: now we would have a cluster state that either knows about the two models or needs to be able to distinguish between what kind of model is required.

@keycloak keycloak deleted a comment from slaskawi Feb 11, 2020
@keycloak keycloak deleted a comment from pb82 Feb 11, 2020
cmd/manager/main.go Outdated Show resolved Hide resolved
stianst
stianst previously approved these changes Feb 19, 2020
slaskawi
slaskawi previously approved these changes Feb 19, 2020
Copy link
Contributor

@slaskawi slaskawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - @pb82 Could you please rebase it? Once it's rebase, I'll review it and @stianst will merge it.

@pb82
Copy link
Contributor Author

pb82 commented Feb 19, 2020

@slaskawi it's rebased now

@pb82
Copy link
Contributor Author

pb82 commented Feb 19, 2020

ping @slaskawi , rebased again

Copy link
Contributor

@slaskawi slaskawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work @pb82 !

@stianst May I ask you to merge this one?

@stianst stianst merged commit 979290a into keycloak:master Feb 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants