Setup Openshift clusters on PSI for use by the team

[mohammedzee1000]

[kind/Enhancement]

Linked to #1799
Initiate creation of cluster on PSI resources

Acceptance Criteria

• Create Controller node
• Setup ocp 3.11 cluster
• Setup ocp 4.1 cluster
• No DNS setup should be required for clients connecting to the cluser.

[mohammedzee1000]

Sorted out a considerable number of issues in cluster bring up

[ppitonak]

@mohammedzee1000 I can provision 4.2, ping me if you need help

[mohammedzee1000]

@ppitonak sure i was able to successfully bring up 3.11 cluster

[amitkrout]

/assign @amitkrout

[kadel]

• Create Controller node

what is Controller node?

[kadel]

• No DNS setup should be required for clients connecting to the cluser.

added one additional important Acceptance criteria

[mohammedzee1000]

I have working on updating the scripts provided by pavol to setup 4.x cluster on PSI. The scripts make use of aws route 53 by default to create route to cluster (basically if aws command works on your system and you have a DNS there under your control the scripts will work with minor modifications to a couple of files).

I have spent some time creating copies of the script with DNS config stripped out and I have it mostly working as well.

However, DNS is going to a big question here.

As Openshift clusters map dns to services/pods in the cluster, but Openshift itself does not do DNS resolution. This will need to be handled externally with DNS server routing traffic to OpenShift Master (who knows how to map dns to respective services/pods)

We currently do have a DNS server on PSI but users are finding it a little hard configure dns server on their machine. So we might need to figure out something that does not need DNS config on client side.

So we could do aws route53 itself, external cost factor may be involved or maybe internal red hat DNS or something

[kadel]

As far as I know, there are two kinds of DNS in each Kubernetes cluster.
One is internal that that translates service names to endpoints ip, that one is usually handled by CoreDNS running inside the cluster. We don't have to worry about that one as this is never exposed to the outside and it is internal to the cluster.

The other DNS is one that handles DNS name resolution for Routes/Ingresses, api endpoint and console. This one is an external one. This is where I suspect is AWS Route53 used. We shouldn't need Route53 for it. All we need is to have a domain for which we can create a wildcard DNS record, something like this:

odo.cluster.com          A        10.11.12.13
*.apps.odo.cluster.com   CNAME    odo.cluster.com

[ppitonak]

I think that @kadel described it correctly. You don't need Route53, any DNS server that has API or can be configured in non-interactive way is OK. We used Route53 as it was the quickest and cheapest solution (<$1 per month)

IMHO the cleanest solution would be to deploy Designate in PSI.

[mohammedzee1000]

Yea, although we will need to ensure that the server does not need to be explicitly configured on client side. If you are on the Red Hat Network, then you should be able to get to any dns there

[kadel]

I think that @kadel described it correctly. You don't need Route53, any DNS server that has API or can be configured in non-interactive way is OK. We used Route53 as it was the quickest and cheapest solution (<$1 per month)

IMHO the cleanest solution would be to deploy Designate in PSI.

Our goal is to create a long-lived cluster. It should be ok to use even a DNS that requires interactive configuration, as it will be just a one-time thing for us.

Yea, although we will need to ensure that the server does not need to be explicitly configured on client-side. If you are on the Red Hat Network, then you should be able to get to any dns there

If there is internal DNS that we can use great. That will be a clean solution. But we might be able to get away with using an external domain with dns, for example, we already own openshfitdo.org domain. We can try to use that. It won't be up the DNS standard and it is probably against some RFC, as private and unreachable IPs should never appear in public DNS records, but it should work.

[kadel]

@mohammedzee1000 Any updates on this?
This is getting more and more important.

/priority high
/remove-priority medium

[mohammedzee1000]

i will be working on getting the 4.x cluster on top priority this sprint
cc @girishramnani

[mohammedzee1000]

I have registered domain psiodo.net on aws route53. Waiting on registration to be complete before I setup the permanent cluster.

Once PSI problems listed below are sorted then we can move away from route53

• Install designate for PSI allowing us to use openstack cli to setup domains

[mohammedzee1000]

@kadel ^

[mohammedzee1000]

I think that @kadel described it correctly. You don't need Route53, any DNS server that has API or can be configured in non-interactive way is OK. We used Route53 as it was the quickest and cheapest solution (<$1 per month)

IMHO the cleanest solution would be to deploy Designate in PSI.

@ppitonak but you still got to register for a year :(
Also I have asked dv to talk to PSI guys about setting up designate

[mohammedzee1000]

We have clusters now for 4.3.9 and 3.11 with aws route53 dns
The last thing is to fix the swift quota but that is not a blocker for this issue.

[girishramnani]

Ac of this issue is satisfied so closing this