Add support for HTTPs when using NodePort. #1688

OndrejHome · 2024-01-21T12:33:57Z

SUMMARY

Add support for HTTPs when using NodePort.

Introduces two new options:

nodeport_protocol - selectable http(default)/https
nodeport_tls_secret - name of secret containing TLS certificate and key

When nodeport_protocol is set to https the nodeport_tls_secret must contain name of secret containing TLS key and certificate that will be used by nginx in AWX web container exposing HTTPS traffic through NodePort.

Github issues asking for https on nodeport support: #1559, #1563

ISSUE TYPE

New or Enhanced Feature

ADDITIONAL INFORMATION

Tested on kubernetes 1.28.5, deploying AWX 23.5.1.
Unless the nodeport_protocol: https is specified in spec there should be no change in behaviour (NodePort will be HTTP without encryption).
When nodeport_protocol: https is specified in spec then code will add secret specified in nodeport_tls_secret to web container so that nginx running inside can read both TLS key and certificate from it. Additionally nginx configuration in such case will include SSL configuration on port 8053 and the service pointing to web container will point to port 8053 where HTTPS is exposed instead of the port 8052 with HTTP traffic.

A lot of changes are re-using logic from Route TLS configuration. Documentation was updated as well to contain both original example with NodePort operating in HTTP mode and new changed configuration showing the use of HTTPS.

Why is this change useful? To allow kubernetes without ingress to expose AWX web interface via HTTPS - this allows the awx.awx ansible modules to be used as they require the encrypted connection (plain HTTP will not work with them).

Comment and suggestions are welcomed. My responses may be delayed during weekdays.

Introduces two new options: - 'nodeport_protocol' - selectable 'http'(default)/'https' - 'nodeport_tls_secret' - name of secret containing TLS certificate and key When 'nodeport_protocol' is set to 'https' the 'nodeport_tls_secret' must contain name of secret containing TLS key and certifacate that will be used by nginx in AWX web container exposing HTTPS traffic through NodePort. Github issues asking for https on nodeport support: ansible#1559, ansible#1563

rooftopcellist · 2024-01-24T19:17:47Z

roles/installer/templates/networking/service.yaml.j2

    - port: 80
      protocol: TCP
      targetPort: 8052
      name: http
 {% if nodeport_port is defined %}
      nodePort: {{ nodeport_port }}
 {% endif %}
+{% elif service_type | lower == "nodeport" and nodeport_protocol | lower == "https" %}
+    - port: 80


Shouldn't this be 443?

Thank you, that is nice catch! yes it should be 443 for consistency. Now addressed in commit 778b32d

rooftopcellist · 2024-01-24T19:19:47Z

I haven't looked closely yet, but we should double-check and make sure that there are not any changes needed in the nginx.conf:

https://github.com/ansible/awx-operator/blob/devel/roles/installer/templates/configmaps/config.yaml.j2#L130

OndrejHome · 2024-01-28T13:34:25Z

I haven't looked closely yet, but we should double-check and make sure that there are not any changes needed in the nginx.conf:
* https://github.com/ansible/awx-operator/blob/devel/roles/installer/templates/configmaps/config.yaml.j2#L130

Yes, this PR is adjusting the nginx.conf - more specifically the "https nodeport" is using same config as the "route passthrough". Or you mean if there are any changes needed inside (lines 131-139) ?

libpoet1312 · 2024-07-01T22:14:43Z

Any progress on this?

OndrejHome · 2024-07-07T05:34:00Z

@rooftopcellist, Are there any further changes needed for this PR to be merged?

For completeness I have retested the original submission and today (2024-07-07) I have tried on different branch to merge same changes with current devel of awx-operator (no conflicts). In short both works for me (on kubernetes 1.30.1).

For those wanting to test out the operator image for this PR, instructions below can be adjusted and used. Note you will have to provide the PV or storage AND the SSL certificate for use with https nodeport yourself.

Operator container images build by me can be temporarily found (without authentication, https-only) on following URLs:

kr.famera.cz/tmp-registry/awx-operator:2.19.1-pr1688
kr.famera.cz/tmp-registry/awx-operator:2.10.0-pr1688

# git clone -b nodeport-https https://github.com/OndrejHome/awx-operator pr1688-2024-01-28
# cd pr1688-2024-01-28
# IMG=kr.famera.cz/tmp-registry/awx-operator:2.10.0-pr1688 make deploy
# kubectl create -f /root/01_local_pv01.yaml
# kubectl create -n awx secret tls sample-tls-secret --cert=/root/test.crt --key=/root/test.key
# cat > awx.yaml <<EOF
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: NodePort
  nodeport_port: 30443
  nodeport_protocol: https
  nodeport_tls_secret: sample-tls-secret
  admin_user: admin
  admin_email: root@localhost 
  postgres_storage_class: awx-storage
EOF
# kubectl create -n awx -f awx.yaml

# podman images |grep -E 'awx|postgres|redis|IMAGE'
REPOSITORY                               TAG              IMAGE ID      CREATED         SIZE
kr.famera.cz/tmp-registry/awx-operator   2.10.0-pr1688    537fedc630e5  39 minutes ago  576 MB
quay.io/ansible/awx-ee                   latest           6b7885a2e2c8  2 hours ago     1.84 GB
docker.io/library/redis                  7                9c893be668ac  6 weeks ago     120 MB
docker.io/library/postgres               13               2cf221a08d3b  8 weeks ago     427 MB
quay.io/ansible/awx                      23.6.0           0aba4f714265  6 months ago    908 MB

# kubectl get svc/awx-service -n awx
NAME          TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
awx-service   NodePort   10.101.232.251   <none>        443:30443/TCP   5m53s
# curl -s http://10.101.232.251:443 |head -2
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
# curl -sk https://10.101.232.251:443 |head -2
<!doctype html><html lang="en"><head><script nonce="Y19ylwNoRDbfW4ADBDK4Gj/tPPii8icOFJ26cDx+fM0=" type="text/javascript">window.NONCE_ID="Y19ylwNoRDbfW4ADBDK4Gj/tPP...

# kubectl delete -n awx -f awx.yaml
# IMG=kr.famera.cz/tmp-registry/awx-operator:2.10.0-pr1688 make undeploy
# kubectl delete -f /root/01_local_pv01.yaml
# rm -rf /var/container_data/pv01/*

# git clone -b nodeport-https-2024-07-07 https://github.com/OndrejHome/awx-operator pr1688-2024-07-07
# cd pr1688-2024-07-07
# IMG=kr.famera.cz/tmp-registry/awx-operator:2.19.1-pr1688 make deploy
# kubectl create -f /root/01_local_pv01.yaml
# kubectl create -n awx secret tls sample-tls-secret --cert=/root/test.crt --key=/root/test.key
# cat > awx.yaml <<EOF
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: NodePort
  nodeport_port: 30443
  nodeport_protocol: https
  nodeport_tls_secret: sample-tls-secret
  admin_user: admin
  admin_email: root@localhost 
  postgres_storage_class: awx-storage
EOF
# kubectl create -n awx -f awx.yaml

# podman images |grep -E 'awx|postgres|redis|IMAGE'
REPOSITORY                               TAG              IMAGE ID      CREATED         SIZE
kr.famera.cz/tmp-registry/awx-operator   2.19.1-pr1688    c7f4c2ea5e16  46 minutes ago  625 MB
quay.io/ansible/awx-ee                   latest           6b7885a2e2c8  2 hours ago     1.84 GB
quay.io/sclorg/postgresql-15-c9s         latest           951f17818a33  2 days ago      418 MB
quay.io/ansible/awx                      24.6.1           87ab0ba4bf68  4 days ago      1.02 GB
docker.io/library/redis                  7                9c893be668ac  6 weeks ago     120 MB

# kubectl get svc/awx-service -n awx
NAME          TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
awx-service   NodePort   10.96.105.20   <none>        443:30443/TCP   106s
# curl -s http://10.96.105.20:443 |head -2
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
# curl -sk https://10.96.105.20:443 |head -2
<!doctype html><html lang="en"><head><script nonce="PxO/BfnrT6NxX3o4J/myMQ1SP5lr3qb8wjbPuyRR6Kc=" type="text/javascript">window.NONCE_ID="PxO/BfnrT6NxX3o4J/myMQ1SP5lr3q...

# kubectl delete -n awx -f awx.yaml
# IMG=kr.famera.cz/tmp-registry/awx-operator:2.19.1-pr1688 make undeploy
# kubectl delete -f /root/01_local_pv01.yaml
# rm -rf /var/container_data/pv01/*

# cat > awx.yaml <<EOF
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: NodePort
  nodeport_port: 30080
  admin_user: admin
  admin_email: root@localhost 
  postgres_storage_class: awx-storage
EOF
# kubectl create -n awx -f awx.yaml

# kubectl get svc/awx-service -n awx
NAME          TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
awx-service   NodePort   10.97.71.21   <none>        80:30080/TCP   9s
# curl -s http://10.97.71.21:80 |head -2
<!doctype html><html lang="en"><head><script nonce="JgTcHoaSBDg8rl9uFvwVZ0Ed74yImaiCCoxsiO3FHFs=" type="text/javascript">window.NONCE_ID="JgTcHoaSBDg...
# curl -sk https://10.97.71.21:80 |head -2
(no output)
# curl -k https://10.97.71.21:80
curl: (35) OpenSSL/3.2.1: error:0A0000C6:SSL routines::packet length too long

# kubectl delete -n awx -f awx.yaml
# IMG=kr.famera.cz/tmp-registry/awx-operator:2.19.1-pr1688 make undeploy
# kubectl delete -f /root/01_local_pv01.yaml
# rm -rf /var/container_data/pv01/*

github-actions bot added the community label Jan 21, 2024

OndrejHome mentioned this pull request Jan 21, 2024

awx-web to use https using NodePort and SSL #1563

Open

9 tasks

rooftopcellist reviewed Jan 24, 2024

View reviewed changes

fix nodeport https 'port:' 80->443

778b32d

Merge branch 'devel' into nodeport-https

304c59f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for HTTPs when using NodePort. #1688

Add support for HTTPs when using NodePort. #1688

OndrejHome commented Jan 21, 2024

rooftopcellist Jan 24, 2024

OndrejHome Jan 28, 2024

rooftopcellist commented Jan 24, 2024

OndrejHome commented Jan 28, 2024

libpoet1312 commented Jul 1, 2024

OndrejHome commented Jul 7, 2024

Add support for HTTPs when using NodePort. #1688

Are you sure you want to change the base?

Add support for HTTPs when using NodePort. #1688

Conversation

OndrejHome commented Jan 21, 2024

SUMMARY

ISSUE TYPE

ADDITIONAL INFORMATION

rooftopcellist Jan 24, 2024

Choose a reason for hiding this comment

OndrejHome Jan 28, 2024

Choose a reason for hiding this comment

rooftopcellist commented Jan 24, 2024

OndrejHome commented Jan 28, 2024

libpoet1312 commented Jul 1, 2024

OndrejHome commented Jul 7, 2024