Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

https route does not work as expected #1071

Closed
hkmo99 opened this issue Jan 24, 2024 · 3 comments
Closed

https route does not work as expected #1071

hkmo99 opened this issue Jan 24, 2024 · 3 comments

Comments

@hkmo99
Copy link

hkmo99 commented Jan 24, 2024

Description of the problem

When I define a simple Kubernetes Gateway, HTTPRoute, Service and EndpointSlice for http requests, the haproxy http frontend is correctly generated and the requests work fine. However, if I replace http with https the https frontend is wrongly generated and the requests get 404.

Expected behavior

https frontend is correctly generated (see below expected frontend).
https request "curl -v https://example.api.net" should return 200 OK.

Steps to reproduce the problem

helm install haproxy-ingress haproxy-ingress/haproxy-ingress  --create-namespace \
                         --namespace ingress-controller  --version 0.14.5  -f haproxy-ingress-values.yaml
kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v0.4.1" | kubectl apply -f -

--
more haproxy-ingress-values.yaml
controller:
  hostNetwork: true
  ingressClassResource:
    enabled: true
  extraArgs:
    enable-endpointslices-api: "true"
    default-ssl-certificate: "default/haproxy-tls-secret"
    log-zap: "true"
    log-dev: "true"
    allow-cross-namespace: "true"
  config:
    log-level: "debug"
--

more my_gatewayclass.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: GatewayClass
metadata:
  name: haproxy
spec:
  controllerName: haproxy-ingress.github.io/controller

--
more my_gateway.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
  name: haproxy-gateway
  namespace: default
spec:
  gatewayClassName: haproxy
  listeners:
  - name: https-listener
    port: 443
    protocol: HTTPS

--
more my_httproute.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
  labels:
    gateway: haproxy
  name: httpbin-example-httproute
  namespace: default
spec:
  hostnames:
  - example.api.net
  parentRefs:
  - kind: Gateway
    name: haproxy-gateway
  rules:
  - backendRefs:
    - kind: Service
      name: httpbin-example-proxy
      port: 443
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /
        
--
more my_service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
  name: httpbin-example-proxy
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP

--
 more my_endpointslice.yaml
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  labels:
    kubernetes.io/service-name: httpbin-example-proxy
  name: httpbin-example-endpointslice-1
  namespace: default
addressType: IPv4
endpoints:
- addresses:
  - 192.168.76.2
  conditions:
    ready: true
ports:
- name: https
  port: 31590
  protocol: TCP

kubectl apply -f my_gatewayclass.yaml
kubectl apply -f my_gateway.yaml
kubectl apply -f my_httproute.yaml
kubectl apply -f my_service.yaml
kubectl apply -f my_endpointslice.yaml

Environment information

HAProxy Ingress version: v0.14.5

Expected https frontend:

frontend _front_https
    mode http
    bind :443 ssl alpn h2,http/1.1 crt-list /etc/haproxy/maps/_front_bind_crt.list ca-ignore-err all crt-ignore-err all
    http-request set-var(req.path) path
    http-request set-var(req.host) hdr(host),field(1,:),lower
    http-request set-var(req.base) var(req.host),concat(\#,req.path)
    http-request set-header X-Forwarded-Proto https
    http-request del-header X-SSL-Client-CN
    http-request del-header X-SSL-Client-DN
    http-request del-header X-SSL-Client-SHA1
    http-request del-header X-SSL-Client-SHA2
    http-request del-header X-SSL-Client-Cert
    http-request set-var(req.hostbackend) var(req.base),map_dir(/etc/haproxy/maps/_front_https_host__prefix.map)
    use_backend %[var(req.hostbackend)] if { var(req.hostbackend) -m found }
    default_backend _error404

Generated https frontend:

frontend _front_https
    mode http
    bind :443 ssl alpn h2,http/1.1 crt-list /etc/haproxy/maps/_front_bind_crt.list ca-ignore-err all crt-ignore-err all
    http-request set-var(req.path) path
    http-request set-var(req.host) hdr(host),field(1,:),lower
    http-request set-var(req.base) var(req.host),concat(\#,req.path)
    http-request set-header X-Forwarded-Proto https
    http-request del-header X-SSL-Client-CN
    http-request del-header X-SSL-Client-DN
    http-request del-header X-SSL-Client-SHA1
    http-request del-header X-SSL-Client-SHA2
    http-request del-header X-SSL-Client-Cert
    use_backend %[var(req.hostbackend)] if { var(req.hostbackend) -m found }
    default_backend _error404

Expected "_front_https_host__prefix.map":

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# #
# #   HAProxy Ingress Controller
# #   --------------------------
# #   This file is automatically updated, do not edit
# #
#
example.api.net#/ default_httpbin-example-httproute__rule0

The problem is that "_front_https_host__prefix.map" was not generated at all, which causes the 404.

Is there any workaround or fix to resolve this issue?
Thanks

@hkmo99
Copy link
Author

hkmo99 commented Jan 27, 2024

Adding "tls" in the gateway allows to generate the expected "_front_https_host__prefix.map":

## add the following listener to the Gateway resource
  - name: https-listener
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      certificateRefs:
        kind: Secret
        group: core
        name: haproxy-tls-secret

kubectl apply -f haproxy_gateway.yaml

The "tls" configuration in gateway listener should be enough, but I had to add the command-line parameter "default-ssl-certificate" to make the frontend tls use-case work. Maybe something to improve at this level to make the configuration self-contained in one place.

@jcmoraisjr
Copy link
Owner

Hi, in fact Gateway API is not fully implemented on our side, so we're not properly updating status (so you don't know what's going wrong). Regarding the HTTPS configuration, we already have the info in place:

Gateway’s Listener Port and Protocol wasn’t implemented - Port uses the global bind-port configuration and Protocol is based on the presence or absence of the TLS attribute.

Source: doc

Regarding the self-contained configuration, if I understood you correctly, a gateway resource has the ability to configure just a single hostname, and the certificate for that single hostname. If you check the hostname certificate:

$ :|openssl s_client -servername example.api.net -connect 192.168.107.2:443 |openssl x509 -noout -dates -issuer -subject
depth=0 CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = localhost
verify return:1
DONE
notBefore=May  4 17:38:25 2024 GMT
notAfter=Jun  3 17:38:25 2024 GMT
issuer= /CN=localhost
subject= /CN=localhost

you can see that haproxy uses the one I configured in the listener. But if however you need to use the same one as default if a hostname doesn't match or is not provided, you should configure it via --default-ssl-certificate, otherwise a self generated fake one is used.

Copy link

This issue got stale and will be closed in 7 days.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants