Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 49 additions & 117 deletions content/en/docs/deployment/private-cloud/private-cloud-cluster/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,123 +216,7 @@ If an app's MDA was built using a newer Java version, Mendix Operator 2.15.0 (an

### Endpoint (network) Configuration {#advanced-network-settings}

The OperatorConfiguration contains the following user-editable options for network configuration:

When using **Ingress** for network endpoints:

```yaml
apiVersion: privatecloud.mendix.com/v1alpha1
kind: OperatorConfiguration
# ...
# omitted lines for brevity
# ...
spec:
# Endpoint (Network) configuration
endpoint:
# Endpoint type: ingress, openshiftRoute or service
type: ingress
# Optional, can be omitted: Service annotations
serviceAnnotations:
# example: custom AWS CLB configuration
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:account:certificate/id
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
# Ingress configuration: used only when type is set to ingress
ingress:
# Optional, can be omitted: annotations which should be applied to all Ingress Resources
annotations:
# default annotation: allow uploads of files up 500 MB in the NGINX Ingress Controller
nginx.ingress.kubernetes.io/proxy-body-size: 500m
# example: use the specified cert-manager ClusterIssuer to generate TLS certificates with Let's Encrypt
cert-manager.io/cluster-issuer: staging-issuer
# example: deny access to /rest-doc
nginx.ingress.kubernetes.io/configuration-snippet: |
location /rest-doc {
deny all;
return 403;
}
# App URLs will be generated for subdomains of this domain, unless an app is using a custom appURL
domain: mendix.example.com
# Enable or disable TLS
enableTLS: true
# Optional: name of a kubernetes.io/tls secret containing the TLS certificate
# This example is a template which lets cert-manager to generate a unique certificate for each app
tlsSecretName: '{{.Name}}-tls'
# Optional: specify the Ingress class name
ingressClassName: alb
# Optional, can be omitted : specify the Ingress path
path: "/"
# Optional, can be omitted : specify the Ingress pathType
pathType: ImplementationSpecific
# ...
# omitted lines for brevity
# ...
```

When using **OpenShift Routes** for network endpoints:

```yaml
apiVersion: privatecloud.mendix.com/v1alpha1
kind: OperatorConfiguration
spec:
# Endpoint (Network) configuration
endpoint:
# Endpoint type: ingress, openshiftRoute, or service
type: openshiftRoute
# OpenShift Route configuration: used only when type is set to openshiftRoute
openshiftRoute:
# Optional, can be omitted: annotations which should be applied to all Ingress Resources
annotations:
# example: use HSTS headers
haproxy.router.openshift.io/hsts_header: max-age=31536000;includeSubDomains;preload
# Optional: App URLs will be generated for subdomains of this domain, unless an app is using a custom appURL
domain: mendix.example.com
# Enable or disable TLS
enableTLS: true
# Optional: name of a kubernetes.io/tls secret containing the TLS certificate
# This example is the name of an existing secret, which should be a wildcard matching subdomains of the domain name
tlsSecretName: 'mendixapps-tls'
```

When using **Services** for network endpoints (without an Ingress or OpenShift route):

```yaml
apiVersion: privatecloud.mendix.com/v1alpha1
kind: OperatorConfiguration
spec:
# Endpoint (Network) configuration
endpoint:
# Endpoint type: ingress, openshiftRoute, or service
type: service
# Optional, can be omitted: the Service type
serviceType: LoadBalancer
# Optional, can be omitted: Service annotations
serviceAnnotations:
# example: annotations required for AWS NLB
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
# Optional, can be omitted: Service ports
servicePorts:
- 80
- 443
```

You can change the following options:

* **type**: – select the Endpoint type, possible options are `ingress`, `openshiftRoute` and `service`; this parameter is also configured through the **Configuration Tool**
* **ingress**: - specify the Ingress configuration, required when **type** is set to `ingress`
* **openshiftRoute**: - specify the OpenShift Route configuration, required when **type** is set to `openshiftRoute`
* **annotations**: - optional, can be used to specify the Ingress or OpenShift Route annotations, can be a template: `{{.Name}}` will be replaced with the name of the CR for the Mendix app, and {{.Domain}} will be replaced with the application's domain name
* **serviceAnnotations**: - optional, can be used to specify the Service annotations, can be a template: `{{.Name}}` will be replaced with the name of the CR for the Mendix app, and {{.Domain}} will be replaced with the application's domain name
* **ingressClassName**: - optional, can be used to specify the Ingress Class name
* **path**: - optional, can be used to specify the Ingress path; default value is `/`
* **pathType**: - optional, can be used to specify the Ingress pathType; if not set, no pathType will be specified in Ingress objects
* **domain**: - optional for `openshiftRoute`, required for `ingress`, used to generate the app domain in case no app URL is specified; if left empty when using OpenShift Routes, the default OpenShift `apps` domain will be used; this parameter is also configured through the **Configuration Tool**
* **enableTLS**: - allows you to enable or disable TLS for the Mendix App's Ingress or OpenShift Route
* **tlsSecretName**: - optional name of a `kubernetes.io/tls` secret containing the TLS certificate, can be a template: `{{.Name}}` will be replaced with the name of the CR for the Mendix app; if left empty, the default TLS certificate from the Ingress Controller or OpenShift Router will be used
* **serviceType**: - can be used to specify the Service type, possible options are `ClusterIP` and `LoadBalancer`; if not specified, Services will be created with the `ClusterIP` type
* **servicePorts**: - can be used to specify a list of custom ports for the Service; if not specified, Services will use be created with port `8080`
For information on using advanced network configuration settings, see [Network Ingress Settings section](/developerportal/deploy/private-cloud-cluster/private-cloud-ingress-settings/).

{{% alert color="info" %}}
When switching between Ingress and OpenShift Routes, you need to [restart the Mendix Operator](#restart-after-changing-network-cr) for the changes to be fully applied.
Expand Down Expand Up @@ -938,6 +822,54 @@ Alternatively, for Standalone clusters, pod labels can be specified in the `Mend
The Mendix Operator uses some labels for internal use. To avoid conflicts with these internal pod labels, please avoid using labels starting with the `privatecloud.mendix.com/` prefix.
{{% /alert %}}

### Pod Annotations (General) {#pod-annotations}

Mendix Operator version 2.27.0 or above allows you to specify default pod annotations for task pods (build and storage provisioners) and runtime (app) pods.

To specify the default pod annotations for a namespace, specify them in `customPodAnnotations.general` in `OperatorConfiguration`:

```yaml
apiVersion: privatecloud.mendix.com/v1alpha1
kind: OperatorConfiguration
spec:
# ...
# Other configuration options values
# Optional: custom pod annotations
customPodAnnotations:
# Optional: general pod annotations (applied to all app-related pods)
general:
# Example: use FQDN instead of IP addresses when communicating with the Kubernetes API server
kubernetes.azure.com/set-kube-service-host-fqdn: "true"
```

Alternatively, for Standalone clusters, pod annotations for an app can be specified in the `MendixApp` CR.

{{% alert color="warning" %}}
The Mendix Operator uses some annotations for internal use. To avoid conflicts with these internal pod annotations, please avoid using labels starting with the `privatecloud.mendix.com/` prefix.
{{% /alert %}}

### Node Selector (General) {#node-selector}

Mendix Operator version 2.27.0 or above allows you to specify the default `nodeSelector` for task pods (build and storage provisioners) and runtime (app) pods.

To configure the default pod `nodeSelector` for a namespace, specify them in `customPodNodeSelector.general` in `OperatorConfiguration`:

```yaml
apiVersion: privatecloud.mendix.com/v1alpha1
kind: OperatorConfiguration
spec:
# ...
# Other configuration options values
# Optional: custom pod nodeSelector
customPodNodeSelector:
# Optional: general pod nodeSelector (applied to all app-related pods)
general:
# Example: use Amazon EKS Auto Mode
eks.amazonaws.com/compute-type: auto
```

Alternatively, for Standalone clusters, pod `nodeSelector` configuration can be specified in the `MendixApp` CR for a specific app.

### Delaying App Shutdown {#termination-delay}

In some situations, shutting down a replica immediately can cause isses. For example, the [Azure Gateway Ingress Controller](https://azure.github.io/application-gateway-kubernetes-ingress/how-tos/minimize-downtime-during-deployments/) needs up to 90 seconds to remove a pod from its routing table. Stopping an app pod immediately would still send traffic to the pod for a few minutes, causing random 502 errors to appear in the client web browser.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ weight: 10

## Introduction

Based on your organization's networking and security needs, you must configure the available networking components of your Mendix web applications to be accessible to end users, either over a private network or the Internet. This article describes three possible network configurations: Kubernetes Ingress, OpenShift Routes, and Service-Only. Each configuration is tailored to meet specific use cases.
Based on your organization's networking and security needs, you must configure the available networking components of your Mendix web applications to be accessible to end users, either over a private network or the Internet. This article describes three possible network configurations: Kubernetes Ingress, Gateway Routes, OpenShift Routes, and Service-Only. Each configuration is tailored to meet specific use cases.

## Prerequisites

Expand All @@ -22,6 +22,8 @@ Mendix on Kubernetes supports the following Ingress configurations:

{{< figure src="/attachments/deployment/private-cloud/private-cloud-cluster/private-cloud-networking/k8s-ingress.png" class="no-border" >}}

* [Gateway Routes](/developerportal/deploy/private-cloud-cluster/private-cloud-ingress-settings/gateway-route/) - A next generation Kubernetes standard for Ingresses and load balancing. Supported by Mendix Operator v2.27.0 or later versions.

* [OpenShift Routes](/developerportal/deploy/private-cloud-cluster/private-cloud-ingress-settings/openshift/) - The OpenShift-native method for exposing services externally provides a more streamlined setup, though it offers less flexibility compared to Kubernetes Ingress.

{{< figure src="/attachments/deployment/private-cloud/private-cloud-cluster/private-cloud-networking/openshift-routes.png" class="no-border" >}}
Expand All @@ -42,17 +44,17 @@ The following table compares the functionality of the three supported options, i
When switching between Ingress, OpenShift Routes, and Service Only, you must restart the Mendix Operator for the changes to be fully applied.
{{% /alert %}}

| Feature | Kubernetes Ingress | Openshift Routes | Service Only |
| --- | --- | --- | --- |
| Ease of use | Requires setup, but offers more flexibility. | Simplest option for OpenShift users, built-in. | Fine-grained control over networking and security offers maximum flexibility, but requires significant effort and expertise to configure and maintain. Note that the networking setup beyond the Mendix Operator's scope, up to the service object, is not supported by Mendix. |
| Native cloud integration | Can integrate with cloud-native services like AWS Application Load Balancer. | No direct integration with cloud providers. | Full control over networking setup. |
| Performance and scalability | Scales with cloud load balancers, better supports horizontal scaling. | Limited to OpenShift Router performance. | Full flexibility - scales according to your load balancer and proxy setup. |
| Supported providers | [NGINX Ingress Controller](https://kubernetes.github.io/ingress-nginx/); [Traefik](https://traefik.io/traefik/); [AWS Application Load Balancer](https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html); [Ingress for External Application Load Balancer](https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb); [Azure Application Gateway Ingress Controller](https://learn.microsoft.com/en-us/azure/application-gateway/ingress-controller-overview) | [OpenShift Router (HAProxy-based Ingress Controller)](https://docs.openshift.com/container-platform/4.17/networking/networking_operators/ingress-operator.html) | None |
| TLS/SSL termination | Can leverage cloud provider-managed TLS; supports Cert-Manager. | Supported with OpenShift's HAProxy router. | Fully flexible but requires manual setup; TLS can terminate at application load balancer, network load balancer, or app level |
| Security (WAF, ACLs, Auth) | More advanced security integrations (for example AWS Web Application Firewall, authentication). | Basic access control via OpenShift OAuth. | Full control - can integrate with AWS Web Application Firewall, API Gateway, authentication proxies with manual configuration. |
| Traffic splitting | Fully supported through ingress rules. | Supported through HAProxy-based Route annotations. | Depends on external networking setup. |
| External DNS support | Some ingress controllers (for example, application load balancers) support native DNS updates. | Requires ExternalDNS integration. | Fully configurable - can use ExternalDNS, Amazon Route 53, or others. |
| Custom annotations | Supports Kubernetes Ingress annotations, provider-specific features.| Supports OpenShift-specific annotations. | No restrictions - fully customizable in external networking. |
| Feature | Kubernetes Ingress | Gateway Routes | Openshift Routes | Service Only |
| --- | --- | --- | --- | --- |
| Ease of use | Requires setup, but offers more flexibility. | Requires setup, offers flexibility with separation of scope by [roles and personas](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/) | Simplest option for OpenShift users, built-in. | Fine-grained control over networking and security offers maximum flexibility, but requires significant effort and expertise to configure and maintain. Note that the networking setup beyond the Mendix Operator's scope, up to the service object, is not supported by Mendix. |
| Native cloud integration | Can integrate with cloud-native services like AWS Application Load Balancer. | Can integrate with cloud-native services like AWS Application Load Balancer. | No direct integration with cloud providers. | Full control over networking setup. |
| Performance and scalability | Scales with cloud load balancers, better supports horizontal scaling. | Scales with cloud load balancers, better supports horizontal scaling. | Limited to OpenShift Router performance. | Full flexibility - scales according to your load balancer and proxy setup. |
| Supported providers | [NGINX Ingress Controller](https://kubernetes.github.io/ingress-nginx/); [Traefik](https://traefik.io/traefik/); [AWS Application Load Balancer](https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html); [Ingress for External Application Load Balancer](https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb); [Azure Application Gateway Ingress Controller](https://learn.microsoft.com/en-us/azure/application-gateway/ingress-controller-overview) | [Gateway API v1.4 compliant implementations](https://gateway-api.sigs.k8s.io/docs/implementations/versions/v1.4/) | [OpenShift Router (HAProxy-based Ingress Controller)](https://docs.openshift.com/container-platform/4.17/networking/networking_operators/ingress-operator.html) | None |
| TLS/SSL termination | Can leverage cloud provider-managed TLS; supports Cert-Manager. | Configured in the Gateway [listener](https://gateway-api.sigs.k8s.io/guides/user-guides/tls/#listeners-and-tls) by the cluster operator | Supported with OpenShift's HAProxy router. | Fully flexible but requires manual setup; TLS can terminate at application load balancer, network load balancer, or app level |
| Security (WAF, ACLs, Auth) | More advanced security integrations (for example AWS Web Application Firewall, authentication). | Provided by the Gateway implementation | Basic access control via OpenShift OAuth. | Full control - can integrate with AWS Web Application Firewall, API Gateway, authentication proxies with manual configuration. |
| Traffic splitting | Fully supported through ingress rules. | Fully supported. | Supported through HAProxy-based Route annotations. | Depends on external networking setup. |
| External DNS support | Some ingress controllers (for example, application load balancers) support native DNS updates. | Fully configurable - can use ExternalDNS, Amazon Route 53, or others. | Requires ExternalDNS integration. | Fully configurable - can use ExternalDNS, Amazon Route 53, or others. |
| Custom annotations | Supports Kubernetes Ingress annotations, provider-specific features. | Gateway implementations typically use vendor-specific CRDs, annotations are supported as well. | Supports OpenShift-specific annotations. | No restrictions - fully customizable in external networking. |

## Recommended Configuration

Expand All @@ -75,6 +77,9 @@ While OpenShift Routes are a viable option, we recommend NGINX Ingress Controlle
OpenShift Routes remain a suitable choice if meet your current needs and you do not require advanced features.
{{% /alert %}}

If your cluster has a configured and working Gateway API implementation, using it might be a preferrable option.
The Gateway API standardized typical use cases, and the Mendix Operator only uses HTTPRoute features defined in the [v1.4 standard](https://gateway-api.sigs.k8s.io/reference/api-spec/1.4/spec/), without relying on any vendor-specific feature.

## Known Issues

* AWS Application Load Balancers do not work correctly with HTTP2 WebSockets.
Expand All @@ -83,3 +88,4 @@ OpenShift Routes remain a suitable choice if meet your current needs and you do

* Some application load balancer firewall rules can block file uploads or other Mendix app features.
* Linkerd does not work correctly with AWS Application Load Balancer and Azure Gateway Ingress Controller.
* Some Gateway API implementations do not fully implement the base v1.4 spec, or are experimental (unsupported).
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Advanced Ingress Settings in Mendix on Kubernetes"
linktitle: "Advanced Settings"
url: /developerportal/deploy/private-cloud-cluster/private-cloud-ingress-settings/advanced/
description: "Describes how to configure advanced Ingress settings."
weight: 40
weight: 50
---

## Introduction
Expand Down
Loading