New Features

• MongoDBOpsManager: Added support for deploying Ops Manager Application on multiple Kubernetes clusters. See documentation for more information.
• (Public Preview) MongoDB, OpsManager: Introduced opt-in Static Architecture (for all types of deployments) that avoids pulling any binaries at runtime.
  * This feature is recommended only for testing purposes, but will become the default in a later release.
  * You can activate this mode by setting the MDB_DEFAULT_ARCHITECTURE environment variable at the Operator level to static. Alternatively, you can annotate a specific MongoDB or OpsManager Custom Resource with mongodb.com/v1.architecture: "static".
  • The Operator supports seamless migration between the Static and non-Static architectures.
  • To learn more please see the relevant documentation:
    • Use Static Containers
    • Migrate to Static Containers
• MongoDB: Recover Resource Due to Broken Automation Configuration has been extended to all types of MongoDB resources, now including Sharded Clusters. For more information see https://www.mongodb.com/docs/kubernetes-operator/master/reference/troubleshooting/#recover-resource-due-to-broken-automation-configuration
• MongoDB, MongoDBMultiCluster: Placeholders in external services.
  • You can now define annotations for external services managed by the operator that contain placeholders which will be automatically replaced to the proper values.
  • Previously, the operator was configuring the same annotations for all external services created for each pod. Now, with placeholders the operator is able to customize
    annotations in each service with values that are relevant and different for the particular pod.
  • To learn more please see the relevant documentation:
    • MongoDB: spec.externalAccess.externalService.annotations
    • MongoDBMultiCluster: spec.externalAccess.externalService.annotations
• kubectl mongodb:
• Added printing build info when using the plugin.
• setup command:
  • Added --image-pull-secrets parameter. If specified, created service accounts will reference the specified secret on ImagePullSecrets field.
  • Improved handling of configurations when the operator is installed in a separate namespace than the resources it's watching and when the operator is watching more than one namespace.
  • Optimized roles and permissions setup in member clusters, using a single service account per cluster with correctly configured Role and RoleBinding (no ClusterRoles necessary) for each watched namespace.
• OpsManager: Added the spec.internalConnectivity field to allow overrides for the service used by the operator to ensure internal connectivity to the OpsManager pods.
• Extended the existing event based reconciliation by a time-based one, that is triggered every 24 hours. This ensures all Agents are always upgraded on timely manner.
• OpenShift / OLM Operator: Removed the requirement for cluster-wide permissions. Previously, the operator needed these permissions to configure admission webhooks. Now, webhooks are automatically configured by OLM.
• Added optional MDB_WEBHOOK_REGISTER_CONFIGURATION environment variable for the operator. It controls whether the operator should perform automatic admission webhook configuration. Default: true. It's set to false for OLM and OpenShift deployments.

Breaking Change

• MongoDBOpsManager Stopped testing against Ops Manager 5.0. While it may continue to work, we no longer officially support Ops Manager 5 and customers should move to a later version.

Helm Chart

• New operator.webhook.registerConfiguration parameter. It controls whether the operator should perform automatic admission webhook configuration (by setting MDB_WEBHOOK_REGISTER_CONFIGURATION environment variable for the operator). Default: true. It's set to false for OLM and OpenShift deployments.
• Changing the default agent.version to 107.0.0.8502-1, that will change the default agent used in helm deployments.
• Added operator.additionalArguments (default: []) allowing to pass additional arguments for the operator binary.
• Added operator.createResourcesServiceAccountsAndRoles (default: true) to control whether to install roles and service accounts for MongoDB and Ops Manager resources. When mongodb kubectl plugin is used to configure the operator for multi-cluster deployment, it installs all necessary roles and service accounts. Therefore, in some cases it is required to not install those roles using the operator's helm chart to avoid clashes.

Bug Fixes

• MongoDBMultiCluster: Fields spec.externalAccess.externalDomain and spec.clusterSpecList[*].externalAccess.externalDomains were reported as required even though they weren't
  used. Validation was triggered prematurely when structure spec.externalAccess was defined. Now, uniqueness of external domains will only be checked when the external domains are
  actually defined in spec.externalAccess.externalDomain or spec.clusterSpecList[*].externalAccess.externalDomains.
• MongoDB: Fixed a bug where upon deleting a MongoDB resource the controlledFeature policies are not unset on the related OpsManager/CloudManager instance, making cleanup in the UI impossible in the case of losing the kubernetes operator.
• OpsManager: The admin-key Secret is no longer deleted when removing the OpsManager Custom Resource. This enables easier Ops Manager re-installation.
• MongoDB ReadinessProbe Fixed the misleading error message of the readinessProbe: "... kubelet Readiness probe failed:...". This affects all mongodb deployments.
• Operator: Fixed cases where sometimes while communicating with Opsmanager the operator skipped TLS verification, even if it was activated.

Improvements

Kubectl plugin: The released plugin binaries are now signed, the signatures are published with the release assets. Our public key is available at this address. They are also notarized for MacOS.
Released Images signed: All container images published for the enterprise operator are cryptographically signed. This is visible on our Quay registry, and can be verified using our public key. It is available at this address.

New Features

• MongoDBOpsManager: Added support for the upcoming 7.0.x series of Ops Manager Server.

Bug Fixes

• Fix a bug that prevented terminating backup correctly.

MongoDB Enterprise Kubernetes Operator 1.23.0

Warnings and Breaking Changes

• Starting from 1.23 component image version numbers will be aligned to the MongoDB Enterprise Operator release tag. This allows clear identification of all images related to a specific version of the Operator. This affects the following images:
  • quay.io/mongodb/mongodb-enterprise-database-ubi
  • quay.io/mongodb/mongodb-enterprise-init-database-ubi
  • quay.io/mongodb/mongodb-enterprise-init-appdb-ubi
  • quay.io/mongodb/mongodb-enterprise-init-ops-manager-ubi
• Removed spec.exposedExternally in favor of spec.externalAccess from the MongoDB Customer Resource. spec.exposedExternally was deprecated in operator version 1.19.

Bug Fixes

• Fix a bug with scaling a multi-cluster replica-set in the case of losing connectivity to a member cluster. The fix addresses both the manual and automated recovery procedures.
• Fix of a bug where changing the names of the automation agent and MongoDB audit logs prevented them from being sent to Kubernetes pod logs. There are no longer restrictions on MongoDB audit log file names (mentioned in the previous release).
• New log types from the mongodb-enterprise-database container are now streamed to Kubernetes logs.
  • New log types:
    • agent-launcher-script
    • monitoring-agent
    • backup-agent
  • The rest of available log types:
    • automation-agent-verbose
    • automation-agent-stderr
    • automation-agent
    • mongodb
    • mongodb-audit
• MongoDBUser Fix a bug ignoring the Spec.MongoDBResourceRef.Namespace. This prevented storing the user resources in another namespace than the MongoDB resource.

MongoDB Enterprise Kubernetes Operator 1.22.0

Breaking Changes

• All Resources: The Operator no longer uses the "Reconciling" state. In most of the cases it has been replaced with "Pending" and a proper message

Deprecations

None

Bug Fixes

• MongoDB: Fix support for setting autoTerminateOnDeletion=true for sharded clusters. This setting makes sure that the operator stops and terminates the backup before the cleanup.

New Features

• MongoDB: An Automatic Recovery mechanism has been introduced for MongoDB resources and is turned on by default. If a Custom Resource remains in Pending or Failed state for a longer period of time (controlled by MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S environment variable at the Operator Pod spec level, the default is 20 minutes)
  the Automation Config is pushed to the Ops Manager. This helps to prevent a deadlock when an Automation Config can not be pushed because of the StatefulSet not being ready and the StatefulSet being not ready because of a broken Automation Config.
  The behavior can be turned off by setting MDB_AUTOMATIC_RECOVERY_ENABLE environment variable to false.
• MongoDB: MongoDB audit logs can now be routed to Kubernetes pod logs.
  • Ensure MongoDB audit logs are written to /var/log/mongodb-mms-automation/mongodb-audit.log file. Pod monitors this file and tails its content to k8s logs.
  • Use the following example configuration in MongoDB resource to send audit logs to k8s logs:

  spec:
    additionalMongodConfig:
      auditLog:
        destination: file
        format: JSON
        path: /var/log/mongodb-mms-automation/mongodb-audit.log
  

  • Audit log entries are tagged with the "mongodb-audit" key in pod logs. Extract audit log entries with the following example command:

  kubectl logs -c mongodb-enterprise-database replica-set-0 | jq -r 'select(.logType == "mongodb-audit") | .contents'
  

• MongoDBOpsManager: Improved handling of unreachable clusters in AppDB Multi-Cluster resources
  • In the last release, the operator required a healthy connection to the cluster to scale down processes, which could block the reconcile process if there was a full-cluster outage.
  • Now, the operator will still successfully manage the remaining healthy clusters, as long as they have a majority of votes to elect a primary.
  • The associated processes of an unreachable cluster are not automatically removed from the automation config and replica set configuration. These processes will only be removed under the following conditions:
    • The corresponding cluster is deleted from spec.applicationDatabase.clusterSpecList or has zero members specified.
    • When deleted, the operator scales down the replica set by removing processes tied to that cluster one at a time.
• MongoDBOpsManager: Add support for configuring logRotate on the automation-agent for appdb.
• MongoDBOpsManager: systemLog can now be configured to differ from the otherwise default of /var/log/mongodb-mms-automation.

MongoDB Enterprise Kubernetes Operator 1.21.0

Breaking changes

• The environment variable to track the operator namespace has been renamed from CURRENT_NAMESPACE to NAMESPACE. If you set this variable manually via YAML files, you should update this environment variable name while upgrading the operator deployment.

Bug fixes

• Fixes a bug where passing the labels via statefulset override mechanism would not lead to an override on the actual statefulset.

New Feature

• Support for Label and Annotations Wrapper for the following CRDs: mongodb, mongodbmulti and opsmanager
  • Additionally, to the specWrapper for statefulsets we now support overriding metadata.Labels and metadata.Annotations via the MetadataWrapper.

MongoDBOpsManager Resource

New Features

• Support configuring OpsManager with a highly available applicationDatabase across multiple Kubernetes clusters by introducing the following fields:
  • om.spec.applicationDatabase.topology which can be one of MultiCluster and SingleCluster.
  • om.spec.applicationDatabase.clusterSpecList for configuring the list of Kubernetes clusters which will have For extended considerations for the multi-cluster AppDB configuration, check the official guide and the OpsManager resource specification.
    The implementation is backwards compatible with single cluster deployments of AppDB, by defaulting om.spec.applicationDatabase.topology to SingleCluster. Existing OpsManager resources do not need to be modified to upgrade to this version of the operator.
• Support for providing a list of custom certificates for S3 based backups via secret references spec.backup.[]s3Stores.customCertificateSecretRefs and spec.backup.[]s3OpLogStores.customCertificateSecretRefs
  • The list consists of single certificate strings, each references a secret containing a certificate authority.
  • We do not support adding multiple certificates in a chain. In that case, only the first certificate in the chain is imported.
  • Note:
    • If providing a list of customCertificateSecretRefs, then those certificates will be used instead of the default certificates setup in the JVM Trust Store (in Ops Manager or Cloud Manager).
    • If none are provided, the default JVM Truststore certificates will be used instead.

Breaking changes

• The appdb-ca is no longer automatically added to the JVM Trust Store (in Ops Manager or Cloud Manager). Since a bug introduced in version 1.17.0, automatically adding these certificates to the JVM Trust Store has no longer worked.
  • This will only impact you if:
    • You are using the same custom certificate for both appdb-ca and for your S3 compatible backup store
    • AND: You are using an operator prior to 1.17.0 (where automated inclusion in the JVM Trust Store worked) OR had a workaround (such as mounting your own trust store to OM)
  • If you do need to use the same custom certificate for both appdb-ca and for your S3 compatible backup store then you now need to utilise spec.backup.[]s3Config.customCertificateSecretRefs (introduced in this release and covered below in the release notes) to specify the certificate authority for use for backups.
  • The appdb-ca is the certificate authority saved in the configmap specified under om.spec.applicationDatabase.security.tls.ca.

Bug fixes

• Allowed setting an arbitrary port number in spec.externalConnectivity.port when LoadBalancer service type is used for exposing Ops Manager instance externally.
• The operator is now able to import the appdb-ca which consists of a bundle of certificate authorities into the ops-manager JVM trust store. Previously, the keystore had 2 problems:
  • It was immutable.
  • Only the first certificate authority out of the bundle was imported into the trust store.
  • Both could lead to certificates being rejected by Ops Manager during requests to it.

Deprecation

• The setting spec.backup.[]s3Stores.customCertificate and spec.backup.[]s3OpLogStores.customCertificate are being deprecated in favor of spec.backup.[]s3OpLogStores.[]customCertificateSecretRefs and spec.backup.[]s3Stores.[]customCertificateSecretRefs
  • Previously, when enabling customCertificate, the operator would use the appdb-ca as the custom certificate. Currently, this should be explicitly set via customCertificateSecretRefs.

Release of the multi-cluster-cli for 1.20.1 since it failed for the previous 1.20.1 release

This release fixes an issue that prevented upgrading the Kubernetes Operator to 1.20.0 in OpenShift. Upgrade to this release instead.

MongoDBOpsManager Resource

• Added support for votes, priority and tags by introducing the spec.applicationDatabase.memberConfig.votes, spec.applicationDatabase.memberConfig.priority
  and spec.applicationDatabase.memberConfig.tags field.
• Introduced automatic change of the AppDB's image version suffix -ent to -ubi8.
  • This enables migration of AppDB images from the legacy repository (quay.io/mongodb/mongodb-enterprise-appdb-database-ubi) to the new official one (quay.io/mongodb/mongodb-enterprise-server) without changing the version in MongoDBOpsManager's applicationDatabase.version field.
  • The change will result a rolling update of AppDB replica set pods to the new, official images (referenced in Helm Chart in values.mongodb.name field), which are functionally equivalent to the previous ones (the same MongoDB version).
  • Suffix change occurs under specific circumstances:
    • Helm setting for appdb image: mongodb.name will now default to mongodb-enterprise-server.
    • The operator will automatically replace the suffix for image repositories
      that end with mongodb-enterprise-server.
      Operator will replace the suffix -ent with the value set in the environment variable
      MDB_IMAGE_TYPE, which defaults to -ubi8.
      For instance, the operator will migrate:
      • quay.io/mongodb/mongodb-enterprise-server:4.2.11-ent to quay.io/mongodb/mongodb-enterprise-server:4.2.11-ubi8.
      • MDB_IMAGE_TYPE=ubuntu2024 quay.io/mongodb/mongodb-enterprise-server:4.2.11-ent to quay.io/mongodb/mongodb-enterprise-server:4.2.11-ubuntu2024.
      • The operator will do the automatic migration of suffixes only for images
        that reference the name mongodb-enterprise-server.
        It won't perform migration for any other image name, e.g.:
        • mongodb-enterprise-appdb-database-ubi:4.0.0-ent will not be altered
      • To stop the automatic suffix migration behavior,
        set the following environment variable to true: MDB_APPDB_ASSUME_OLD_FORMAT=true
        or alternatively in the following helm chart setting: mongodb.appdbAssumeOldFormat=true
• Added support for defining bare versions in spec.applicationDatabase.version. Previously, it was required to specify AppDB's version with -ent suffix. Currently, it is possible to specify a bare version, e.g. 6.0.5 and the operator will convert it to 6.0.5-${MDB_IMAGE_TYPE}. The default for environment variable MDB_IMAGE_TYPE is -ubi8.

Bug fixes

• Fixed MongoDBMultiCluster not watching Ops Manager's connection configmap and secret.
• Fixed support for rotating the clusterfile secret, which is used for internal x509 authentication in MongoDB and MongoDBMultiCluster resources.

Helm Chart

• All images reference ubi variants by default (added suffix -ubi)
  • quay.io/mongodb/mongodb-enterprise-database-ubi
  • quay.io/mongodb/mongodb-enterprise-init-database-ubi
  • quay.io/mongodb/mongodb-enterprise-ops-manager-ubi
  • quay.io/mongodb/mongodb-enterprise-init-ops-manager-ubi
  • quay.io/mongodb/mongodb-enterprise-init-appdb-ubi
  • quay.io/mongodb/mongodb-agent-ubi
  • quay.io/mongodb/mongodb-enterprise-appdb-database-ubi
• Changed default AppDB repository to official MongoDB Enterprise repository in values.mongodb.name field: quay.io/mongodb/mongodb-enterprise-server.
• Introduced values.mongodb.imageType variable to specify a default image type suffix added to AppDB's version used by MongoDBOpsManager resource.

Breaking changes

• Removal of appdb.connectionSpec.Project since it has been deprecated for over 2 years.

Warning

• Avoid using the 1.19.0 version of the operator as it is tied to a broken release on the Openshift Marketplace.

MongoDB Resource

• Added support for setting replica set member votes by introducing the spec.memberOptions.[*].votes field.
• Added support for setting replica set member priority by introducing the spec.memberOptions.[*].priority field.
• Added support for setting replica set member tags by introducing the spec.memberOptions.[*].tags field.

MongoDBMulti Resouce

• Added support for setting replica set member votes by introducing the spec.clusterSpecList.[*].memberOptions.[*].votes field.
• Added support for setting replica set member priority by introducing the spec.clusterSpecList.[*].memberOptions.[*].priority field.
• Added support for setting replica set member tags by introducing the spec.clusterSpecList.[*].memberOptions.[*].tags field.

Improvements

• New guidance for multi-Kubernetes-cluster deployments without a Service Mesh. It covers use of a Load Balancer Service
  to expose ReplicaSet members on an externally reachable domain (spec.externalAccess.externalDomain).
  This leverages setting the process.hostname field in the Automation Config.
  This tutorial provides full guidance.
• spec.security.authentication.ldap.transportSecurity: "none" is now a valid configuration to use no transportSecurity.
• Allows you to configure podSpec per shard in a MongoDB Sharded cluster by specifying an array of podSpecs under spec.shardSpecificPodSpec for each shard.

Deprecations

• Making the field orgID in the project configmap a requirement. Note: If explicitly an empty orgID = "" has been chosen then OM will try to create an ORG with the project name.
• Ubuntu-based images were deprecated in favor of UBI-based images in operator version 1.17.0. In the 1.19.0 release we are removing the support for Ubuntu-based images. The ubuntu based images won't be rebuilt daily with updates. Please upgrade to the UBI-based images by following these instructions: https://www.mongodb.com/docs/kubernetes-operator/master/tutorial/migrate-k8s-images/#migrate-k8s-images
• The spec.exposedExternally option becomes deprecated in favor of spec.externalAccess. The deprecated option will be removed in MongoDB Enterprise Operator 1.22.0.

Bug fixes

• Fixed handling of WATCH_NAMESPACE='*' environment variable for multi-cluster deployments with cluster-wide operator. In some specific circumstances, API clients for member clusters were configured incorrectly resulting in deployment errors.
  • Example error in this case:
    • The secret object 'mdb-multi-rs-cert' does not contain all the valid certificates needed: secrets "mdb-multi-rs-cert-pem" already exists
  • These specific circumstances were:
    • WATCH_NAMESPACE='*' environment variable passed to the operator deployment
    • specific namespace set in kubeconfig for member clusters
    • not using multi-cluster cli tool for configuring
  • Possible workarounds:
    • set WATCH_NAMESPACE environment variable to specific namespaces instead of '*'
    • make sure that kubeconfigs for member clusters doesn't specify a namespace

Breaking changes

• Renaming of the multicluster CRD MongoDBMulti to MongoDBMultiCluster

• The spec.members field is required to be set in case of MongoDB deployment of type ReplicaSet.

Bug fixes

• Fixed a panic when CertificatesSecretsPrefix was set but no further spec.security.tls setting was set i.e. tls.additionalCertificateDomains or tls.ca.

Improvements

• Added support for the missing features for Ops Manager Backup configuration page. This includes:
  • KMIP Backup Configuration support by introducing spec.backup.encryption.kmip in both OpsManager and MongoDB resources.
  • Backup Assignment Labels settings in spec.backup.[*].assignmentLabels elements of the OpsManager resource.
  • Backup Snapshot Schedule configuration via spec.backup.snapshotSchedule in the OpsManager resource.
• Added SCRAM-SHA-1 support for both user and Agent authentication. Before enabling this capability, make sure you use both MONGODB-CR and SCRAM-SHA-1 in the authentication modes.

Bug fixes

• Fixed liveness probe reporting positive result when the agent process was killed. This could cause database pods to run without automation agent.
• Fixed startup script in database pod, that could in some situations report errors on pod's restart.

Breaking changes and deprecations

• The field spec.security.tls.secretRef.prefix has been removed from MongoDB and OpsManager resources. It was deprecated in the MongoDB Enterprise
  1.15.0 and removed from the Operator runtime in 1.17.0. Before upgrading to this version, make sure you migrated to the new TLS format using the following Migration Guide before upgrading the Operator.