Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions local-antora-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,14 @@ ui:
snapshot: true
asciidoc:
attributes:
# Report prop: macro targets missing from the published property JSON.
property-validate: warn
extensions:
- '@asciidoctor/tabs'
- '@redpanda-data/docs-extensions-and-macros/macros/rp-connect-components'
- '@redpanda-data/docs-extensions-and-macros/macros/glossary'
- '@redpanda-data/docs-extensions-and-macros/macros/config-ref'
- '@redpanda-data/docs-extensions-and-macros/macros/prop'
- '@redpanda-data/docs-extensions-and-macros/macros/badge'
- '@redpanda-data/docs-extensions-and-macros/macros/helm-ref'
- '@redpanda-data/docs-extensions-and-macros/asciidoc-extensions/add-line-numbers-highlights'
Expand Down
4 changes: 2 additions & 2 deletions modules/develop/pages/topics/create-topic.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ The default is *infinite*.
| *Message size*
| The maximum size of a message or batch for a newly-created topic.

The default is *20 MiB* for BYOC and Dedicated clusters, and *8 MiB* for Serverless clusters. You can increase this value up to *32 MiB* for BYOC and Dedicated clusters, and *20 MiB* for Serverless clusters, with the `max.message.bytes` topic property.
The default is *20 MiB* for BYOC and Dedicated clusters, and *8 MiB* for Serverless clusters. You can increase this value up to *32 MiB* for BYOC and Dedicated clusters, and *20 MiB* for Serverless clusters, with the prop:max.message.bytes[] topic property.

| *Segment size*
| The maximum size of a log segment. When a segment reaches this size, Redpanda closes it and starts a new one.

Redpanda Cloud sets the segment size for your cluster automatically. You can override it for a topic with the `segment.bytes` property.
Redpanda Cloud sets the segment size for your cluster automatically. You can override it for a topic with the prop:segment.bytes[] property.
|===

== Next steps
Expand Down
2 changes: 1 addition & 1 deletion modules/get-started/pages/cloud-overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ xref:develop:data-transforms/how-transforms-work.adoc[Data transforms] (Wasm) pr

== Redpanda Cloud vs Self-Managed feature compatibility

Because Redpanda Cloud is a fully-managed service that provides maintenance, data and partition balancing, upgrades, and recovery, much of the cluster maintenance required for Self-Managed users is not necessary for Redpanda Cloud users. Also, Redpanda Cloud is opinionated about Kafka configurations. For example, automatic topic creation is disabled. Some systems expect the Kafka service to automatically create topics when a message is produced to a topic that doesn't exist. (You can enable this for BYOC and Dedicated clusters with the `auto_create_topics_enabled` cluster property.)
Because Redpanda Cloud is a fully-managed service that provides maintenance, data and partition balancing, upgrades, and recovery, much of the cluster maintenance required for Self-Managed users is not necessary for Redpanda Cloud users. Also, Redpanda Cloud is opinionated about Kafka configurations. For example, automatic topic creation is disabled. Some systems expect the Kafka service to automatically create topics when a message is produced to a topic that doesn't exist. (You can enable this for BYOC and Dedicated clusters with the prop:auto_create_topics_enabled[] cluster property.)

New clusters in Redpanda Cloud generally include functionality added in Self-Managed versions immediately. Existing clusters include new functionality when they get upgraded to the latest version.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,10 +123,10 @@ rpk topic create <topic_name> -c redpanda.remote.readreplica=<source-cluster-buc
```

- For `<topic_name>`, use the same name as the original topic.
- For `<source-cluster-bucket-name>`, use the bucket specified in the `cloud_storage_bucket` properties for the origin cluster.
- For `<source-cluster-bucket-name>`, use the bucket specified in the prop:cloud_storage_bucket[] properties for the origin cluster.

For standard BYOC clusters, the source cluster bucket name follows the pattern: `redpanda-cloud-storage-${SOURCE_CLUSTER_ID}`

== Optional: Tune for live topics

For remote read replicas reading from a live topic (that is, a topic that's being actively written to by a source cluster), it may be advantageous to control how often segments are flushed to object storage. By default, this is set to 60 minutes. To tune `cloud_storage_segment_max_upload_interval_sec` on the source cluster, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. (For cold topics, where segments are closed and older than 60 minutes, this configuration is unnecessary: the data is already uploaded to object storage.)
For remote read replicas reading from a live topic (that is, a topic that's being actively written to by a source cluster), it may be advantageous to control how often segments are flushed to object storage. By default, this is set to 60 minutes. To tune prop:cloud_storage_segment_max_upload_interval_sec[] on the source cluster, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. (For cold topics, where segments are closed and older than 60 minutes, this configuration is unnecessary: the data is already uploaded to object storage.)
12 changes: 6 additions & 6 deletions modules/get-started/pages/whats-new-cloud.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ Redpanda SQL is now available on BYOVPC clusters on AWS. Provision the required

=== Decode Iceberg keys, values, and headers independently

On BYOC and BYOVPC clusters with Iceberg-enabled topics, the `redpanda.iceberg.mode` topic property now accepts a section-based syntax that lets you independently control how Redpanda translates the record key, value, and headers into the Iceberg table. You can decode keys and header values using a schema or store them as UTF-8 strings, instead of the default raw bytes. See xref:manage:iceberg/specify-iceberg-schema.adoc#configure-key-value-and-header-translation[Configure key, value, and header translation].
On BYOC and BYOVPC clusters with Iceberg-enabled topics, the prop:redpanda.iceberg.mode[] topic property now accepts a section-based syntax that lets you independently control how Redpanda translates the record key, value, and headers into the Iceberg table. You can decode keys and header values using a schema or store them as UTF-8 strings, instead of the default raw bytes. See xref:manage:iceberg/specify-iceberg-schema.adoc#configure-key-value-and-header-translation[Configure key, value, and header translation].

=== Resolve Iceberg topic schemas within a Schema Registry context

On BYOC and BYOVPC clusters, you can now set the `redpanda.schema.registry.context` topic property on an Iceberg-enabled topic to resolve its schemas within a specific xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] instead of the default context. See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context].
On BYOC and BYOVPC clusters, you can now set the prop:redpanda.schema.registry.context[] topic property on an Iceberg-enabled topic to resolve its schemas within a specific xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] instead of the default context. See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context].

=== Expanded Kafka client validation

Expand Down Expand Up @@ -97,7 +97,7 @@ BYOC clusters on GCP can now use GCP Lakehouse as an Iceberg REST catalog. See x

=== Customer-managed default topic settings

You can now set cluster-wide defaults for new topics on BYOC and Dedicated clusters. The xref:reference:properties/cluster-properties.adoc#default_topic_partitions[`default_topic_partitions`], xref:reference:properties/cluster-properties.adoc#log_retention_ms[`log_retention_ms`], and xref:reference:properties/cluster-properties.adoc#retention_bytes[`retention_bytes`] cluster properties are now customer-managed. The default topic retention period (`log_retention_ms`) previously could only be changed by Redpanda support. See xref:manage:cluster-maintenance/config-cluster.adoc[Configure Cluster Properties].
You can now set cluster-wide defaults for new topics on BYOC and Dedicated clusters. The xref:reference:properties/cluster-properties.adoc#default_topic_partitions[`default_topic_partitions`], xref:reference:properties/cluster-properties.adoc#log_retention_ms[`log_retention_ms`], and xref:reference:properties/cluster-properties.adoc#retention_bytes[`retention_bytes`] cluster properties are now customer-managed. The default topic retention period (prop:log_retention_ms[]) previously could only be changed by Redpanda support. See xref:manage:cluster-maintenance/config-cluster.adoc[Configure Cluster Properties].

=== Redpanda Connect updates

Expand Down Expand Up @@ -173,7 +173,7 @@ You can now subscribe to Redpanda Cloud directly through Google Cloud Marketplac

=== Iceberg: Configurable table namespace

You can now set a custom namespace for Iceberg tables instead of the default `redpanda` namespace, using the `iceberg_default_catalog_namespace` cluster property. A custom namespace is useful when multiple clusters write to the same catalog provider (such as AWS Glue), because each cluster must use a distinct namespace to avoid table name collisions. This property must be set when you first enable Iceberg and cannot be changed afterward. See xref:manage:iceberg/about-iceberg-topics.adoc#enable-iceberg-integration[Enable Iceberg integration].
You can now set a custom namespace for Iceberg tables instead of the default `redpanda` namespace, using the prop:iceberg_default_catalog_namespace[] cluster property. A custom namespace is useful when multiple clusters write to the same catalog provider (such as AWS Glue), because each cluster must use a distinct namespace to avoid table name collisions. This property must be set when you first enable Iceberg and cannot be changed afterward. See xref:manage:iceberg/about-iceberg-topics.adoc#enable-iceberg-integration[Enable Iceberg integration].

=== Group-based access control (GBAC)

Expand Down Expand Up @@ -347,7 +347,7 @@ You can view details about Kafka client connections using `rpk` or the Data Plan

=== Increased message size limit

Redpanda Cloud increased the xref:develop:topics/create-topic.adoc[message size limit] on newly-created topics. BYOC and Dedicated clusters have a default message size limit of 20 MiB with a maximum of 32 MiB. Serverless clusters have a default message size limit of 8 MiB with a maximum of 20 MiB. Configure the message size limit with the `max.message.bytes` topic property.
Redpanda Cloud increased the xref:develop:topics/create-topic.adoc[message size limit] on newly-created topics. BYOC and Dedicated clusters have a default message size limit of 20 MiB with a maximum of 32 MiB. Serverless clusters have a default message size limit of 8 MiB with a maximum of 20 MiB. Configure the message size limit with the prop:max.message.bytes[] topic property.

The message size setting on existing topics is not changed, but the message size limit on existing topics can only be updated to the new maximum.

Expand Down Expand Up @@ -421,7 +421,7 @@ The Redpanda Cloud Management MCP Server uses the Model Context Protocol (MCP) t

=== Automatic topic creation and topic limit

For BYOC and Dedicated clusters, you can now configure the `auto_create_topics_enabled` cluster property to automatically create a topic if a client produces to a non-existent topic.
For BYOC and Dedicated clusters, you can now configure the prop:auto_create_topics_enabled[] cluster property to automatically create a topic if a client produces to a non-existent topic.

For all clusters: each cluster now has a limit of 40,000 topics.

Expand Down
10 changes: 5 additions & 5 deletions modules/manage/pages/cluster-maintenance/config-cluster.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Cluster configuration properties are set to their default values and are automat

For a complete list of the cluster properties available in Redpanda Cloud, see xref:reference:properties/cluster-properties.adoc[Cluster Configuration Properties] and xref:reference:properties/object-storage-properties.adoc[Object Storage Properties].

NOTE: Some properties are read-only and cannot be changed. For example, `cluster_id` is a read-only property that is automatically set when the cluster is created.
NOTE: Some properties are read-only and cannot be changed. For example, prop:cluster_id[] is a read-only property that is automatically set when the cluster is created.

== Prerequisites

Expand Down Expand Up @@ -120,10 +120,10 @@ You can set cluster-wide defaults that apply to new topics:
| Default number of partitions for new topics.

| xref:reference:properties/cluster-properties.adoc#log_retention_ms[`log_retention_ms`]
| Default length of time to retain topic data before it becomes eligible for deletion. This sets the default topic retention period, which applies when a topic doesn't set its own `retention.ms`.
| Default length of time to retain topic data before it becomes eligible for deletion. This sets the default topic retention period, which applies when a topic doesn't set its own prop:retention.ms[].

| xref:reference:properties/cluster-properties.adoc#retention_bytes[`retention_bytes`]
| Default maximum size per partition before the oldest data becomes eligible for deletion. Applies when a topic doesn't set its own `retention.bytes`.
| Default maximum size per partition before the oldest data becomes eligible for deletion. Applies when a topic doesn't set its own prop:retention.bytes[].
|===

Set these properties using `rpk` or the Cloud API. For example, to set the default topic retention period to 24 hours:
Expand Down Expand Up @@ -166,9 +166,9 @@ curl -H "Authorization: Bearer ${RP_CLOUD_TOKEN}" -X PATCH \
[[max-message-size]]
== Configure the maximum message size

To control the maximum message size in Redpanda Cloud, set the `max.message.bytes` property on each topic. The default is 20 MiB for BYOC and Dedicated clusters and 8 MiB for Serverless clusters. You can increase it up to 32 MiB for BYOC and Dedicated clusters and 20 MiB for Serverless clusters. To set this property, see xref:develop:topics/create-topic.adoc[].
To control the maximum message size in Redpanda Cloud, set the prop:max.message.bytes[] property on each topic. The default is 20 MiB for BYOC and Dedicated clusters and 8 MiB for Serverless clusters. You can increase it up to 32 MiB for BYOC and Dedicated clusters and 20 MiB for Serverless clusters. To set this property, see xref:develop:topics/create-topic.adoc[].

The cluster property `kafka_batch_max_bytes` cannot be set through the Cloud API or the xref:manage:terraform-provider.adoc[Redpanda Terraform provider]. Attempting to set it returns `REASON_INVALID_INPUT` with the message `The properties [kafka_batch_max_bytes] are not allowed`.
The cluster property prop:kafka_batch_max_bytes[] cannot be set through the Cloud API or the xref:manage:terraform-provider.adoc[Redpanda Terraform provider]. Attempting to set it returns `REASON_INVALID_INPUT` with the message `The properties [kafka_batch_max_bytes] are not allowed`.

== View cluster property values

Expand Down
2 changes: 1 addition & 1 deletion modules/manage/pages/terraform-provider.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,7 @@ resource "redpanda_cluster" "test" {
}
----

Only the properties listed in xref:reference:properties/cluster-properties.adoc[] and xref:reference:properties/object-storage-properties.adoc[] can be configured through the provider. Setting an unsupported property returns a `REASON_INVALID_INPUT` error indicating that the property is not allowed. For example, `kafka_batch_max_bytes` cannot be set through the provider. To control the maximum message size, set the `max.message.bytes` property on each topic instead. For more information, see xref:manage:cluster-maintenance/config-cluster.adoc[].
Only the properties listed in xref:reference:properties/cluster-properties.adoc[] and xref:reference:properties/object-storage-properties.adoc[] can be configured through the provider. Setting an unsupported property returns a `REASON_INVALID_INPUT` error indicating that the property is not allowed. For example, prop:kafka_batch_max_bytes[] cannot be set through the provider. To control the maximum message size, set the prop:max.message.bytes[] property on each topic instead. For more information, see xref:manage:cluster-maintenance/config-cluster.adoc[].

NOTE: Cluster properties are not available on Serverless clusters or on BYOC and Dedicated clusters running on Azure.

Expand Down
Loading