Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions modules/manage/examples/kubernetes/schema-crds.feature
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,105 @@ Feature: Schema CRDs
"""
And schema "order-event" is successfully synced
Then I should be able to check compatibility against "order-event" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage fully compatible schema (Avro)
Given there is no schema "fully-compatible-schema" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::full-compatibility-schema-manifest[]
# This manifest creates an Avro schema named "fully-compatible-schema" in the "basic" cluster.
# The schema uses Full compatibility, ensuring backward and forward compatibility across versions.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: fully-compatible-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
schemaType: avro
compatibilityLevel: Full
text: |
{
"type": "record",
"name": "ExampleRecord",
"fields": [
{ "type": "string", "name": "field1" },
{ "type": "int", "name": "field2" }
]
}
# end::full-compatibility-schema-manifest[]
"""
And schema "fully-compatible-schema" is successfully synced
Then I should be able to check compatibility against "fully-compatible-schema" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage product schema (Avro)
Given there is no schema "product-schema" in cluster "basic"
When I apply Kubernetes manifest:
"""
# This manifest creates an Avro schema named "product-schema" in the "basic" cluster.
# This schema will be referenced by other schemas to demonstrate schema references.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: product-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
schemaType: avro
compatibilityLevel: Backward
text: |
{
"type": "record",
"name": "Product",
"fields": [
{ "type": "string", "name": "product_id" },
{ "type": "string", "name": "name" }
]
}
"""
And schema "product-schema" is successfully synced
Then I should be able to check compatibility against "product-schema" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage order schema with references (Avro)
Given there is a schema "product-schema" in cluster "basic"
And there is no schema "order-schema" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::schema-references-manifest[]
# This manifest creates an Avro schema named "order-schema" that references another schema.
# Schema references enable modular and reusable schema components for complex data structures.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: order-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
references:
- name: product-schema
subject: product
version: 1
text: |
{
"type": "record",
"name": "Order",
"fields": [
{ "name": "product", "type": "Product" }
]
}
# end::schema-references-manifest[]
"""
And schema "order-schema" is successfully synced
Then I should be able to check compatibility against "order-schema" in cluster "basic"
52 changes: 52 additions & 0 deletions modules/manage/examples/kubernetes/topic-crds.feature
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,55 @@ Feature: Topic CRDs
"""
And topic "topic1" is successfully synced
Then I should be able to produce and consume from "topic1" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage topic with write caching
Given there is no topic "chat-room" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::write-caching-topic-example[]
# This manifest creates a topic called "chat-room" with write caching enabled.
# Write caching provides better performance at the expense of durability.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: chat-room
spec:
cluster:
clusterRef:
name: basic
partitions: 3
replicationFactor: 1
additionalConfig:
write.caching: "true"
# end::write-caching-topic-example[]
"""
And topic "chat-room" is successfully synced
Then I should be able to produce and consume from "chat-room" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage topic with cleanup policy
Given there is no topic "delete-policy-topic" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::cleanup-policy-topic-example[]
# This manifest creates a topic with the cleanup policy set to "delete".
# The cleanup policy determines how partition log files are managed when they reach a certain size.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: delete-policy-topic
spec:
cluster:
clusterRef:
name: basic
partitions: 3
replicationFactor: 1
additionalConfig:
cleanup.policy: "delete"
# end::cleanup-policy-topic-example[]
"""
And topic "delete-policy-topic" is successfully synced
Then I should be able to produce and consume from "delete-policy-topic" in cluster "basic"
35 changes: 7 additions & 28 deletions modules/manage/pages/kubernetes/k-manage-topics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ The Redpanda Operator automatically configures the Topic controller to connect t

The `clusterRef` method automatically discovers connection details from the referenced Redpanda resource:

.`topic.yaml`
[,yaml]
----
apiVersion: cluster.redpanda.com/v1alpha2
Expand All @@ -138,6 +139,7 @@ spec:

The `staticConfiguration` method requires manually specifying all connection details:

.`topic.yaml`
[,yaml]
----
apiVersion: cluster.redpanda.com/v1alpha2
Expand Down Expand Up @@ -220,6 +222,7 @@ In addition to the general prerequisites for managing topics, you must have the

To create an Iceberg topic, set the `redpanda.iceberg.mode` configuration in the `additionalConfig` property of the `Topic` resource.

.`iceberg-topic.yaml`
[source,yaml]
----
apiVersion: cluster.redpanda.com/v1alpha2
Expand Down Expand Up @@ -326,21 +329,9 @@ With `write_caching` enabled at the cluster level, Redpanda fsyncs to disk accor
To override the cluster-level setting at the topic level, set the topic-level property xref:reference:topic-properties.adoc#writecaching[`write.caching`]:

.`example-topic.yaml`
[,yaml,lines=9]
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: chat-room
namespace: <namespace>
spec:
cluster:
clusterRef:
name: <cluster-name>
partitions: 3
replicationFactor: 3
additionalConfig:
write.caching: true
include::manage:example$kubernetes/topic-crds.feature[tags=write-caching-topic-example,indent=0]
----

With `write.caching` enabled at the topic level, Redpanda fsyncs to disk according to xref:reference:topic-properties.adoc#flushms[`flush.ms`] and xref:reference:topic-properties.adoc#flushbytes[`flush.bytes`], whichever is reached first.
Expand Down Expand Up @@ -627,21 +618,9 @@ CAUTION: Do not use `rpk` or any other Kafka clients to edit topics that you cre
The following example changes the cleanup policy for a topic:

.`example-topic.yaml`
[,yaml,lines=8-9]
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: <topic-name>
namespace: <namespace>
spec:
cluster:
clusterRef:
name: <cluster-name>
partitions: 3
replicationFactor: 3
additionalConfig:
cleanup.policy: "delete"
include::manage:example$kubernetes/topic-crds.feature[tags=cleanup-policy-topic-example,indent=0]
----

[,bash]
Expand Down
49 changes: 6 additions & 43 deletions modules/manage/pages/kubernetes/k-schema-controller.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -172,28 +172,10 @@ Compatibility modes determine how schema versions within a subject can evolve wi

For example, to set full compatibility, configure the Schema resource with:

[source,yaml]
.`schema.yaml`
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: fully-compatible-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
schemaType: avro
compatibilityLevel: Full
text: |
{
"type": "record",
"name": "ExampleRecord",
"fields": [
{ "type": "string", "name": "field1" },
{ "type": "int", "name": "field2" }
]
}
include::manage:example$kubernetes/schema-crds.feature[tags=full-compatibility-schema-manifest,indent=0]
----

Compatibility settings are essential for maintaining data consistency, especially when updating schemas over time.
Expand All @@ -206,29 +188,10 @@ NOTE: This feature is supported for Avro and Protobuf schemas.

Define a schema reference using the `references` field. The reference includes the name, subject, and version of the referenced schema:

[source,yaml]
.`schema.yaml`
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: order-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
references:
- name: product-schema
subject: product
version: 1
text: |
{
"type": "record",
"name": "Order",
"fields": [
{ "name": "product", "type": "Product" }
]
}
include::manage:example$kubernetes/schema-crds.feature[tags=schema-references-manifest,indent=0]
----

== Update a schema
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,14 @@ In this example, a RedpandaRole called `read-only-role` is created in the cluste

The Group resource supports Schema Registry ACLs for OIDC groups.

In this example, ACLs are created for an OIDC group called `engineering` in the cluster called `sasl`. The authorization rules grant `Read` and `Describe` access to all topics with names starting with `team-` using a `prefixed` pattern type, and the same `Read` and `Describe` access to Schema Registry subjects matching the same prefix.

.`group-with-sr-acls.yaml`
[,yaml,indent=0]
----
include::manage:example$kubernetes/group-crds.feature[tags=manage-group-acls,indent=0]
----

In this example, ACLs are created for an OIDC group called `engineering` in the cluster called `sasl`. The authorization rules grant `Read` and `Describe` access to all topics with names starting with `team-` using a `prefixed` pattern type, and the same `Read` and `Describe` access to Schema Registry subjects matching the same prefix.

== Common use cases

The following examples show common patterns for configuring Schema Registry ACLs using the User resource.
Expand Down
Loading