Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions modules/develop/partials/delete-topic-records.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

Redpanda lets you delete data from the beginning of a partition up to a specific event, also known as offset. The offset represents the true creation time of the event, not the time when it was stored by Redpanda. Deleting records frees up disk space, which is especially helpful if your producers are pushing more data than you anticipated in your retention plan. Do this when you know that all consumers have read up to that given offset, and the data is no longer needed.

There are different ways to delete records from a topic, including using the xref:reference:rpk/rpk-topic/rpk-topic-trim-prefix.adoc[`rpk topic trim-prefix`] command or using the `DeleteRecords` Kafka API with Kafka clients.
There are different ways to delete records from a topic, including using the xref:reference:rpk/rpk-topic/rpk-topic-trim-prefix.adoc[`rpk topic trim-prefix`] command, using the `DeleteRecords` Kafka API with Kafka clients, or using {ui}.

[NOTE]
====
- To delete records, `cleanup.policy` must be set to `delete` or `compact,delete`.
- Object storage is deleted asynchronously. After messages are deleted, the partition's start offset will have advanced, but garbage collection of deleted segments may not be complete.
- Similar to Kafka, after deleting records, local storage and object storage may still contain data for deleted offsets. (Redpanda does not truncate segments. Instead, it bumps the start offset, then it attempts to delete as many whole segments as possible.) Data before the new start offset is not visible to clients but could be read by someone with access to the local disk of a Redpanda node.
====
====

include::shared:partial$warning-delete-records.adoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@ within the segment before the requested offset can no longer be read.

The `--offset/-o` flag allows you to indicate which index you want to set the
partition's low watermark (start offset) to. It can be a single integer value
denoting the offset, or a timestamp if you prefix the offset with an '@'. You may
select which partition you want to trim the offset from with the `--partitions/-p`
flag.
denoting the offset, or it can be a timestamp if you prefix the offset with an '@'. You can select which partition you want to trim the offset from with the `--partitions/-p` flag.

The `--from-file` option allows to trim the offsets specified in a text file with
the following format:
Expand All @@ -25,6 +23,8 @@ the following format:

or the equivalent keyed JSON/YAML file.

include::shared:partial$warning-delete-records.adoc[]

== Examples

* Trim records in 'foo' topic to offset 120 in partition 1:
Expand Down
9 changes: 9 additions & 0 deletions modules/shared/partials/warning-delete-records.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[WARNING]
====
When you pass a timestamp to `rpk topic trim-prefix`, Redpanda advances the partition start offset to the first record whose timestamp is after the threshold. If record timestamps are not in order with respect to offsets, this may truncate an entire partition. Before using a timestamp, verify that timestamps increase in the same order as offsets in the topic to avoid unintended data loss. For example:

[,bash]
----
rpk topic consume <topic> -n 50 --format '%o %d{go[2006-01-02T15:04:05Z07:00]} %k %v'
----
====