diff --git a/modules/develop/partials/delete-topic-records.adoc b/modules/develop/partials/delete-topic-records.adoc index a58b67720a..fe6a83d1eb 100644 --- a/modules/develop/partials/delete-topic-records.adoc +++ b/modules/develop/partials/delete-topic-records.adoc @@ -1,12 +1,14 @@ == Delete records from a topic -Redpanda lets you delete data from the beginning of a partition up to a specific event, also known as offset. The offset represents the true creation time of the event, not the time when it was stored by Redpanda. Deleting records frees up disk space, which is especially helpful if your producers are pushing more data than you anticipated in your retention plan. Do this when you know that all consumers have read up to that given offset, and the data is no longer needed. +Redpanda lets you delete data from the beginning of a partition up to a specific offset (a monotonically increasing sequence number for records in a partition). Deleting records frees up disk space, which is especially helpful if your producers are pushing more data than you anticipated in your retention plan. Delete records when you know that all consumers have read up to that given offset, and the data is no longer needed. -There are different ways to delete records from a topic, including using the xref:reference:rpk/rpk-topic/rpk-topic-trim-prefix.adoc[`rpk topic trim-prefix`] command or using the `DeleteRecords` Kafka API with Kafka clients. +There are different ways to delete records from a topic, including using the xref:reference:rpk/rpk-topic/rpk-topic-trim-prefix.adoc[`rpk topic trim-prefix`] command, using the `DeleteRecords` Kafka API with Kafka clients, or using {ui}. [NOTE] ==== - To delete records, `cleanup.policy` must be set to `delete` or `compact,delete`. - Object storage is deleted asynchronously. After messages are deleted, the partition's start offset will have advanced, but garbage collection of deleted segments may not be complete. - Similar to Kafka, after deleting records, local storage and object storage may still contain data for deleted offsets. (Redpanda does not truncate segments. Instead, it bumps the start offset, then it attempts to delete as many whole segments as possible.) Data before the new start offset is not visible to clients but could be read by someone with access to the local disk of a Redpanda node. -==== \ No newline at end of file +==== + +include::shared:partial$warning-delete-records.adoc[] \ No newline at end of file diff --git a/modules/reference/pages/rpk/rpk-topic/rpk-topic-trim-prefix.adoc b/modules/reference/pages/rpk/rpk-topic/rpk-topic-trim-prefix.adoc index ef400e3db7..0456498004 100644 --- a/modules/reference/pages/rpk/rpk-topic/rpk-topic-trim-prefix.adoc +++ b/modules/reference/pages/rpk/rpk-topic/rpk-topic-trim-prefix.adoc @@ -5,14 +5,12 @@ Trim records from topics This command allows you to trim records from topics, where Redpanda sets the LogStartOffset for partitions to the requested offset. All segments -whose base offset is less then the requested offset are deleted, and any records +whose base offset is less than the requested offset are deleted, and any records within the segment before the requested offset can no longer be read. The `--offset/-o` flag allows you to indicate which index you want to set the partition's low watermark (start offset) to. It can be a single integer value -denoting the offset, or a timestamp if you prefix the offset with an '@'. You may -select which partition you want to trim the offset from with the `--partitions/-p` -flag. +denoting the offset, or it can be a timestamp if you prefix the offset with an '@'. You can select which partition you want to trim the offset from with the `--partitions/-p` flag. The `--from-file` option allows to trim the offsets specified in a text file with the following format: @@ -25,6 +23,8 @@ the following format: or the equivalent keyed JSON/YAML file. +include::shared:partial$warning-delete-records.adoc[] + == Examples * Trim records in 'foo' topic to offset 120 in partition 1: diff --git a/modules/shared/partials/warning-delete-records.adoc b/modules/shared/partials/warning-delete-records.adoc new file mode 100644 index 0000000000..ff81382261 --- /dev/null +++ b/modules/shared/partials/warning-delete-records.adoc @@ -0,0 +1,9 @@ +[WARNING] +==== +When you delete records from a topic with a timestamp, Redpanda advances the partition start offset to the first record whose timestamp is after the threshold. If record timestamps are not in order with respect to offsets, this may truncate an entire partition. Before using a timestamp, verify that timestamps increase in the same order as offsets in the topic to avoid unintended data loss. For example: + +[,bash] +---- +rpk topic consume -n 50 --format '%o %d{go[2006-01-02T15:04:05Z07:00]} %k %v' +---- +==== \ No newline at end of file