retention.ms doesn't take effect on docker image

[hseagle]

Version & Environment

Redpanda version: v22.2.5

I'm using the latest redpanda docker image, version number is v22.2.5

after set retention.ms to 1000 and wait for a while, the old message is still kept.

The following is the detail operation.

docker run -e TZ=Asia/Shanghai -d --name=redpanda-1 \
-p 8081:8081 \
-p 8082:8082 \
-p 9092:9092 \
-p 9644:9644 \
vectorized/redpanda:v22.2.5 \
redpanda start \
--overprovisioned \
--smp 3  \
--memory 1G \
--reserve-memory 0M \
--node-id 0 \
--check=false

create a new topic and change the value of retention.ms and log_compaction_interval_ms

docker exec -it redpanda-1 rpk topic create twitch_chat
docker exec -it redpanda-1 rpk topic alter-config --set retention.ms=1000
# set the value for log_compaction_interval_ms to 60000
docker exec -it redpanda-1 rpk cluster config edit

ingest some data into topic twitch_chat and wait for some time, the ingested message is still existed. The behavior is unexpected.

[dotnwat]

The log is partitioned into fixed size segments, and appends are placed into the latest segment. Only segments that aren't the latest segment are compacted. So it may be that you need to write more data into the system to observe compaction taking place, or alternatively, you could restart Redpanda which will cause new segments to be created and make the data eligible for compaction.

[hseagle]

Thank you for your suggestion. Unfortunately, the compaction did not happen even after change the following config items and injested a lot of data into the topic.

docker exec -it redpanda-1 rpk topic alter-config twitch_chat --set segment.bytes=10240

Login into the docker container and check the log file, we can find that no compaction has happened

drwxr-xr-x 2 redpanda redpanda 43 Oct 10 09:53 0_9
total 161M
-rw-r--r-- 1 redpanda redpanda 154M Oct 10 11:15 0-1-v1.log
-rw-r--r-- 1 redpanda redpanda   59 Oct 10 09:53 tx.snapshot
-rw-r--r-- 1 redpanda redpanda 154M Oct 10 11:15 kafka/twitch_chat/0_9/0-1-v1.log
-rw-r--r-- 1 redpanda redpanda 59 Oct 10 09:53 kafka/twitch_chat/0_9/tx.snapshot

[emaxerrno]

this happens after the active segment closes. think about it like this.

to save data we open a file and look up the size on the config. we keep it until it fills up, once we roll the segmment (close,open the next one), then we look up the next size, and so on.

[hseagle]

Ok, I got it. Set compacted_log_segement_size to a smaller value and it will generate new segment file quickly. When the condition of retention is satisfied, the old segment will be removed. If there is only one segment, even the message inside the segment is expired, the segment will still be kept.

Thank you for your explaination.

[emaxerrno]

You can reduce the segment size and produce at least 1 full segment and you will see compaction working.

[emaxerrno]

Moving this to a discussion as is not an issue.