New retention config not working for old metrics?

[wolfzhaoshuai]

I set --storage.tsdb.retention=7d months ago, but this does not met our new need, so I adjusted --storage.tsdb.retention=15d. But when I queried metrics today, I can only find metrics of 7 days.
My config:
/data/apps/prometheus-2.1.0/prometheus --config.file="/data/apps/prometheus-2.1.0/prometheus.yml" --storage.tsdb.path=/data/data/prometheus --storage.tsdb.retention=15d --log.level=debug --storage.tsdb.min-block-duration=1h --storage.tsdb.max-block-duration=3h

[simonpasquier]

When have you changed --storage.tsdb.retention? Are you running version 2.1.0? If yes, I would recommend to upgrade to v2.4.3.

Also with 15-days retention, you shouldn't have to change --storage.tsdb.min-block-duration/--storage.tsdb.max-block-duration.

[krasi-georgiev]

Could you run the tsdb cli so we can see the time ranges of all blocks. this will help narrow down the issue.

https://github.com/prometheus/tsdb/tree/master/cmd/tsdb

tsdb ls

also double check in ...:9090/flags that the retention is indeed set to 15d.

[krasi-georgiev]

@wolfzhaoshuai any progress with this?

[krasi-georgiev]

@wolfzhaoshuai closing for no activity. Please reopen if you are still facing this and will continue the troubleshooting.