The first attempt at fixing the way to report cleanup.policy was
incorrect, because it meant that we reported the config as
DEFAULT_CONFIG even if the config was explicitly set but to the
cluster-default value.

Instead, in this case, we want to report this as a DYNAMIC_TOPIC_CONFIG.
We can do this now since we stopped setting the cleanup.policy value
explicitly for each topic.

Furthermore, the compat tests was simplified because now there are 2
separate fix attepts deployed at multiple redpanda versions, so to keep
the test sane, we just ignore the diff until a version (24.2.1) where we
know that this version will contain the fixes and all following version
we might compare against also contains the fixes.

Lastly, the change to the DescribeTopicConfigs ducktape test was also
reverted to correctly report the cleanup.policy as DYNAMIC_TOPIC_CONFIG
when this was explicitly specified on the topic creation request.

This reverts commit 261dc93.

Pure refactor. Adds helpers to check if compaction or deletion are set
in the cleanup policy flags.

This changes all uses of `ntp_config::is_compacted` and
`ntp_config::is_collectible` to fall back to the cluster-wide default
config `log_cleanup_policy`.

Since it is no longer the case that all topics have a topic-level
override for the cleanup policy, we fall back to using the cluster-level
default cleanup policy when the topic-level override is not set during
the partition recovery process.

This fixes a bug in the handling of archiver restarts when the read
replica topic config changes. Previously the archiver would incorrectly
restart on every topic config change if "read_replica" was set to
`std::optional<bool>(false)`.

Pure refactor except that the log messages emitted when cloud storage
needs to be restart change. The message emitted now states the reason
for restarting cloud storage instead of printing the old and new ntp
configs.

Pure refactor, no behaviour change intended.

This changes the partition-raft-log interface to decouple topic-level
config overrides and changes to compaction. This allows calling
`log::notify_compaction_update` independently from `log::set_overrides`
whenever `log_cleanup_policy` changes in the follow up commit.

Now that the cleanup policy is not always set on the topic-level
overrides, we need to watch for changes to the `log_cleanup_policy`
config and react to that by:
 * Restarting the archiver if compaction changes
 * (Un)marking the segments for compaction if compaction changes

This adds tests to verify that archiver is restarted whenever there are
changes to the compaction configs, either at the cluster-level defaults
or at the topic-level overrides.