k/topics: make write caching configs settable at creation #17515
Conversation
|
new failures in https://buildkite.com/redpanda/redpanda/builds/47122#018e924b-6194-4a17-aae7-de9fb3938d38: new failures in https://buildkite.com/redpanda/redpanda/builds/47122#018e923b-431e-4887-879c-a7633839d0c2: new failures in https://buildkite.com/redpanda/redpanda/builds/47122#018e923b-4319-4618-b660-198fdc0a633c: new failures in https://buildkite.com/redpanda/redpanda/builds/47122#018e924b-618f-482e-8b7b-ba648d70a0f2: new failures in https://buildkite.com/redpanda/redpanda/builds/47130#018e95c7-31e6-4929-91cf-8a0e308115ee: new failures in https://buildkite.com/redpanda/redpanda/builds/47130#018e95c7-31eb-45d4-991e-49536258b1c6: |
|
Should the name be “write cache” vs “write caching” I don’t see other properties having “ing” @mattschumpert - this can be in a separate change |
Jitter was implemented but the actual waiter source was not updated to use the jittered duration.
With very large flush delays (eg: ns::max()), the background flusher CV runs into an UB because of a seastar bug that doesn't accept large timeouts supposedly used as indefinitel waits. This commit worksaround by converting the delay into larger duration type (thus smaller integral value) avoiding the overflow.
Missed this in the original patch series that added these properties.
Certain kafka clients set flush.ms default to long_max which oveflows during serde serialization as it requires that any duration does not exceed nanoseconds::max(). If we encounter such a value for flush.ms during topic creation it seems logical to clamp the value to nanoseconds::max(), rather than failing the topic creation for better user experience and backward compatibility. This commit clamps flush.ms to nanoseconds::max() during topic creation and alteration.
bharathv
force-pushed
the
wc_create_properties
branch
from
02efbd1
to
c006b66
Compare
|
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47174#018e9bbd-93e1-46bf-b3f8-728db985eea4 ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47174#018e9bbd-93e4-4179-b58b-4fdaffe99768 |
|
Good to merge this, @bharathv ? We can trigger a build after we merge this if @ballard26 needs this change. |
|
@emaxerrno a bunch of properties do end in 'ing' related to data balancing, controller log rate limiting, principal mapping etc. I don't have a strong opinion here, but given it is one cache per broker and this is a global cluster config, I have a small concern that using 'cache' may make people think of 1 shared cache per cluster, especially in certain cloud contexts where the broker count is less in the foreground. So, a slight preference to keep the general capability name with 'ing' |
Allows having write caching property overrides at topic creation time.
Backports Required
Release Notes