Don't delay internally generated writes

[duarten]

In the context of the MV backpressure algorithm, writes are delayed based on the target base replica's view update backlog, with the aim to lower the rate at which the client is sending. However, some of these writes are internal, such as view updates themselves and hints. Those should not be delayed by the storage_proxy.

[vladzcloudius]

@eliransin ping.
What's the status of this?

[eliransin]

@vladzcloudius We still haven't started this. But take into account that this issue states that the
internal writes are delayed as normal writes. This is what you wanted to validate (as the correct
behavior). This issue actually states that what we wanted to validate as the correct behavior is in
fact a bug. But to your question we still haven't started this one, I will update when we do.

[vladzcloudius]

@eliransin I think the issue is not in a contradiction with my other request: the issue is talking about view updates and view hints while I was talking about base table hints.

Regular and view hints are not the same to begin with - they have different context and maybe we need to treated differently, including what this issue is talking about.

More importantly this issue is mainly not about view hints but more about view updates and my previous question as for the status of this issue was absolutely NOT related to hints being out of MV backpressure scope.

[vladzcloudius]

@eliransin Ping.

Do you still think that this issue is relevant? If yes, let's define it a priority, if not - let's close it.

[kostja]

This has been suggested elsewhere as a possible alternative to view backpressure handling @gleb-cloudius @nyh @wmitros @piodul and doesn't need to be tracked separately.

[nyh]

@kostja I'm in favor of closing this issue, but not for the reason you mentioned (which I don't understand).

Rather, the original issue seems to suggest that "view updates" and "hints" experience the MV flow control sleep. But I think it's not true for "view updates" (since views don't have views of their own), and for base-table hints perhaps it is true - but it sounds like a good idea, since they create view updates, should indeed have the usual flow control - we don't want a million hints replayed to generate a backlog of a million unfinished view updates.

[wmitros]

@kostja I'm in favor of closing this issue, but not for the reason you mentioned (which I don't understand).

Rather, the original issue seems to suggest that "view updates" and "hints" experience the MV flow control sleep. But I think it's not true for "view updates" (since views don't have views of their own), and for base-table hints perhaps it is true - but it sounds like a good idea, since they create view updates, should indeed have the usual flow control - we don't want a million hints replayed to generate a backlog of a million unfinished view updates.

From what I've seen in testing, currently the mv flow control does delay view updates as well, which I think was the focus of this issue and I believe it is currently a bug (why make existing view updates keep resources longer when we're trying to make them use less resources)
We can also consider avoiding the mv flow control for writes that do not generate view updates, but you could argue that more throttling isn't bed when we're overloaded anyway.
I actually found an old attempt that tried implementing just that: https://groups.google.com/u/2/g/scylladb-dev/c/oou5-Y74yXc/m/vGrxAsjOBAAJ