A generic backend for request throttling

[adamw]

Maybe it would be possible to create a generic backend wrapper (maybe generic for all backends, maybe only for asynchronous ones?) which would allow throttling of requests?

Throttling could be per-host or per-host+path.

Or maybe there is a throttling library which we could use?

[aappddeevv]

I think the best you might be able to do is attach a set of generic properties (very untyped in general) that provides hints to the backend if that backend supports it.

But I think this would need to be per-request so that per-request "hints" would work. There may be an easy way to accommodate this somehow by using a more clever .send method that takes a "hint" object, specific to the backend for those that just gotta have it. That way people that don't need portability fo the actual code can do something backend-specific each time. I find that for libraries like sttp, as long as its API is portable in concept, its close enough that I benefit from using it in multiple programs but where I need to be specific to a platform, I need to do that.

[adamw]

@aappddeevv the "hints" you mention are already there: each request carries a tags: Map[String, Any], which is used e.g. by the brave backend to carry the trace id.

[aappddeevv]

Oh...however, I don't see that on the async backend, which is pretty much all I use these days.

[adamw]

@aappddeevv hm ... maybe I misunderstood you then, that's a field on the Request data structure, so is available in any backend in the send method :)

[aappddeevv]

Yes. If its one thing I have learned, its crazy HTTP request wild-west world out there. You could also overload .send to be backend specific with a hints object so that at least when sending, you have statically typed "options" for sending. That ties you to a backend, but I think that this is Ok because when you need it, you need it.

[adamw]

This can be implemented independently of sttp, in a more general way: we don't need to do anything HTTP-specific, but to rate-limit an F[T] effect. Similar reasoning applies as in #23. Also in this case, we've documented how to use an external rate limiter: https://sttp.readthedocs.io/en/latest/backends/custom.html#example-backend-with-rate-limiter.