Abort request when too many Druid filters are generated

[QubitPi]

No description provided.

[archolewa]

This is WAY too small a default. Any system that can't handle 10 values isn't
going to last 5 minutes in any production environment. I would make this
at least 10000, but probably 100,000. Basically, I think a default ceiling
should be very high. High enough that you'll start experiencing some pain before
you hit the ceiling.

[cdeszaq]

High enough that you'll start experiencing some pain before
you hit the ceiling.

This can be dangerous because the pain is likely felt across multiple requests as a generally elevated latency. I think it's OK to have the default limit start very high (eg. 100k) when the feature is introduced, since that should generally be backwards-compatible, but I think the default limit should be lowered for new installations after some sane period of time.

What I'm essentially saying is that we should deprecate the high limit and lower it in a later version. That gives folks time to adjust the config value as needed, while also retaining a general policy of "sane by default".

I agree than the "safe" limit needs to be much higher than 10, but 100k seems a bit too high, since that would generate a massive query that would incur ser/deser and over-the-wire latency. 1k seems reasonable, but I could be convinced 10k is sane if there were some "here's the impact that size has on latency" data.

[QubitPi]

@cdeszaq I have an issue created for your request - #705

[cdeszaq]

I'd love to see this value show up in the default config file as well, so that it's obvious it's a configurable value.