Remove header bloat introduced by BestStandardsSupport middleware

[nilbus]

After the Rails server ran for long enough, Chome started returning a net::ERR_RESPONSE_HEADERS_TOO_BIG error on every page load, and curl would error with curl: (27) Avoided giant realloc for header (max is 102400)!. A server restart would reset the headers and made the problem temporarily go away.

Investigating showed that the same IE=Edge, headers were being appended on every request until finally the header size exceeded the 100k limit that browsers impose.

This patch ensures that the IE=Edge and other header variables only get appended to the X-UA-Compatible header if they are not there already, and it makes the test pass that I modified to catch the problem.

The commit that introduced this problem (d8c1404, @nikitug) was applied to both master and 3-2-stable, so this fix should be backported to 3-2-stable as well.

[nilbus]

@carlosantoniodasilva, could you look at this since you originally pulled in the affected code? Thanks!

[carlosantoniodasilva]

@nilbus thanks, I'll take a look and reply soon.

[steveklabnik]

I wonder what the performance is on introducing a split/each in middleware.

[steveklabnik]

This obviously needs fixed, yeah.

[nilbus]

I was wondering about that too. I'll benchmark and see if I can get the time down if it's significant.

[guilleiguaran]

We decided to remove BestStandardsSupport middleware in favor of config.action_dispatch.default_headers

[nilbus]

@guilleiguaran, when will that happen? Depending on how soon, maybe this should be applied anyway for the mean time.

@steveklabnik, I did some benchmarks using some variants. If I cut out the split/each and just search the original header to see if it contains @header, then it only increases the time 15% instead of 60%.

500,000 repetitions
-------------------
original               0.65483  secs    Fastest
fixed_without_loop     0.773287 secs    15% Slower
fixed_using_index      0.819192 secs    20% Slower
fixed_without_split    1.323478 secs    50% Slower
fixed                  1.640181 secs    60% Slower

[carlosantoniodasilva]

Can you please rebase from current master and squash your commits? I'll merge after that, thanks.

[nilbus]

Squashed and rebased.

[rafaelfranca]

Thank you

[nilbus]

And thanks for the quick response! I'm glad it's fixed.

[carlosantoniodasilva]

❤️