Reduce default max content-length

[Tim-Brooks]

By default Elasticsearch supports a content-length of 100MB at the http layer. This default suggests that this is an appropriate size for bulk requests. 100MB is certainly much larger than necessary in order to achieve improved performance through reduced round trips and combined FSYNCs. Additionally, 100MB probably crosses the threshold, where performance is harmed due to the fact the content is not currently incrementally circuit broken and we probably do not handle that size of memory very effectively.

We should consider reducing this default to something more reasonable (8-32MB?). This is a static setting, so it would require clear release notes for 100MB users to reconfigure at rolling restart. Or it would need work to make it a dynamic setting (tricky).

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[original-brownbear]

@tbrooks8 do you still remember what we concluded needs to be done here?

[Tim-Brooks]

We discussed and decided not to take immediate action. I think we were in a agreement that 100MB bulk requests is larger than we would like. But making this change might need (if we did) would need to be coordinated with clients.