Update Jackson to >= 2.6

[drcrallen]

There are numerous problems with sticking to Jackson < 2.6, including lacking spark compatibility and lacking compatibility with recent versions of various cloud SDKs.

This ask is that Jackson be upgraded.

[gianm]

Can you add some color about why Jackson has not yet been upgraded and what makes it difficult?

[leventov]

Hadoop will always have the most stale dependencies, due to it's development "velocity". Always having Druid to stay on as old library versions as Hadoop use (and not the newest Hadoop!) is very sad. Is it approachable to decouple Hadoop and have a separate set of dependencies for it? For example Jackson sends JSON after all, so on protocol level there should be no difference between any Jackson version.

[gianm]

Newer hadoop versions have some better classloader control, it's possible we could find a set of parameters that allows us to update jackson, and tell users to use those parameters. I'm not totally sure if this will work though.

[drcrallen]

@gianm #1674 and more specifically #1669 (comment)

[drcrallen]

A profile -P spark2 was added in #4808