Adding the dependency via --packages would be ideal except that the NIO file readers rely on Guava 20.0+.
hadoop 2.6+ breaks with guava 17.0+ (
So I am using the shaded jar is the only way to prevent naming conflicts.
I don't believe we can shade using the --packages flag, but let me know if anyone knows of a workaround.
To clarify, the error that pops up when using --packages is: https://gist.github.com/Georgehe4/2897f2ead685a9fcf014cbf01cee4375.