Flatten Spark runner libraries.

[kennknowles]

Flatten Spark runner libraries into a single package (or two) so that everything is private.
See [~kenn] comment: #2050 (comment)

Imported from Jira BEAM-1563. Original Jira may contain additional context.
Reported by: amitsela.