Yarn has been the default orchestration platform for tools from Hadoop ecosystem. This has started changing in recent times. Especially with Spark, which integrates very well with storage platforms like S3 and isn't tightly coupled with other Hadoop ecosystem tools.
Refer this post for detailed steps on deploying Spark on Kubernetes: https://developer.sh/posts/spark-kubernetes-guide