Spark-­on-­YARN: The Road Ahead

Slides PDF Video

YARN is becoming a popular way to deploy Spark applications. In this presentation we’ll explore some of the existing features that make YARN a popular choice, and talk about some of the future work to make it even easier to deploy Spark applications on YARN. This includes enhancements to dynamic allocation such as supporting better data locality, all­-encompassing security and support for long­-running applications on secure environments, which is particularly important for Spark Streaming.