Spark Summit 2013 brought the Apache Spark community together on December 2-3, 2013 at the Hotel Nikko in San Francisco. It featured production users of Spark, Shark, Spark Streaming and related projects.
High-quality downstream distributions of open-source projects benefit everyone. End-users enjoy convenient installation and upgrades, dependency management, system integration, and the fruits of a thriving testing and support community. Downstream packagers contribute testing and fixes to upstream developers and free up core teams to focus on enhancements and fixes rather than on the details of packaging. In this talk, we’ll discuss these benefits and present our efforts — along with the Fedora Big Data SIG — to package Spark for Fedora. We’ll cover some of the unique challenges presented by the impedance mismatch between traditional downstream packaging models and the Scala and big data ecosystems, present our current progress, and discuss opportunities for other members of the community to get involved.