Apache Spark continues to grow quickly, with new features including data frames, R support, and machine learning pipelines added in the past few releases. We’re also seeing fast diversification of the user community, with exciting use cases beyond the traditional ones in Internet companies. In this talk, we’ll cover some of the recent additions to Spark, focusing on what’s new in the 1.4 release, and discuss how new use cases are driving our Spark development roadmap at Databricks.
Matei Zaharia is an assistant professor of computer science at MIT and CTO of Databricks. He started the Spark project at UC Berkeley and continues to serve as Spark’s vice president at Apache.
Patrick Wendell is a cofounder and engineer at Databricks as well as a founding Committer and PMC member of Apache Spark . In the Spark project, Patrick has acted as release manager for several Spark releases, including Spark’s recent 1.4 release. Patrick also maintains several subsystems of Spark’s core engine. Before helping start Databricks, Patrick obtained an M.S. in Computer Science at UC Berkeley. His research focused on low latency scheduling for large scale analytics workloads. He holds a B.S.E in Computer Science from Princeton University.