Mike Olson from Cloudera has called Spark the killer App for Scala.
Not only is it written in Scala but it is also deeply integrated in
its toolchain, making intensive use of the Scala REPL and with it the
Scala compiler. One good way to see Spark is as a domain specific
language embedded in Scala as a host language. The domain in this
setting is big data, or, more specifically, collections distributed
over a cluster. This makes Spark the most popular and in some sense
the most powerful implementation of Scala-like collections.
In this talk I will highlight similarities and differences of the
collections used in Scala and Spark. I will outline some areas where
Scala’s collections could benefit from the experience gained from
Spark, and, conversely, where improved Scala tooling can help Spark.
Martin Odersky is a founder of Typesafe and a professor at EPFL in Lausanne, Switzerland. He is best known as the inventor of the Scala programming language. His research interests cover fundamental as well as applied aspects of programming languages. The main focus of his work lies in the integration of object-oriented and functional programming in general and on the further development of Scala in particular. He has also influenced the development of Java as a co-designer of Java generics and as the original author of the current javac reference compiler.