Spark Summit 2013 brought the Apache Spark community together on December 2-3, 2013 at the Hotel Nikko in San Francisco. It featured production users of Spark, Shark, Spark Streaming and related projects.
Flint is a collection of software to make it easy to build and manage elastic clusters running on Amazon. The design goal of Flint was to bring all the “BDAS goodness” to Adobe AWS users in a convenient, easy-to-deploy package. In particular, it was designed and built to simplify the setup necessary to get a Spark or Shark cluster running in AWS and to handle issues like using and managing an AWS Spark cluster through a corporate firewall.
I’ll talk about how Flint works (“I’ve got an AWS account, now how do I run Spark?”), some of the “hard parts” of Flint, and how Flint is evolving.