San Francisco
June 30 - July 2, 2014

Spark Summit 2014 brought the Apache Spark community together on June 30- July 2, 2014 at the The Westin St. Francis in San Francisco. It featured production users of Spark, Shark, Spark Streaming and related projects.


Spark Summit 2014
Spark Job Server: Easy Spark Job Management
Evan Chan, Kelvin Chu (Ooyala, Inc.)

As your enterprise starts deploying more and more Spark jobs, you will discover many common issues: deploying job jars; managing job lifecycles and progress; serializing and processing job results; keeping track of failures, job statuses, and jars; managing Spark contexts for fast interactive jobs. Also, every Job is an application with its own interface and parameters. Submitting and running jobs Hadoop-style just doesn’t work. Our open-source Spark Job Server offers a RESTful API for managing Spark jobs, jars, and contexts, turning Spark into an easy-to-use service, and offering a uniform API for all jobs. We will talk about our job server, its APIs, current and upcoming features in much greater detail.

Learn how the Spark Job Server can turn Spark into a easy to use service for your organization. As a developer, learn how the job server can let you focus on the job algorithm instead of on nitty gritty infrastructure details.

Evan leads the Compute and Data big data platform team at Ooyala and has led the design and implementation of multiple big data platforms based on Storm, Spark, Kafka, and Cassandra, including a columnar real-time distributed query engine. He co-created the Spark Job Server and leads open-source and blogging efforts at Ooyala. Before Ooyala, he helped create enterprise server OSes at VMware. He holds Bachelor’s and Master’s of Electrical Engineering from Stanford University.

Kelvin is co-creator of Spark Job Server which is an open source RESTful server for submitting, running, and managing Spark jobs, jars and contexts. At Ooyala, he works on its big data infrastructure and builds real-time query engine for video analytics by Spark Job Server. Before joining Ooyala, Kelvin designed and implemented large scale machine learning on recruiting and job seekers data at Jobvite. Kelvin holds a Master’s degree in Computer Science from the University of Maryland at College Park.

Slides PDF |Video