Speaker

Tathagata Das, Software Engineer at Databricks

Tathagata Das

Software Engineer, Databricks

Tathagata Das is an Apache Spark committer and a member of the PMC. He’s the lead developer behind Spark Streaming, which he started while a PhD student in the UC Berkeley AMPLab, and is currently employed at Databricks. Prior to Databricks, Tathagata worked at the AMPLab, conducting research about data-center frameworks and networks with Scott Shenker and Ion Stoica.

Sessions

Easy, Scalable, Fault-Tolerant Stream Processing with Structured Streaming in Apache Spark

Last year, in Apache Spark 2.0, Databricks introduced Structured Streaming, a new stream processing engine built on Spark SQL, which revolutionized how developers could write stream processing application. Structured Streaming enables users to express their… Read more

Easy, Scalable, Fault-Tolerant Stream Processing with Structured Streaming in Apache Spark - continues

Last year, in Apache Spark 2.0, Databricks introduced Structured Streaming, a new stream processing engine built on Spark SQL, which revolutionized how developers could write stream processing application. Structured Streaming enables users to express their… Read more

Deep Dive into Stateful Stream Processing in Structured Streaming

Stateful processing is one of the most challenging aspects of distributed, fault-tolerant stream processing. The DataFrame APIs in Structured Streaming make it very easy for the developer to express their stateful logic, either implicitly (streaming… Read more