Speaker

Shriya Arora, Data Engineer at Netflix

Shriya Arora

Data Engineer, Netflix

I am a data engineer at Netflix in the Data Personalization team that is responsible for generating datasets that are used for machine learning pipelines that power the Netflix recommendations. We have been actively using Spark over Pig/Hive for our batch jobs and are now exploring Spark streaming.
Before Netflix, I was at Walmart Labs, where I helped build and architect their new generation item-setup, moving from batch processing to stream .We used Storm-Kafka to enable a micro-services architecture that can allow for products to be updated near real-time as opposed to once-a-day update on the legacy framework.

Sessions

Going Real-Time: Creating Frequently-Updating Datasets for Personalization

Streaming applications have often been complex to design and maintain because of the significant upfront infrastructure investment required. However, with the advent of Spark an easy transition to stream processing is now available, enabling personalization… Read more