Photo of

Josh Rosen

Software Engineer, Databricks

Josh Rosen is a Spark Committer working at Databricks. Previously, he was a grad student at UC Berkeley, studying databases and distributed systems. He has been instrumental in the design and implementation of many Spark components, including PySpark and Tungsten.


Deep Dive into Project Tungsten: Bringing Spark Closer to Bare Metal

Project Tungsten focuses on substantially improving the efficiency of memory and CPU for Spark applications, to push performance closer to the limits of modern hardware. This effort includes three initiatives: 1. Memory Management and Binary…