Speaker

Ed Barnes

Performance Architect, Sqrrl Data

Edwin Barnes is the performance architect at Sqrrl Data. He leads efforts in design and deployment of large scale performance testing systems for high-throughput distributed data processing and analytics applications. Previously he worked at a range of top tech and data companies including Vertica Systems, Unidesk and Dataupia. He has a vast experience in cloud computing and is a leading expert in quality assurance and performance optimization of distributed systems.

Sessions

Monitoring the Dynamic Resource Usage of Scala and Python Spark Jobs in Yarn

We all dread “Lost task” and “Container killed by YARN for exceeding memory limits” messages in our scaled-up spark yarn applications. Even answering the question “How much memory did my application use?” is surprisingly tricky… Read more