Photo of

Andrew Or

Software Engineer, Databricks

Andrew is a Spark PMC member. In the past, he has contributed several large features to the project, including event logging, external spilling, history server, dynamic allocation, and DAG visualization on the SparkUI. He is an active maintainer of the Spark on YARN integration component.


SparkUI visualization: a lens into your application

Your application is slow but you don’t know why. It could be doing unnecessary shuffles, or evicting heavily-used cached data from memory, or suffering from data skew, or… With the recent visualization additions to the…