Speaker

Min Qiu, Software Engineer at Former Huawei employee

Min Qiu

Software Engineer, Former Huawei employee

Min Qiu is currently a Software Engineer at a stealth mode startup. He was a Staff Software Engineer from Oct 2014 to May 2016 at Huawei Technologies where he worked on building a big data analytics platform based on Apache Spark. He has extensive hands-on experience on Spark SQL. Prior to joining Huawei, he was a Senior Member of the Technical Staff at Oracle, where he worked on the TimesTen In-Memory database team. Min holds a Master’s degree in Computer Science from the University of Wisconsin at Madison.

Sessions

Enhancing Spark SQL Optimizer With Reliable Statistics

In Spark SQL’s Catalyst optimizer, many rule based optimization techniques have been implemented, but the optimizer itself can still be improved. For example, without detailed column statistics information on data distribution, it is difficult to… Read more