Databricks automates various steps of the data science workflow including augmented data preparation, visualization, feature engineering, hyperparameter tuning, model search, and finally automatic model tracking, reproducibility, and deployment, through a combination of native product offerings, partnerships, and custom solutions for a fully controlled and transparent AutoML experience.
See for example how you can run hyperparameter tuning at scale on Databricks with enhanced Hyperopt and MLflow integration:
MLflow Experiments Tracking
Track, compare, and visualize hundreds of thousands of experiments using open source or Managed MLflow.
Automated Hyperparameter Tuning for Distributed Machine Learning
Deep integration with PySpark MLlib’s Cross Validation to automatically track MLlib experiments in MLflow.
Automated Hyperparameter Tuning for Single-node Machine Learning
Optimized and distributed hyperparameter search with enhanced Hyperopt and automated tracking to MLflow.
Automated Model Search for Single-node Machine Learning
Optimized and distributed conditional hyperparameter search with enhanced Hyperopt and automated tracking to MLflow.
Microsoft Azure Machine Learning
Azure Databricks integrates with Microsoft Azure Machine Learning and enables access to the service’s automated machine learning capabilities, and together these provide an end-to-end solution for machine learning on Azure.
DataRobot integration on Databricks brings the power of auto-modeling to Databricks users, allowing them to quickly determine and use the best machine learning model for their problem.