Automated machine learning

Part of a series on
Machine learning and data mining
Problems Classification Clustering Regression Anomaly detection Data Cleaning AutoML Association rules Reinforcement learning Structured prediction Feature engineering Feature learning Online learning Semi-supervised learning Unsupervised learning Learning to rank Grammar induction
Supervised learning (classification • regression) Decision trees Ensembles Bagging Boosting Random forest k-NN Linear regression Naive Bayes Artificial neural networks Logistic regression Perceptron Relevance vector machine (RVM) Support vector machine (SVM)
Clustering BIRCH CURE Hierarchical k-means Expectation–maximization (EM) DBSCAN OPTICS Mean shift
Dimensionality reduction Factor analysis CCA ICA LDA NMF PCA PGD t-SNE
Structured prediction Graphical models Bayes net Conditional random field Hidden Markov
Anomaly detection k-NN Local outlier factor
Artificial neural network Autoencoder Cognitive computing Deep learning DeepDream Multilayer perceptron RNN LSTM GRU ESN Restricted Boltzmann machine GAN SOM Convolutional neural network U-Net Transformer Vision Spiking neural network Memtransistor Electrochemical RAM (ECRAM)
Reinforcement learning Q-learning SARSA Temporal difference (TD)
Theory Kernel machines Bias–variance tradeoff Computational learning theory Empirical risk minimization Occam learning PAC learning Statistical learning VC theory
Machine-learning venues NeurIPS ICML ML JMLR ArXiv:cs.LG
Related articles Glossary of artificial intelligence List of datasets for machine-learning research Outline of machine learning
v t

Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems. AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning.^[1][2] The high degree of automation in AutoML aims to allows non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning. Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models.

Comparison to the standard approach[]

In a typical machine learning application, practitioners have a set of input data points to be used for training. The raw data may not be in a form that all algorithms can be applied to it. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. Each of these steps may be challenging, resulting in significant hurdles to using machine learning.

AutoML aims to simplify these steps for non-experts.

Targets of automation[]

Automated machine learning can target various stages of the machine learning process.^[2] Steps to automate are:

• Data preparation and ingestion (from raw data and miscellaneous formats)
  • Column type detection; e.g., boolean, discrete numerical, continuous numerical, or text
  • Column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature
  • Task detection; e.g., binary classification, regression, clustering, or ranking
• Feature engineering
  • Feature selection
  • Feature extraction
  • Meta learning and transfer learning
  • Detection and handling of skewed data and/or missing values
• Model selection - choosing which machine learning algorithm to use, often including multiple competing software implementations
• Ensembling - a form of consensus where using multiple models often gives better results than any single model^[3]
• Hyperparameter optimization of the learning algorithm and featurization
• Pipeline selection under time, memory, and complexity constraints
• Selection of evaluation metrics and validation procedures
• Problem checking
  • Leakage detection
  • Misconfiguration detection
• Analysis of obtained results
• Creating user interfaces and visualizations

See also[]

• Neural architecture search
• Neuroevolution
• Self-tuning
• Neural Network Intelligence
• AutoAI
• ModelOps

References[]

Further reading[]

• "Open Source AutoML Tools: AutoGluon, TransmogrifAI, Auto-sklearn, and NNI". Bizety. 2020-06-16.