Outline of machine learning

From Wikipedia, the free encyclopedia
Part of a series on
Machine learning
and data mining
Multi-Layer Neural Network-Vector-Blank.svg
hide
Problems
  • Classification
  • Clustering
  • Regression
  • Anomaly detection
  • Data Cleaning
  • AutoML
  • Association rules
  • Reinforcement learning
  • Structured prediction
  • Feature engineering
  • Feature learning
  • Online learning
  • Semi-supervised learning
  • Unsupervised learning
  • Learning to rank
  • Grammar induction
hide
Supervised learning
(classification • regression)
  • Decision trees
  • Ensembles
    • Bagging
    • Boosting
    • Random forest
  • k-NN
  • Linear regression
  • Naive Bayes
  • Artificial neural networks
  • Logistic regression
  • Perceptron
  • Relevance vector machine (RVM)
  • Support vector machine (SVM)
hide
Clustering
hide
Dimensionality reduction
  • Factor analysis
  • CCA
  • ICA
  • LDA
  • NMF
  • PCA
  • PGD
  • t-SNE
hide
Structured prediction
  • Graphical models
    • Bayes net
    • Conditional random field
    • Hidden Markov
hide
Anomaly detection
  • k-NN
  • Local outlier factor
hide
Artificial neural network
  • Autoencoder
  • Cognitive computing
  • Deep learning
  • DeepDream
  • Multilayer perceptron
  • RNN
    • LSTM
    • GRU
    • ESN
  • Restricted Boltzmann machine
  • GAN
  • SOM
  • Convolutional neural network
    • U-Net
  • Transformer
  • Spiking neural network
  • Memtransistor
  • Electrochemical RAM (ECRAM)
hide
Reinforcement learning
  • Q-learning
  • SARSA
  • Temporal difference (TD)
hide
Theory
hide
Machine-learning venues
hide
Related articles
  • v
  • t

The following outline is provided as an overview of and topical guide to machine learning. Machine learning is a subfield of soft computing within computer science that evolved from the study of pattern recognition and computational learning theory in artificial intelligence.[1] In 1959, Arthur Samuel defined machine learning as a "field of study that gives computers the ability to learn without being explicitly programmed".[2] Machine learning explores the study and construction of algorithms that can learn from and make predictions on data.[3] Such algorithms operate by building a model from an example training set of input observations in order to make data-driven predictions or decisions expressed as outputs, rather than following strictly static program instructions.

What type of thing is machine learning?[]

  • An academic discipline
  • A branch of science
    • An applied science
      • A subfield of computer science
        • A branch of artificial intelligence
        • A subfield of soft computing
      • Application of statistics

Branches of machine learning[]

Subfields of machine learning[]

  • Computational learning theory – studying the design and analysis of machine learning algorithms.[4]
  • Grammar induction
  • Meta learning

Cross-disciplinary fields involving machine learning[]

  • Adversarial machine learning
  • Predictive analytics
  • Quantum machine learning
  • Robot learning
    • Developmental robotics

Applications of machine learning[]

  • Applications of machine learning
  • Bioinformatics
  • Biomedical informatics
  • Computer vision
  • Customer relationship management –
  • Data mining
  • Email filtering
  • Inverted pendulum – balance and equilibrium system.
  • Natural language processing (NLP)
    • Automatic summarization
    • Automatic taxonomy construction
    • Dialog system
    • Grammar checker
    • Language recognition
      • Handwriting recognition
      • Optical character recognition
      • Speech recognition
    • Machine translation
    • Question answering
    • Speech synthesis
    • Text mining
    • Text simplification
  • Pattern recognition
    • Facial recognition system
    • Handwriting recognition
    • Image recognition
    • Optical character recognition
    • Speech recognition
  • Recommendation system
    • Collaborative filtering
    • Content-based filtering
    • Hybrid recommender systems (Collaborative and content-based filtering)
  • Search engine
    • Search engine optimization
  • Social Engineering

Machine learning hardware[]

Machine learning tools[]

Machine learning frameworks[]

Proprietary machine learning frameworks[]

  • Amazon Machine Learning
  • Microsoft Azure Machine Learning Studio
  • DistBelief – replaced by TensorFlow

Open source machine learning frameworks[]

  • Apache Singa
  • Apache MXNet
  • Caffe
  • PyTorch
  • mlpack
  • TensorFlow
  • Torch
  • CNTK
  • Accord.Net

Machine learning libraries[]

Machine learning algorithms[]

Machine learning methods[]

Instance-based algorithm[]

  • K-nearest neighbors algorithm (KNN)
  • Learning vector quantization (LVQ)
  • Self-organizing map (SOM)

Regression analysis[]

  • Logistic regression
  • Ordinary least squares regression (OLSR)
  • Linear regression
  • Stepwise regression
  • Multivariate adaptive regression splines (MARS)
  • Regularization algorithm
    • Ridge regression
    • Least Absolute Shrinkage and Selection Operator (LASSO)
    • Elastic net
    • Least-angle regression (LARS)
  • Classifiers
    • Probabilistic classifier
      • Naive Bayes classifier
    • Binary classifier
    • Linear classifier
    • Hierarchical classifier

Dimensionality reduction[]

Dimensionality reduction

  • Canonical correlation analysis (CCA)
  • Factor analysis
  • Feature extraction
  • Feature selection
  • Independent component analysis (ICA)
  • Linear discriminant analysis (LDA)
  • Multidimensional scaling (MDS)
  • Non-negative matrix factorization (NMF)
  • Partial least squares regression (PLSR)
  • Principal component analysis (PCA)
  • Principal component regression (PCR)
  • Projection pursuit
  • Sammon mapping
  • t-distributed stochastic neighbor embedding (t-SNE)

Ensemble learning[]

Ensemble learning

  • AdaBoost
  • Boosting
  • Bootstrap aggregating (Bagging)
  • Ensemble averaging – process of creating multiple models and combining them to produce a desired output, as opposed to creating just one model. Frequently an ensemble of models performs better than any individual model, because the various errors of the models "average out."
  • Gradient boosted decision tree (GBDT)
  • Gradient boosting machine (GBM)
  • Random Forest
  • Stacked Generalization (blending)

Meta learning[]

Meta learning

  • Inductive bias
  • Metadata

Reinforcement learning[]

Reinforcement learning

Supervised learning[]

Supervised learning

  • AODE
  • Artificial neural network
  • Association rule learning algorithms
    • Apriori algorithm
    • Eclat algorithm
  • Case-based reasoning
  • Gaussian process regression
  • Gene expression programming
  • Group method of data handling (GMDH)
  • Inductive logic programming
  • Instance-based learning
  • Lazy learning
  • Learning Automata
  • Learning Vector Quantization
  • Logistic Model Tree
  • Minimum message length (decision trees, decision graphs, etc.)
    • Nearest Neighbor Algorithm
    • Analogical modeling
  • Probably approximately correct learning (PAC) learning
  • Ripple down rules, a knowledge acquisition methodology
  • Symbolic machine learning algorithms
  • Support vector machines
  • Random Forests
  • Ensembles of classifiers
    • Bootstrap aggregating (bagging)
    • Boosting (meta-algorithm)
  • Ordinal classification
  • Information fuzzy networks (IFN)
  • Conditional Random Field
  • ANOVA
  • Quadratic classifiers
  • k-nearest neighbor
  • Boosting
    • SPRINT
  • Bayesian networks
    • Naive Bayes
  • Hidden Markov models

Bayesian[]

Bayesian statistics

  • Bayesian knowledge base
  • Naive Bayes
  • Gaussian Naive Bayes
  • Multinomial Naive Bayes
  • Averaged One-Dependence Estimators (AODE)
  • Bayesian Belief Network (BBN)
  • Bayesian Network (BN)

Decision tree algorithms[]

Decision tree algorithm

  • Decision tree
  • Classification and regression tree (CART)
  • Iterative Dichotomiser 3 (ID3)
  • C4.5 algorithm
  • C5.0 algorithm
  • Chi-squared Automatic Interaction Detection (CHAID)
  • Decision stump
  • Conditional decision tree
  • ID3 algorithm
  • Random forest
  • SLIQ

Linear classifier[]

Linear classifier

  • Fisher's linear discriminant
  • Linear regression
  • Logistic regression
  • Multinomial logistic regression
  • Naive Bayes classifier
  • Perceptron
  • Support vector machine

Unsupervised learning[]

Unsupervised learning

  • Expectation-maximization algorithm
  • Vector Quantization
  • Generative topographic map
  • Information bottleneck method

Artificial neural networks[]

Artificial neural network

  • Feedforward neural network
    • Extreme learning machine
    • Convolutional neural network
  • Recurrent neural network
    • Long short-term memory (LSTM)
  • Logic learning machine
  • Self-organizing map

Association rule learning[]

Association rule learning

  • Apriori algorithm
  • Eclat algorithm
  • FP-growth algorithm

Hierarchical clustering[]

Hierarchical clustering

  • Single-linkage clustering
  • Conceptual clustering

Cluster analysis[]

Cluster analysis

  • BIRCH
  • DBSCAN
  • Expectation-maximization (EM)
  • Fuzzy clustering
  • Hierarchical Clustering
  • K-means clustering
  • K-medians
  • Mean-shift
  • OPTICS algorithm

Anomaly detection[]

Anomaly detection

  • k-nearest neighbors classification (k-NN)
  • Local outlier factor

Semi-supervised learning[]

Semi-supervised learning

  • Active learning – special case of semi-supervised learning in which a learning algorithm is able to interactively query the user (or some other information source) to obtain the desired outputs at new data points.[5][6]
  • Generative models
  • Low-density separation
  • Graph-based methods
  • Co-training
  • Transduction

Deep learning[]

Deep learning

  • Deep belief networks
  • Deep Boltzmann machines
  • Deep Convolutional neural networks
  • Deep Recurrent neural networks
  • Hierarchical temporal memory
  • Generative Adversarial Networks
  • Deep Boltzmann Machine (DBM)
  • Stacked Auto-Encoders

Other machine learning methods and problems[]

  • Anomaly detection
  • Association rules
  • Bias-variance dilemma
  • Classification
    • Multi-label classification
  • Clustering
  • Data Pre-processing
  • Empirical risk minimization
  • Feature engineering
  • Feature learning
  • Learning to rank
  • Occam learning
  • Online machine learning
  • PAC learning
  • Regression
  • Reinforcement Learning
  • Semi-supervised learning
  • Statistical learning
  • Structured prediction
    • Graphical models
      • Bayesian network
      • Conditional random field (CRF)
      • Hidden Markov model (HMM)
  • Unsupervised learning
  • VC theory

Machine learning research[]

History of machine learning[]

History of machine learning

  • Timeline of machine learning

Machine learning projects[]

Machine learning projects

  • DeepMind
  • Google Brain
  • OpenAI

Machine learning organizations[]

Machine learning organizations

  • Knowledge Engineering and Machine Learning Group

Machine learning conferences and workshops[]

  • Artificial Intelligence and Security (AISec) (co-located workshop with CCS)
  • Conference on Neural Information Processing Systems (NIPS)
  • ECML PKDD
  • International Conference on Machine Learning (ICML)
  • ML4ALL (Machine Learning For All)

Machine learning publications[]

Books on machine learning[]

Books about machine learning

Machine learning journals[]

Persons influential in machine learning[]

  • Alberto Broggi
  • Andrei Knyazev
  • Andrew McCallum
  • Andrew Ng
  • Armin B. Cremers
  • Ayanna Howard
  • Barney Pell
  • Ben Goertzel
  • Ben Taskar
  • Bernhard Schölkopf
  • Brian D. Ripley
  • Christopher G. Atkeson
  • Corinna Cortes
  • Demis Hassabis
  • Douglas Lenat
  • Eric Xing
  • Ernst Dickmanns
  • Geoffrey Hinton – co-inventor of the backpropagation and contrastive divergence training algorithms
  • Hans-Peter Kriegel
  • Hartmut Neven
  • Heikki Mannila
  • Ian Goodfellow – Father of Generative & adversarial networks
  • Jacek M. Zurada
  • Jaime Carbonell
  • Jerome H. Friedman
  • John D. Lafferty
  • John Platt – invented SMO and Platt scaling
  • Julie Beth Lovins
  • Jürgen Schmidhuber
  • Karl Steinbuch
  • Katia Sycara
  • Leo Breiman – invented bagging and random forests
  • Lise Getoor
  • Luca Maria Gambardella
  • Léon Bottou
  • Marcus Hutter
  • Mehryar Mohri
  • Michael Collins
  • Michael I. Jordan
  • Michael L. Littman
  • Nando de Freitas
  • Ofer Dekel
  • Oren Etzioni
  • Pedro Domingos
  • Peter Flach
  • Pierre Baldi
  • Pushmeet Kohli
  • Ray Kurzweil
  • Rayid Ghani
  • Ross Quinlan
  • Salvatore J. Stolfo
  • Sebastian Thrun
  • Selmer Bringsjord
  • Sepp Hochreiter
  • Shane Legg
  • Stephen Muggleton
  • Steve Omohundro
  • Tom M. Mitchell
  • Trevor Hastie
  • Vasant Honavar
  • Vladimir Vapnik – co-inventor of the SVM and VC theory
  • Yann LeCun – invented convolutional neural networks
  • Yasuo Matsuyama
  • Yoshua Bengio
  • Zoubin Ghahramani

See also[]

Other[]

Further reading[]

  • Trevor Hastie, Robert Tibshirani and Jerome H. Friedman (2001). The Elements of Statistical Learning, Springer. ISBN 0-387-95284-5.
  • Pedro Domingos (September 2015), The Master Algorithm, Basic Books, ISBN 978-0-465-06570-7
  • Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar (2012). Foundations of Machine Learning, The MIT Press. ISBN 978-0-262-01825-8.
  • Ian H. Witten and Eibe Frank (2011). Data Mining: Practical machine learning tools and techniques Morgan Kaufmann, 664pp., ISBN 978-0-12-374856-0.
  • David J. C. MacKay. Information Theory, Inference, and Learning Algorithms Cambridge: Cambridge University Press, 2003. ISBN 0-521-64298-1
  • Richard O. Duda, Peter E. Hart, David G. Stork (2001) Pattern classification (2nd edition), Wiley, New York, ISBN 0-471-05669-3.
  • Christopher Bishop (1995). Neural Networks for Pattern Recognition, Oxford University Press. ISBN 0-19-853864-2.
  • Vladimir Vapnik (1998). Statistical Learning Theory. Wiley-Interscience, ISBN 0-471-03003-1.
  • Ray Solomonoff, An Inductive Inference Machine, IRE Convention Record, Section on Information Theory, Part 2, pp., 56–62, 1957.
  • Ray Solomonoff, "An Inductive Inference Machine" A privately circulated report from the 1956 Dartmouth Summer Research Conference on AI.

References[]

  1. ^ http://www.britannica.com/EBchecked/topic/1116194/machine-learning  This tertiary source reuses information from other sources but does not name them.
  2. ^ Phil Simon (March 18, 2013). Too Big to Ignore: The Business Case for Big Data. Wiley. p. 89. ISBN 978-1-118-63817-0.
  3. ^ Ron Kohavi; Foster Provost (1998). "Glossary of terms". Machine Learning. 30: 271–274. doi:10.1023/A:1007411609915.
  4. ^ "ACL - Association for Computational Learning".
  5. ^ Settles, Burr (2010), "Active Learning Literature Survey" (PDF), Computer Sciences Technical Report 1648. University of Wisconsin–Madison, retrieved 2014-11-18
  6. ^ Rubens, Neil; Elahi, Mehdi; Sugiyama, Masashi; Kaplan, Dain (2016). "Active Learning in Recommender Systems". In Ricci, Francesco; Rokach, Lior; Shapira, Bracha (eds.). Recommender Systems Handbook (2 ed.). Springer US. doi:10.1007/978-1-4899-7637-6. hdl:11311/1006123. ISBN 978-1-4899-7637-6. S2CID 11569603.

External links[]

Retrieved from ""