Timeline of machine learning


This page is a timeline of machine learning. Major discoveries, achievements, milestones and other major events are included.

Overview

Timeline

YearEvent typeCaptionEvent
1763DiscoveryThe Underpinnings of Bayes' TheoremThomas Bayes's work An Essay towards solving a Problem in the Doctrine of Chances is published two years after his death, having been amended and edited by a friend of Bayes, Richard Price. The essay presents work which underpins Bayes theorem.
1805DiscoveryLeast SquareAdrien-Marie Legendre describes the "méthode des moindres carrés", known in English as the least squares method. The least squares method is used widely in data fitting.
1812Bayes' TheoremPierre-Simon Laplace publishes Théorie Analytique des Probabilités, in which he expands upon the work of Bayes and defines what is now known as Bayes' Theorem.
1913DiscoveryMarkov ChainsAndrey Markov first describes techniques he used to analyse a poem. The techniques later become known as Markov chains.
1950Turing's Learning MachineAlan Turing proposes a 'learning machine' that could learn and become artificially intelligent. Turing's specific proposal foreshadows genetic algorithms.
1951First Neural Network MachineMarvin Minsky and Dean Edmonds build the first neural network machine, able to learn, the SNARC.
1952Machines Playing CheckersArthur Samuel joins IBM's Poughkeepsie Laboratory and begins working on some of the very first machine learning programs, first creating programs that play checkers.
1957DiscoveryPerceptronFrank Rosenblatt invents the perceptron while working at the Cornell Aeronautical Laboratory. The invention of the perceptron generated a great deal of excitement and was widely covered in the media.
1963AchievementMachines Playing Tic-Tac-ToeDonald Michie creates a 'machine' consisting of 304 match boxes and beads, which uses reinforcement learning to play Tic-tac-toe.
1967Nearest NeighborThe nearest neighbor algorithm was created, which is the start of basic pattern recognition. The algorithm was used to map routes.
1969Limitations of Neural NetworksMarvin Minsky and Seymour Papert publish their book Perceptrons, describing some of the limitations of perceptrons and neural networks. The interpretation that the book shows that neural networks are fundamentally limited is seen as a hindrance for research into neural networks.
1970Automatic Differentiation Seppo Linnainmaa publishes the general method for automatic differentiation of discrete connected networks of nested differentiable functions. This corresponds to the modern version of backpropagation, but is not yet named as such.
1979Stanford CartStudents at Stanford University develop a cart that can navigate and avoid obstacles in a room.
1979DiscoveryNeocognitronKunihiko Fukushima first publishes his work on the neocognitron, a type of artificial neural network. Neocognition later inspires convolutional neural networks.
1981Explanation Based LearningGerald Dejong introduces Explanation Based Learning, where a computer algorithm analyses data and creates a general rule it can follow and discard unimportant data.
1982DiscoveryRecurrent Neural NetworkJohn Hopfield popularizes Hopfield networks, a type of recurrent neural network that can serve as content-addressable memory systems.
1985NetTalkA program that learns to pronounce words the same way a baby does, is developed by Terry Sejnowski.
1986ApplicationBackpropagationSeppo Linnainmaa's reverse mode of automatic differentiation is used in experiments by David Rumelhart, Geoff Hinton and Ronald J. Williams to learn internal representations.
1989DiscoveryReinforcement LearningChristopher Watkins develops Q-learning, which greatly improves the practicality and feasibility of reinforcement learning.
1989CommercializationCommercialization of Machine Learning on Personal ComputersAxcelis, Inc. releases Evolver, the first software package to commercialize the use of genetic algorithms on personal computers.
1992AchievementMachines Playing BackgammonGerald Tesauro develops TD-Gammon, a computer backgammon program that uses an artificial neural network trained using temporal-difference learning. TD-Gammon is able to rival, but not consistently surpass, the abilities of top human backgammon players.
1995DiscoveryRandom Forest AlgorithmTin Kam Ho publishes a paper describing random decision forests.
1995DiscoverySupport Vector MachinesCorinna Cortes and Vladimir Vapnik publish their work on support vector machines.
1997AchievementIBM Deep Blue Beats KasparovIBM's Deep Blue beats the world champion at chess.
1997DiscoveryLSTMSepp Hochreiter and Jürgen Schmidhuber invent long short-term memory recurrent neural networks, greatly improving the efficiency and practicality of recurrent neural networks.
1998MNIST databaseA team led by Yann LeCun releases the MNIST database, a dataset comprising a mix of handwritten digits from American Census Bureau employees and American high school students. The MNIST database has since become a benchmark for evaluating handwriting recognition.
2002Torch Machine Learning LibraryTorch, a software library for machine learning, is first released.
2006The Netflix PrizeThe Netflix Prize competition is launched by Netflix. The aim of the competition was to use machine learning to beat Netflix's own recommendation software's accuracy in predicting a user's rating for a film given their ratings for previous films by at least 10%. The prize was won in 2009.
2009AchievementImageNetImageNet is created. ImageNet is a large visual database envisioned by Fei-Fei Li from Stanford University, who realized that the best machine learning algorithms wouldn't work well if the data didn't reflect the real world. For many, ImageNet was the catalyst for the AI boom of the 21st century.
2010Kaggle CompetitionKaggle, a website that serves as a platform for machine learning competitions, is launched.
2010Wall Street Journal Profiles Machine Learning InvestingThe WSJ Profiles new wave of investing and focuses on RebellionResearch.com which would be the subject of author Scott Patterson's Novel, Dark Pools.
2011AchievementBeating Humans in JeopardyUsing a combination of machine learning, natural language processing and information retrieval techniques, IBM's Watson beats two human champions in a Jeopardy! competition.
2012AchievementRecognizing Cats on YouTubeThe Google Brain team, led by Andrew Ng and Jeff Dean, create a neural network that learns to recognize cats by watching unlabeled images taken from frames of YouTube videos.
2014Leap in Face RecognitionFacebook researchers publish their work on DeepFace, a system that uses neural networks that identifies faces with 97.35% accuracy. The results are an improvement of more than 27% over previous systems and rivals human performance.
2014SibylResearchers from Google detail their work on Sibyl, a proprietary platform for massively parallel machine learning used internally by Google to make predictions about user behavior and provide recommendations.
2016AchievementBeating Humans in GoGoogle's AlphaGo program becomes the first Computer Go program to beat an unhandicapped professional human player using a combination of machine learning and tree search techniques. Later improved as AlphaGo Zero and then in 2017 generalized to Chess and more two-player games with AlphaZero.