Probabilistic programming


Probabilistic programming is a programming paradigm in which probabilistic models are specified and inference for these models is performed automatically. It represents an attempt to unify probabilistic modeling and traditional general purpose programming in order to make the former easier and more widely applicable. It can be used to create systems that help make decisions in the face of uncertainty.
Programming languages used for probabilistic programming are referred to as "Probabilistic programming languages".

Applications

Probabilistic reasoning has been used for a wide variety of tasks such as predicting stock prices, recommending movies, diagnosing computers, detecting cyber intrusions and image detection. However, until recently, probabilistic programming was limited in scope, and most inference algorithms had to be written manually for each task.
Nevertheless, in 2015, a 50-line probabilistic computer vision program was used to generate 3D models of human faces based on 2D images of those faces. The program used inverse graphics as the basis of its inference method, and was built using the Picture package in Julia. This made possible "in 50 lines of code what used to take thousands".
The Gen probabilistic programming library has been applied to vision and robotics tasks.
More recently, the probabilistic programming systems Turing.jl has been applied in various pharmaceutical and economics applications.
Probabilistic programming in Julia has also been combined with differentiable programming by combining the Julia package Zygote.jl with Turing.jl.

Probabilistic programming languages

PPLs often extend from a basic language. The choice of underlying basic language depends on the similarity of the model to the basic language's ontology, as well as commercial considerations and personal preference. For instance, Dimple and Chimple are based on Java, Infer.NET is based on.NET, while PRISM extends from Prolog. However, some PPLs such as WinBUGS and Stan offer a self-contained language, with no obvious origin in another language.
Several PPLs are in active development, including some in beta test. The two most popular tools are Stan and PyMC3.

Relational

A probabilistic relational programming language is a PPL specially designed to describe and infer with probabilistic relational models.
A PRM is usually developed with a set of algorithms for reducing, inference about and discovery of concerned distributions, which are embedded into the corresponding PRPL.

List of probabilistic programming languages

NameExtends fromHost language
AnalyticaC++
bayesloopPythonPython
CuPPLNOVA
VentureSchemeC++
Probabilistic-CCC
AnglicanClojureClojure
IBALOCaml
BayesDBSQLite, Python
PRISMB-Prolog
Infer.NET.NET Framework.NET Framework
dimpleMATLAB, Java
chimpleMATLAB, Java
BLOGJava
delSATAnswer set programming, SAT
PSQLSQL
BUGS
FACTORIEScala
PMTKMATLABMATLAB
AlchemyC++
DynaProlog
FigaroScala
ChurchSchemeVarious: JavaScript, Scheme
ProbLogPrologPython, Jython
ProBTC++, Python
StanC++
HakaruHaskellHaskell
BAli-Phy HaskellC++
ProbCogJava, Python
GambleRacket
PWhileWhilePython
TuffyJava
PyMC3Python, TheanoPython
PyMC4Python, TensorFlow ProbabilityPython
gretaTensorFlowR
pomegranatePythonPython
LeaPythonPython
WebPPLJavaScriptJavaScript
Let's ChanceScratchJavaScript
PictureJuliaJulia
Turing.jlJuliaJulia
GenJuliaJulia
Low-level First-order PPLPython, Clojure, PytorchVarious: Python, Clojure
TrollMoscow ML
EdwardTensorFlowPython
TensorFlow ProbabilityTensorFlowPython
Edward2TensorFlow ProbabilityPython
PyroPyTorchPython
SaulScalaScala
RankPLJava
BirchC++
PSID

Difficulty

Reasoning about variables as probability distributions causes difficulties for novice programmers, but these difficulties can be addressed through use of Bayesian network visualisations and graphs of variable distributions embedded within the source code editor.