Hadamard product (matrices)


In mathematics, the Hadamard product is a binary operation that takes two matrices of the same dimensions and produces another matrix of the same dimension as the operands where each element is the product of elements of the original two matrices. It should not be confused with the more common matrix product. It is attributed to, and named after, either French mathematician Jacques Hadamard or German mathematician Issai Schur.
The Hadamard product is associative and distributive. Unlike the matrix product, it is also commutative.

Definition

For two matrices and of the same dimension, the Hadamard product is a matrix of the same dimension as the operands, with elements given by
For matrices of different dimensions the Hadamard product is undefined.

Example

For example, the Hadamard product for a 3 × 3 matrix with a 3 × 3 matrix is

Properties

Schur product theorem

The Hadamard product of two positive-semidefinite matrices is positive-semidefinite. This is known as the Schur product theorem, after German mathematician Issai Schur. For two positive-semidefinite matrices and, it is also known that the determinant of their Hadamard product is greater than or equal to the product of their respective determinants:

In programming languages

Hadamard multiplication is built into certain programming languages under various names. In MATLAB, GNU Octave, GAUSS and HP Prime, it is known as array multiplication, or in Julia broadcast multiplication, with the symbol .*. In Fortran, R, APL, J and Wolfram Language, it is done through simple multiplication operator *, whereas the matrix product is done through the function matmul, %*%, +.×, +/.* and the . operators, respectively. In Python with the NumPy numerical library or the SymPy symbolic library, multiplication of array objects as a1*a2 produces the Hadamard product, but otherwise multiplication as a1@a2 or matrix objects m1*m2 will produce a matrix product. The Eigen C++ library provides a cwiseProduct member function for the Matrix class, while the Armadillo library uses the operator % to make compact expressions.

Applications

The Hadamard product appears in lossy compression algorithms such as JPEG. The decoding step involves an entry-for-entry product, in other words the Hadamard product.
It is used in the machine learning literature, for example to describe the architecture of recurrent neural networks as GRUs or LSTMs.

Analogous operations

Other Hadamard operations are also seen in the mathematical literature, namely the Hadamard root and Hadamard power, defined for a matrix such that:
For
and for
The Hadamard inverse reads:
A Hadamard division is defined as: