In mathematics, a symmetric matrix \ M\ with real entries is positive-definite if the real number is positive for every nonzero real column vector where is the row vector transpose of More generally, a Hermitian matrix (that is, a complex matrix equal to its conjugate transpose) is positive-definite if the real number is positive for every nonzero complex column vector where denotes the conjugate transpose of Positive semi-definite matrices are defined similarly, except that the scalars and are required to be positive or zero (that is, nonnegative). Negative-definite and negative semi-definite matrices are defined analogously. A matrix that is not positive semi-definite and not negative semi-definite is sometimes called indefinite.

Ramifications

It follows from the above definitions that a matrix is positive-definite if and only if it is the matrix of a positive-definite quadratic form or Hermitian form. In other words, a matrix is positive-definite if and only if it defines an inner product. Positive-definite and positive-semidefinite matrices can be characterized in many ways, which may explain the importance of the concept in various parts of mathematics. A matrix M is positive-definite if and only if it satisfies any of the following equivalent conditions. A matrix is positive semi-definite if it satisfies similar equivalent conditions where "positive" is replaced by "nonnegative", "invertible matrix" is replaced by "matrix", and the word "leading" is removed. Positive-definite and positive-semidefinite real matrices are at the basis of convex optimization, since, given a function of several real variables that is twice differentiable, then if its Hessian matrix (matrix of its second partial derivatives) is positive-definite at a point \ p, then the function is convex near p, and, conversely, if the function is convex near \ p\ , then the Hessian matrix is positive-semidefinite at \ p ~. The set of positive definite matrices is an open convex cone, while the set of positive semi-definite matrices is a closed convex cone. Some authors use more general definitions of definiteness, including some non-symmetric real matrices, or non-Hermitian complex ones.

Definitions

In the following definitions, is the transpose of is the conjugate transpose of and denotes the n dimensional zero-vector.

Definitions for real matrices

An n \times n symmetric real matrix \ M\ is said to be positive-definite if for all non-zero in Formally, An symmetric real matrix \ M\ is said to be positive-semidefinite or non-negative-definite if for all in Formally, An symmetric real matrix \ M\ is said to be negative-definite if for all non-zero in \ \R^n ~. Formally, An symmetric real matrix \ M\ is said to be negative-semidefinite or non-positive-definite if for all in Formally, An n \times n symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

Definitions for complex matrices

The following definitions all involve the term Notice that this is always a real number for any Hermitian square matrix \ M ~. An Hermitian complex matrix \ M\ is said to be positive-definite if for all non-zero in Formally, An Hermitian complex matrix \ M\ is said to be positive semi-definite or non-negative-definite if for all in Formally, An Hermitian complex matrix \ M\ is said to be negative-definite if for all non-zero in Formally, An Hermitian complex matrix \ M\ is said to be negative semi-definite or non-positive-definite if for all in Formally, An Hermitian complex matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

Consistency between real and complex definitions

Since every real matrix is also a complex matrix, the definitions of "definiteness" for the two classes must agree. For complex matrices, the most common definition says that \ M\ is positive-definite if and only if is real and positive for every non-zero complex column vectors This condition implies that M is Hermitian (i.e. its transpose is equal to its conjugate), since being real, it equals its conjugate transpose for every which implies By this definition, a positive-definite real matrix \ M\ is Hermitian, hence symmetric; and is positive for all non-zero real column vectors However the last condition alone is not sufficient for \ M\ to be positive-definite. For example, if then for any real vector with entries \ a\ and \ b\ we have which is always positive if is not zero. However, if is the complex vector with entries 1 and \ i, one gets which is not real. Therefore, \ M\ is not positive-definite. On the other hand, for a symmetric real matrix \ M, the condition " for all nonzero real vectors does imply that \ M\ is positive-definite in the complex sense.

Notation

If a Hermitian matrix \ M\ is positive semi-definite, one sometimes writes and if \ M\ is positive-definite one writes To denote that \ M\ is negative semi-definite one writes and to denote that \ M\ is negative-definite one writes The notion comes from functional analysis where positive semidefinite matrices define positive operators. If two matrices \ A\ and \ B\ satisfy we can define a non-strict partial order that is reflexive, antisymmetric, and transitive; It is not a total order, however, as \ B - A, in general, may be indefinite. A common alternative notation is \ M > 0, and \ M < 0\ for positive semi-definite and positive-definite, negative semi-definite and negative-definite matrices, respectively. This may be confusing, as sometimes nonnegative matrices (respectively, nonpositive matrices) are also denoted in this way.

Examples

Eigenvalues

Let M be an n \times n Hermitian matrix (this includes real symmetric matrices). All eigenvalues of M are real, and their sign characterize its definiteness: Let be an eigendecomposition of \ M, where \ P\ is a unitary complex matrix whose columns comprise an orthonormal basis of eigenvectors of \ M\ , and \ D\ is a real diagonal matrix whose main diagonal contains the corresponding eigenvalues. The matrix \ M\ may be regarded as a diagonal matrix \ D\ that has been re-expressed in coordinates of the (eigenvectors) basis \ P ~. Put differently, applying M to some vector giving is the same as changing the basis to the eigenvector coordinate system using \ P^{-1}, giving applying the stretching transformation \ D\ to the result, giving and then changing the basis back using \ P\ , giving With this in mind, the one-to-one change of variable shows that is real and positive for any complex vector if and only if is real and positive for any \ y\ ; in other words, if \ D\ is positive definite. For a diagonal matrix, this is true only if each element of the main diagonal – that is, every eigenvalue of \ M\ – is positive. Since the spectral theorem guarantees all eigenvalues of a Hermitian matrix to be real, the positivity of eigenvalues can be checked using Descartes' rule of alternating signs when the characteristic polynomial of a real, symmetric matrix \ M\ is available.

Decomposition

Let \ M\ be an Hermitian matrix. \ M\ is positive semidefinite if and only if it can be decomposed as a product of a matrix \ B\ with its conjugate transpose. When \ M\ is real, \ B\ can be real as well and the decomposition can be written as M is positive definite if and only if such a decomposition exists with \ B\ invertible. More generally, \ M\ is positive semidefinite with rank \ k\ if and only if a decomposition exists with a matrix \ B\ of full row rank (i.e. of rank \ k). Moreover, for any decomposition The columns of \ B\ can be seen as vectors in the complex or real vector space respectively. Then the entries of M are inner products (that is dot products, in the real case) of these vectors In other words, a Hermitian matrix \ M\ is positive semidefinite if and only if it is the Gram matrix of some vectors It is positive definite if and only if it is the Gram matrix of some linearly independent vectors. In general, the rank of the Gram matrix of vectors equals the dimension of the space spanned by these vectors.

Uniqueness up to unitary transformations

The decomposition is not unique: if for some matrix \ B\ and if \ Q\ is any unitary k \times k matrix (meaning ), then for However, this is the only way in which two decompositions can differ: The decomposition is unique up to unitary transformations. More formally, if \ A\ is a matrix and \ B\ is a matrix such that then there is a matrix Q with orthonormal columns (meaning ) such that When \ell = k this means Q is unitary. This statement has an intuitive geometric interpretation in the real case: let the columns of A and B be the vectors and in A real unitary matrix is an orthogonal matrix, which describes a rigid transformation (an isometry of Euclidean space ) preserving the 0 point (i.e. rotations and reflections, without translations). Therefore, the dot products and are equal if and only if some rigid transformation of transforms the vectors to (and 0 to 0).

Square root

A Hermitian matrix \ M\ is positive semidefinite if and only if there is a positive semidefinite matrix \ B\ (in particular \ B\ is Hermitian, so \ B^* = B) satisfying This matrix \ B\ is unique, is called the non-negative square root of \ M, and is denoted with When M is positive definite, so is hence it is also called the positive square root of \ M ~. The non-negative square root should not be confused with other decompositions Some authors use the name square root and for any such decomposition, or specifically for the Cholesky decomposition, or any decomposition of the form others only use it for the non-negative square root. If then

Cholesky decomposition

A Hermitian positive semidefinite matrix \ M\ can be written as where \ L\ is lower triangular with non-negative diagonal (equivalently M = B^B where \ B = L^\ is upper triangular); this is the Cholesky decomposition. If \ M\ is positive definite, then the diagonal of \ L\ is positive and the Cholesky decomposition is unique. Conversely if \ L\ is lower triangular with nonnegative diagonal then \ L L^*\ is positive semidefinite. The Cholesky decomposition is especially useful for efficient numerical calculations. A closely related decomposition is the LDL decomposition, where D is diagonal and \ L\ is lower unitriangular.

Williamson theorem

Any 2n\times 2n positive definite Hermitian real matrix M can be diagonalized via symplectic (real) matrices. More precisely, Williamson's theorem ensures the existence of symplectic and diagonal real positive such that.

Other characterizations

Let \ M\ be an real symmetric matrix, and let be the "unit ball" defined by \ M ~. Then we have the following Let M be an Hermitian matrix. The following properties are equivalent to \ M\ being positive definite: A positive semidefinite matrix is positive definite if and only if it is invertible. A matrix \ M\ is negative (semi)definite if and only if \ -M\ is positive (semi)definite.

Quadratic forms

The (purely) quadratic form associated with a real matrix \ M\ is the function such that for all \ M\ can be assumed symmetric by replacing it with since any asymmetric part will be zeroed-out in the double-sided product. A symmetric matrix \ M\ is positive definite if and only if its quadratic form is a strictly convex function. More generally, any quadratic function from to \mathbb{R} can be written as where \ M\ is a symmetric matrix, is a real n vector, and \ c\ a real constant. In the \ n = 1\ case, this is a parabola, and just like in the \ n = 1\ case, we have Theorem: This quadratic function is strictly convex, and hence has a unique finite global minimum, if and only if \ M\ is positive definite. Proof: If \ M\ is positive definite, then the function is strictly convex. Its gradient is zero at the unique point of which must be the global minimum since the function is strictly convex. If \ M\ is not positive definite, then there exists some vector such that so the function is a line or a downward parabola, thus not strictly convex and not having a global minimum. For this reason, positive definite matrices play an important role in optimization problems.

Simultaneous diagonalization

One symmetric matrix and another matrix that is both symmetric and positive definite can be simultaneously diagonalized. This is so although simultaneous diagonalization is not necessarily performed with a similarity transformation. This result does not extend to the case of three or more matrices. In this section we write for the real case. Extension to the complex case is immediate. Let \ M\ be a symmetric and \ N\ a symmetric and positive definite matrix. Write the generalized eigenvalue equation as where we impose that be normalized, i.e. Now we use Cholesky decomposition to write the inverse of \ N\ as Multiplying by \ Q\ and letting we get which can be rewritten as where Manipulation now yields where X is a matrix having as columns the generalized eigenvectors and \Lambda is a diagonal matrix of the generalized eigenvalues. Now premultiplication with X^\top gives the final result: and but note that this is no longer an orthogonal diagonalization with respect to the inner product where In fact, we diagonalized \ M\ with respect to the inner product induced by \ N ~. Note that this result does not contradict what is said on simultaneous diagonalization in the article Diagonalizable matrix, which refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to a simultaneous diagonalization of two quadratic forms, and is useful for optimization of one form under conditions on the other.

Properties

Induced partial ordering

For arbitrary square matrices \ M, \ N\ we write \ M \ge N\ if i.e., \ M - N\ is positive semi-definite. This defines a partial ordering on the set of all square matrices. One can similarly define a strict partial ordering \ M > N ~. The ordering is called the Loewner order.

Inverse of positive definite matrix

Every positive definite matrix is invertible and its inverse is also positive definite. If then Moreover, by the min-max theorem, the kth largest eigenvalue of \ M\ is greater than or equal to the kth largest eigenvalue of \ N ~.

Scaling

If \ M\ is positive definite and \ r > 0\ is a real number, then \ r M\ is positive definite.

Addition

Multiplication

Trace

The diagonal entries \ m_{ii}\ of a positive-semidefinite matrix are real and non-negative. As a consequence the trace, Furthermore, since every principal sub-matrix (in particular, 2-by-2) is positive semidefinite, and thus, when An n \times n Hermitian matrix M is positive definite if it satisfies the following trace inequalities: Another important result is that for any \ M\ and N positive-semidefinite matrices, This follows by writing The matrix is positive-semidefinite and thus has non-negative eigenvalues, whose sum, the trace, is therefore also non-negative.

Hadamard product

If although \ M N\ is not necessary positive semidefinite, the Hadamard product is, (this result is often called the Schur product theorem). Regarding the Hadamard product of two positive semidefinite matrices there are two notable inequalities:

Kronecker product

If although \ M N\ is not necessary positive semidefinite, the Kronecker product

Frobenius product

If although \ M N\ is not necessary positive semidefinite, the Frobenius inner product (Lancaster–Tismenetsky, The Theory of Matrices, p. 218).

Convexity

The set of positive semidefinite symmetric matrices is convex. That is, if \ M\ and \ N\ are positive semidefinite, then for any \ \alpha\ between 0 and 1 , is also positive semidefinite. For any vector : This property guarantees that semidefinite programming problems converge to a globally optimal solution.

Relation with cosine

The positive-definiteness of a matrix A expresses that the angle \ \theta\ between any vector and its image is always the angle between and

Further properties

0 and −1.

Block matrices and submatrices

A positive matrix may also be defined by blocks: where each block is By applying the positivity condition, it immediately follows that \ A\ and \ D\ are hermitian, and We have that for all complex and in particular for Then A similar argument can be applied to \ D, and thus we conclude that both \ A\ and \ D\ must be positive definite. The argument can be extended to show that any principal submatrix of \ M\ is itself positive definite. Converse results can be proved with stronger conditions on the blocks, for instance, using the Schur complement.

Local extrema

A general quadratic form on n real variables can always be written as where \mathbf{x} is the column vector with those variables, and M is a symmetric real matrix. Therefore, the matrix being positive definite means that f has a unique minimum (zero) when \mathbf{x} is zero, and is strictly positive for any other More generally, a twice-differentiable real function f on n real variables has local minimum at arguments if its gradient is zero and its Hessian (the matrix of all second derivatives) is positive semi-definite at that point. Similar statements can be made for negative definite and semi-definite matrices.

Covariance

In statistics, the covariance matrix of a multivariate probability distribution is always positive semi-definite; and it is positive definite unless one variable is an exact linear function of the others. Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.

Extension for non-Hermitian square matrices

The definition of positive definite can be generalized by designating any complex matrix \ M\ (e.g. real non-symmetric) as positive definite if for all non-zero complex vectors where denotes the real part of a complex number \ c ~. Only the Hermitian part determines whether the matrix is positive definite, and is assessed in the narrower sense above. Similarly, if and \ M\ are real, we have for all real nonzero vectors if and only if the symmetric part is positive definite in the narrower sense. It is immediately clear that is insensitive to transposition of \ M ~. Consequently, a non-symmetric real matrix with only positive eigenvalues does not need to be positive definite. For example, the matrix has positive eigenvalues yet is not positive definite; in particular a negative value of is obtained with the choice (which is the eigenvector associated with the negative eigenvalue of the symmetric part of M). In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarization identity. That is no longer true in the real case.

Applications

Heat conductivity matrix

Fourier's law of heat conduction, giving heat flux in terms of the temperature gradient is written for anisotropic media as in which \ K\ is the symmetric thermal conductivity matrix. The negative is inserted in Fourier's law to reflect the expectation that heat will always flow from hot to cold. In other words, since the temperature gradient always points from cold to hot, the heat flux is expected to have a negative inner product with so that Substituting Fourier's law then gives this expectation as implying that the conductivity matrix should be positive definite.

Sources