In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli, is the discrete probability distribution of a random variable which takes the value 1 with probability p and the value 0 with probability q = 1-p. Less formally, it can be thought of as a model for the set of possible outcomes of any single experiment that asks a yes–no question. Such questions lead to outcomes that are Boolean-valued: a single bit whose value is success/yes/true/one with probability p and failure/no/false/zero with probability q. It can be used to represent a (possibly biased) coin toss where 1 and 0 would represent "heads" and "tails", respectively, and p would be the probability of the coin landing on heads (or vice versa where 1 would represent tails and p would be the probability of tails). In particular, unfair coins would have p \neq 1/2. The Bernoulli distribution is a special case of the binomial distribution where a single trial is conducted (so n would be 1 for such a binomial distribution). It is also a special case of the two-point distribution, for which the possible outcomes need not be 0 and 1.

Properties

If X is a random variable with a Bernoulli distribution, then: The probability mass function f of this distribution, over possible outcomes k, is This can also be expressed as or as The Bernoulli distribution is a special case of the binomial distribution with n = 1. The kurtosis goes to infinity for high and low values of p, but for p=1/2 the two-point distributions including the Bernoulli distribution have a lower excess kurtosis, namely −2, than any other probability distribution. The Bernoulli distributions for form an exponential family. The maximum likelihood estimator of p based on a random sample is the sample mean.

Mean

The expected value of a Bernoulli random variable X is This is due to the fact that for a Bernoulli distributed random variable X with \Pr(X=1)=p and \Pr(X=0)=q we find

Variance

The variance of a Bernoulli distributed X is We first find From this follows With this result it is easy to prove that, for any Bernoulli distribution, its variance will have a value inside [0,1/4].

Skewness

The skewness is. When we take the standardized Bernoulli distributed random variable we find that this random variable attains with probability p and attains with probability q. Thus we get

Higher moments and cumulants

The raw moments are all equal due to the fact that 1^k=1 and 0^k=0. The central moment of order k is given by The first six central moments are The higher central moments can be expressed more compactly in terms of \mu_2 and \mu_3 The first six cumulants are

Entropy and Fisher's Information

Entropy

Entropy is a measure of uncertainty or randomness in a probability distribution. For a Bernoulli random variable X with success probability p and failure probability q = 1 - p, the entropy H(X) is defined as: The entropy is maximized when p = 0.5, indicating the highest level of uncertainty when both outcomes are equally likely. The entropy is zero when p = 0 or p = 1, where one outcome is certain.

Fisher's Information

Fisher information measures the amount of information that an observable random variable X carries about an unknown parameter p upon which the probability of X depends. For the Bernoulli distribution, the Fisher information with respect to the parameter p is given by: Proof: This represents the probability of observing X given the parameter p. It is maximized when p = 0.5, reflecting maximum uncertainty and thus maximum information about the parameter p.

Related distributions