Variance (Welford’s method)

Algorithms for calculating variance play a major role in computational statistics. A key difficulty in the design of good algorithms for this problem is that formulas for the variance may involve sums of squares, which can lead to numerical instability as well as to arithmetic overflow when dealing with large values.

It is often useful to be able to compute the variance in a single pass, inspecting each value  only once for example, when the data are being collected without enough storage to keep all the values, or when costs of memory access dominate those of computation. For such an online algorithm, a recurrence relation is required between quantities from which the required statistics can be calculated in a numerically stable fashion.

(source:wikipedia).

Welford’s method is a usable single-pass method for computing the variance. It can be derived by looking at the differences between the sums of squared differences for N and N-1 samples. It’s really surprising how simple the difference turns out to be.

In my opinion, the Welford’s method (in green in the picture) to compute variance is less noisy and stable than the standard deviation formula (in blue). I let you do your homework with this code.

Share this