Fri 13 May 2016

Order Statistics

Written by Hongjinn Park in Articles

Order Statistics is very powerful and practical. It's not just about the min and max or range. The following joint density function answers all probability statements about the order statistics!

Note that the following in here is only for iid RVs.

$$f_{X_{(1)},...,X_{(n)}} = n! f(x_1)f(x_2) \cdots f(x_n) \quad x_1 < x_2 < \cdots < x_n$$

Let's say there are $2n+1$ RVs then what is the probability that the \textbf{median} which is the $n+1$ order statistic is between some range.

$$f_{X_{(j)}}(x) = \frac{n!}{(j-1)!(n-j)!} [F(x)]^{j-1}[1-F(x)]^{n-j} f(x)$$

Let's say you want to make probability statements on two order statistics at a time (ie the range).

$$f_{X_{(i)},X_{(j)}}(x_i,x_j) = \frac{n!}{(i-1)!(j-i-1)!(n-j)!} [F(x_i)]^{i-1}[F(x_j)-F(x_i)]^{j-i-1} [1-F(x_j)]^{n-j} f(x_i)f(x_j)$$

Specifically for the range then $i$ and $j$ above become $1$ and $n$ and look at $R = X_{(n)} - X_{(1)}$ and with the following joint density function you can answer all probability statements about the first and last order statistic including say $P(R \le a)$

$$f_{X_{(1)},X_{(n)}}(x_1,x_n) = \frac{n!}{(n-2)!} [F(x_n)-F(x_1)]^{n-2}f(x_1)f(x_n)$$

Special situation: Intervals created by uniformly randomly choosing break up points

The joint distribution of the $n+1$ intervals formed by $n$ points uniformly randomly chosen in an interval is invariant under permutations of the intervals.

So the distribution of the kth interval is the same as the first interval. For example, it follows immediately that the expectation of the k-th order statistic of $n$ points uniformly randomly chosen in the unit interval is $\frac{k}{n+1}$ and that the expectation of the position of the k-th red ball out of n red balls among m total balls in a line is $\frac{k}{n+1}(m+1)$.

Example 1

$U_1$ and $U_2$ be iid and $\sim Uniform(0,1)$ then $E[U_{(2)}] = \frac{2}{2+1} = \frac{2}{3}$

Example 2

$n=2$ red balls among $m=8$ balls total then the expected position of the second red ball is $\frac{2}{3}(8+1) = 6$



Articles

Personal notes I've written over the years.