Frequency test

• The frequency test is a test of uniformity.
• Two different methods available, Kolmogorov-Smirnov test and the chi-square test. Both tests measure the agreement between the distribution of a sample of generated random numbers and the theoretical uniform distribution.
• Both tests are based on the null hypothesis of no significant difference between the sample distribution and the theoretical distribution.

The Kolmogorov-Smirnov test

This test compares the cdf of uniform distribution F(x) to the empirical cdf $S_N (X)$ of the sample of N observations.

• $F(x) = x ~~~ 0 \le x \le 1$
• $S_N (x) = \frac{{\rm number ~of} ~ R_1, R_2, ..., R_N \le x}{N}$
• As N becomes larger, $S_N(x)$ should be close to F(x)
• Kolmogorov-Smirnov test is based on the statistic
  
  $$D = {\rm max} \vert F(x) - S_N (x) \vert$$
  
  
  that is the absolute value of the differences.
• Here D is a random variable, its sampling distribution is tabulated in Table A.8.
• If the calcualted D value is greater than the ones listed in the Table, the hypothesis (no disagreement between the samples and the theoretical value) should be rejected; otherwise, we don't have enough information to reject it.

• Following steps are taken to perform the test.
  1. Rank the data from smallest to largest
     
     $$R_{(1)}, \le R_{(2)}, \le ... \le R_{(N)}$$
     
     

  2. Compute
     
     $$D^+ = {\rm max}_{1 \le i \le N} \left\{ \frac{i}{N} - R_{(i)} \right\}$$
     
     
     
     
     $$D^- = {\rm max}_{1 \le i \le N} \left\{ R_{(i)} - \frac{i-1}{N} \right\}$$
     
     

  3. Compute $D = {\rm max} (D^+, D^-)$
  4. Determine the critical value, $D_\alpha$, from Table A.8 for the specified significance level $\alpha$ and the given sample size N.
  5. If the sample statistic D is greater than the critical value $D_\alpha$, the null hypothsis that the sample data is from a uniform distribution is rejected; if $D \le D_\alpha$, then there is no evidence to reject it.
• Example 8.6 on page 300.

Chi-Square test

The chi-square test looks at the issue from the same angle but uses different method. Instead of measure the difference of each point between the samples and the true distribution, chi-square checks the ``deviation'' from the ``expected'' value.

$$x_0^2 = \sum_{i=1}^n \frac{(O_i - E_i)^2} {E_i}$$

where n is the number of classes (e.g. intervals), $O_i$ is the number of samples obseved in the interval, $E_i$ is expected number of samples in the interval. If the sample size is N, in a uniform distribution,

$$E_i = \frac{N} {n}$$

See Example 8.7 on page 302.