Interval Estimation

• Valid interval estimation typically requires a method of estimating the variance of the point estimator $\hat{\theta}$ or $\hat{\phi}$.

• Let $\sigma^2 (\hat{\theta}) = var (\hat{\theta})$ represent the true variance of a point estimator $\hat{\theta}$, and let $\hat{\sigma^2} (\hat{\theta})$ represent an estimator of $\sigma^2 (\hat{\theta})$ based on the data $Y_i, i = 1,2,...,n$.

• Suppose that
  
  $$E[\hat{\sigma^2} (\hat{\theta})] = B \sigma^2 (\hat{\theta})$$
  
  
  where B is called the bias in the variance estimator.

• It is desirable to have
  
  $$B = 1$$
  
  
  in which case $\hat{\sigma^2} (\hat{\theta})$ is said to be an unbiased estimator of variance, $\sigma^2 (\hat{\theta})$.

• If it is an unbiased estimator, the statistic
  
  $$t = \frac{\hat{\theta} - \theta} { \hat{\sigma} ( \hat{\theta} ) }$$
  
  
  is approximatly t distribution with some degree f of freedom.
• An approximate $100(1-\alpha)\%$ confidence interval for $\theta$ is given by
  
  $$\hat{\theta} - t_{\alpha/2,f} \hat{\sigma}(\hat{\theta}) \le ... ...a \le \hat{\theta} + t_{\alpha/2,f} \hat{\sigma}(\hat{\theta})$$
  
  

• This relation involves three parameters, estimator for mean, estimator for variance, and the degree of freedom. How to determine these values?
  • Estimator for mean is calculated as above as a point estimator
    
    $$\hat{\theta} = \frac{1}{n} \sum_{i=1}^n Y_i$$
    
    

  • Estimator for the variance and for the degree of freedom has to consider two separate cases
    1. If $Y_i$s are statistically independent observations then use
       
       $$S^2 = \sum_{i=1}^n \frac{(Y_i - \hat{\theta})^2} {n - 1}$$
       
       
       to calculate
       
       $$\hat{\sigma}^2 (\hat{\theta}) = \frac{S^2} {n}$$
       
       
       with the degree of freedom f = n - 1.

    2. If $Y_i$s are not statistically independent, then the above estimator for variance is biased. $Y_i$s is an autocorrelated sequence, sometimes called a time series.

       In this case,
       

       $$\sigma^2 (\hat{\theta}) = var(\hat{\theta}) = \frac{1}{n^2} \sum_{i=1}^n \sum_{j=1}^n cov(Y_i, Y_j)$$
       
       
       i.e. one needs to calculate co-variance for every possible pair of observations. Too expensive.

       If the simulation is long eough to have passed the transient phase, the output is approximately covariance stationary. That is $Y_{i+k}$ depends on $Y_i$ in the same way as $Y_k$ depends on $Y_1$

       For a covariance stationary time series $Y_i$s, define the lag k autocovariance by
       

       $$\gamma_k = cov(Y_1, Y_{1+k}) = cov(Y_i, Y_{i+k})$$
       
       

       For k = 0, $\gamma_0$ becomes the population variance
       

       $$\gamma_0 = cov(Y_{0+i}, Y_i) = var(Y_i)$$
       
       

       The lag k autocorrelation is the correlation between any two observations k apart.
       

       $$\rho_k = \frac{\gamma_k}{\gamma_0}$$
       
       
       and has the property
       
       $$-1 \le \rho_k \le 1, ~~ k = 1, 2, ...$$
       
       

       If a time series is covariance stationary, then the calculation of sample variance can be substantially simplified.
       

       $$\sigma^2 (\hat{\theta}) = \frac{\gamma_0}{n} \left[ 1 + 2 \sum_{k=1}^{n-1} \left( 1 - \frac{k}{n} \right) \rho_k \right]$$
       
       
       So all we need is to calculate covariance between one sample and every other samples, but not every sample with every other samples.

    Some discussions about why autocorrelation make it difficult to estimate $\sigma^2 (\hat{\theta})$ are skipped