Comparison between Z-test and t-test for normal data with small sample size – when variance is unknown

In the world of hypothesis testing for data which follows normal distribution, Z-test is commonly used when the sample size is large (>30) or when the population variance is known. But, if the sample size is not large and the population variance is unknown, t-test is preferred. 

We can build a simple simulation to compare the type I error rates for the Z – test and t – test, and check whether the t – test performs better when the sample size is small and the variance is unknown. Just to recall, a type I error is rejecting a true null hypothesis. 

The steps for this simulation are as follows:

  1. Generate 25 observations from the normal distribution with mean 160 and standard deviation 20.
  2. Select different values for level of significance (the probability of type I error).
  3. Calculate the test statistic and the corresponding p- value (probability value is the probability of obtaining test results at least as extreme as the results actually observed during the test, assuming that the null hypothesis is correct)[1] for a two sided t-test and Z – test
  4. Calculate the type I error rates for different values of alpha.

The R code is as follows:

Hence, we can see that for normal data with small sample size and unknown variance, the t-test outperforms the Z-test with respect to the type I error rates.

[1] Wikipedia

%d bloggers like this: