Analyzing the relationship between power of t-test and effect size for different sample sizes

This fun and simple simulation helps us to understand the relationship between power of a t-test and effect size for small sample sizes.

Power of a binary hypothesis test is the probability of rejecting the null hypothesis when the alternative hypothesis is true.[1] Mathematically speaking, 

Power = 1 – P(type II error)

Hence, as the power of a test increases, the probability of making a type II error decreases.

Effect size is a measure to quantify the difference between two groups.[2]

In my R-code, I run 10,000 simulations per effect size for different sample sizes (n = 10, 20, 30).

The plot looks like this:

What do we learn:

  1. For a given effect size (especially for effect sizes less than 1.5), bigger samples have greater power than the smaller samples.
  2. For the given sample size, power increases with effect size.

The R code is posted hereunder:

[1] Wikipedia

[2] Coe R, University of Durham – It’s the Effect Size, Stupid

%d bloggers like this: