Analyzing the relationship between power of t-test and effect size for different sample sizes

This fun and simple simulation helps us to understand the relationship between power of a t-test and effect size for small sample sizes.

Power of a binary hypothesis test is the probability of rejecting the null hypothesis when the alternative hypothesis is true.^{[1]} Mathematically speaking,

Power = 1 – P(type II error)

Hence, as the power of a test increases, the probability of making a type II error decreases.

Effect size is a measure to quantify the difference between two groups.^{[2]}

In my R-code, I run 10,000 simulations per effect size for different sample sizes (n = 10, 20, 30).

The plot looks like this:

What do we learn:

For a given effect size (especially for effect sizes less than 1.5), bigger samples have greater power than the smaller samples.

For the given sample size, power increases with effect size.