XLSTAT: Calculate Sample Size and Power for a Comparison of Correlations in Excel

 

Background

When testing a hypothesis using a statistical test, there are several decisions to take:

• The null hypothesis H0 and the alternative hypothesis Ha.

• The statistical test to use.

• The type I error also known as alpha. It occurs when one rejects the null hypothesis when it is true. It is set a priori for each test and is 5%.

The Type II error (beta) is the probability of failing to reject a false null hypothesis.

The power (1-beta) is the chance of correctly rejecting it when false.

XLSTAT calculates power and beta when parameters are known, and can determine the necessary sample size for a desired power, typically before an experiment.

With this example, we will compare height-weight correlations for men and women, aiming for a power of 0.9.

Using Cohen's effect size (0.1 weak, 0.3 moderate, 0.5 strong), larger effects require smaller samples to achieve high power.

Setting Up

• Open XLSTAT.

• Click on the Life Sciences/ Power analysis menu and select the Compare correlations feature.

• Clicking the button opens the dialog box.
• In the General tab, select "Find the sample size" for the objective and "Correlations test for two samples" as the test.
• Set the alternative hypothesis to r1 - r2 ≠ 0, alpha to 0.05, power to 0.9, N1/N2 to 1, and effect size to 0.1, since sample parameters are unknown.

In the Chart tab:

• The simulation plot option is activated and the size of sample 1 will be represented on the vertical axis and the power on the horizontal axis.

• The power varies between 0.8 and 0.95 in intervals of 0.01.

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results

The first table shows the essential results including the sample size followed by an interpretation.

A total of 87 observations are needed for both samples to achieve a power close to 0.9. The table below shows calculations for powers ranging from 0.8 to 0.95.

The simulation plot shows the evolution of the sample size as a function of the power. We see that for a power of 0.8, just over 1573 observations per sample are enough and that for a power of 0.95 we arrive at 2602 observations.

For effect sizes of 0.3 and 0.5, the following results are obtained:

As the difference between correlations increases, the required sample size decreases; for a strong difference, 87 observations per sample suffice.

With more sample information, input parameters can be specified instead of effect size.