XLSTAT: Calculate Sample Size and Power for an ANOVA in Excel

 

Background

XLSTAT offers features to apply analysis of variance (ANOVA), repeated measures analysis of variance and analysis of covariance (ANCOVA). For any of these methods, you may want to estimate the power or calculate the necessary number of observations before running the analysis.

When testing a hypothesis, there are multiple things to consider:

• The null hypothesis H0 and the alternative hypothesis Ha.

• The statistical test to use.

• The type I error also known as alpha. It occurs when one rejects the null hypothesis when it is true. It is set a priori for each test and is 5%.

The type II error or beta is less studied but is of great importance. In fact, it represents the probability that one does not reject the null hypothesis when it is false. We cannot fix it upfront, but based on other parameters of the model we can try to minimize it. The power of a test is calculated as 1-beta1−beta and represents the probability that we reject the null hypothesis when it is false.

We therefore wish to maximize the power of the test. XLSTAT calculates the power (and beta) when other parameters are known. For a given power, it also allows to calculate the sample size that is necessary to reach that power.

The statistical power calculations are usually done before the experiment is conducted. The main application of power calculations is to estimate the number of observations necessary to properly conduct an experiment.

 

Dataset for Example

In this example, three types of microwaves were tested to explain the percentage of edible popcorn after cooking.

Cooking differs depending on the  brand of microwave, the power, and the duration. We will seek to determine whether there is a significant influence of brand, potency, cooking time, and their interactions.

We are in the case of a 3-factor anova. The microwave brand has 3 levels, power and duration have two levels each. So we have 3 x 2 x 2 = 12 groups. In our case, we will seek to know the size of the sample necessary so that the bfactor is a power of 0.9.

As we do not know yet the parameters of our sample, we will use the concept of effect size.

Cohen (1988) introduced this concept which makes it possible to give an order of magnitude for the importance of the effect. We will test 3 sizes of effects:

• 0.1 for a weak effect
• 0.25 for a moderate effect
• 0.4 for a strong effect.

The expectation is that the larger the effect size, the smaller the required sample size will be.

 

Setting Up

• Open XLSTAT.

• Click on the Life Sciences/Power analysis menu and select the ANOVA/ANCOVA feature.

• Once the button is clicked, the dialog box pops up.

In the General tab:

• Choose the Find sample size goal.

• Select the ANOVA Factors and Interactions test.

• The alpha is 0.05. The desired power is 0.9.

• The number of groups, as explained earlier, is 12.

• The number of degrees of freedom of the numerator (Num DDL) is equal to the number of levels of the factor brand - 1, therefore 2.

In the Chart tab:

• Activate the  simulation plot option.

• Select the Power on the horizontal axis. The size of sample 1 will be represented on the vertical axis.

• Power varies between 0.8 and 0.95 by increments of 0.01.

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results

The first table gathers the parameters used as input.

The second table gathers the results of the calculation as well as an interpretation of the results.

We see that 83 observations are needed in our sample to obtain a power as close as possible to 0.9.

The following table gathers the calculations obtained for each value of the power between 0.8 and 0.95.

The simulation plot shows the evolution of the sample size as a function of the power. We see that for a power of 0.8, 64 observations per sample are enough and that for a power of 0.95 we arrive at 100 observations.

For effect sizes of 0.1 and 0.25, we obtain the following results:

The results show that smaller effect sizes require stricter input parameters and thus larger sample sizes.