XLSTAT: Calculate Sample Size and Power for a Logistic Regression in Excel

 

Background

When testing a hypothesis using a statistical test, there are several decisions to take:

• The null hypothesis H0 and the alternative hypothesis Ha.

• The statistical test to use.

• The type I error also known as alpha. It occurs when one rejects the null hypothesis when it is true. It is set a priori for each test and is 5%.

The type II error or beta is less studied but is of great importance. In fact, it represents the probability that one does not reject the null hypothesis when it is false. We cannot fix it upfront, but based on other parameters of the model we can try to minimize it. The power of a test is calculated as 1-beta and represents the probability that we reject the null hypothesis when it is false.

We therefore wish to maximize the power of the test. XLSTAT calculates the power (and beta) when other parameters are known. For a given power, it also allows to calculate the sample size that is necessary to reach that power.

The statistical power calculations are usually done before the experiment is conducted. The main application of power calculations is to estimate the number of observations necessary to properly conduct an experiment.

We want to study the relationship between cancer occurrence and a diet program. The baseline probability of occurrence of this cancer is 6%. It is assumed that the odds ratio is 2, and that 25% of people in the sample are following this particular diet program.

In this example, we will find out what is the right sample size to perform this study in order to obtain a power of 0.9.

 

Setting up

• Open XLSTAT.

• Click on the Life Sciences/ Power analysis menu and select the Logistic regression feature.

• Once the button has been clicked, the dialog box pops up.

In the General tab:

• Choose the Find sample size goal. 

• The alpha is 0.05. The desired power is 0.9.

• As explained earlier, take a reference probability of 0.06 and an odds ratio of 2.

In the Chart tab:

• Activate the Simulation plot option.

• Select the Power on the horizontal axis. The size of sample 1 will be represented on the vertical axis.

• Power varies between 0.8 and 0.95 by increments of 0.01.

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results

The first table gathers the parameters used as input.

The second table gathers the results of the calculation as well as an interpretation.

We see that 776 observations are needed per sample to obtain a power as close as possible to 0.9.

The following table gathers the calculations obtained for each value of the power between 0.8 and 0.95.

The simulation plot shows the evolution of the sample size as a function of the power. We see that for a power of 0.8, 579 observations are enough and that for a power of 0.95 we arrive at 1016 observations.

XLSTAT is therefore a powerful tool both for finding the sample size required for an analysis and for calculating the power of a test. If we are not sure of our sample parameters, it is also possible to restart the analysis with a different odds ratio or a percentage of N with X1 = 1 which can reassure us about the sample size to choose.