XLSTAT: Kolmogorov-Smirnov Test in Excel

 

Background

The Kolmogorov-Smirnov test is a nonparametric test that compares two independent samples' distributions or checks if a sample follows a specific distribution. It measures the maximum difference between their cumulative distribution functions.

 

Dataset for Example

The attached data correspond to scores (0 – 30) measuring the quality of two brands of shoes (brand A and brand B). Scores were computed based on a survey addressed to customers using either brand. 15 customers have answered for brand A and 8 different clients for brand B.  

Using this dataset, we will perform two comparisons:

1. A nonparametric Kolmogorov-Smirnov test to compare the two samples' distributions without assuming any theoretical distribution.
2. A Kolmogorov-Smirnov test to compare one sample's distribution to a theoretical distribution.

 

Setting Up (Comparison 1)

Without assumptions, sample B appears more skewed toward low values than sample A. We will use the Kolmogorov-Smirnov test to compare their distributions.

• Open XLSTAT.

• Click on the Nonparametric tests menu and select the Comparison of two distributions (Kolmogorov-Smirnov, ...) feature:

• Once you have clicked the button, the dialog box appears.

In the General tab:

• In the Data format field, select One column per sample.

• Select the Brand A column for Sample 1.

• Select the Brand B column for Sample 2.

Note: The Kolmogorov-Smirnov test supports unbalanced samples, such as in this case where Sample B contains fewer scores than Sample A.

 

In the Options tab:

• Select a one-tailed alternative hypothesis if applicable.

• Enable the exact computation of the p-value.

In the Charts tab:

• Activate the Cumulative histograms option.

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results 

The results sheet contains the Kolmogorov-Smirnov statistic (0.475) that can be easily extracted (see further, the cumulative histograms chart). This statistic is associated to a p-value (0.190) indicating that the two distributions are not significantly different at alpha = 0.05.

 

The cumulative distributions chart presents the studied variable (survey scores) on the x axis. For a given point on the x axis, a brand’s cumulative relative frequency is the proportion of scores smaller than this point among the scores of the brand. Kolmogorov-Smirnov’s D test statistic is the highest deviation occurring between the two curves. 

It is calculated by the following formula:

where F_n and F are the distribution functions.

In our example, this deviation value falls inside the median region, but this may not necessarily be the case when using other data. The higher the D statistic, the lower the p-value and the more significant the difference between the two distributions.

 

Setting up (Comparison 2)

We compare the French scores of brand A to the US scores' theoretical normal distribution (mean 21.5, SD 2.3) using the Kolmogorov-Smirnov test. This assesses if the French scores significantly differ from the US distribution.

• Open XLSTAT.

• Click on the Nonparametric tests menu and select the Distribution fitting feature:

 

• Once you have clicked the button the dialog box appears.

In the General tab:

• Select the brand A data.

• In the Distribution field, select the normal distribution.

• Enter the following parameters: µ = 21.5 and sigma = 2.3. 

• In the Charts tab, activate the Cumulative histograms option.

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results 

In the results sheet, the histogram (on the left below) shows that the observed distribution of our data lays on low score values compared to the theoretical curve reflecting the US scores distribution (red line).

The Kolmogorov-Smirnov test is associated to a p-value of 0.000 suggesting that the null hypothesis should be rejected and that the observed distribution is significantly different from the theoretical one at alpha = 0.05.

 

Related Articles

What is a Statistical Test?

Statistical Test Guide