XLSTAT: Nonparametric regression (kernel & Lowess) in Excel

 

Background

Kernel regression belongs to the family of non parametric regression methods. It is also sometimes related to the smoothing methods. Kernel regression typically requires three phases:

• a fitting step during which one tries to find the best combine of model type, kernel function, and bandwidth, using a test sample.
• a validation phase that allows to validate the model on new observations for which the prediction is known;
• an application phase, where the model is applied to a new set of data for which the prediction is unknown.

Note: nonparametric regression includes a validation phase as a given observation is never used to build the model that is used to generate the corresponding prediction. However, one can still isolate a sub-sample that is only dedicated to the validation phase, to check the model robustness.

On the opposite to the classical linear regression, the goal is not to find a unique model that describes/explains/predict a phenomenon, but to obtain an efficient predictive method. Nonparametric regression is a kind of a black box. It is numerically intensive, as for each observation a new model is computed (in Robust Lowess regression, up to three models are computed for each observation).

 

Dataset for Example

This simple example is mainly illustrative, but nonparametric regression is valuable for predicting complex phenomena like financial time series, air pollution, or quarterly sales. It's also used to smooth data series.

The dataset, from Lewis and Taylor (1967), includes 237 children described by Gender, Age (months), Height (inches), and Weight (pounds).

The study has two phases: a fitting phase using 217 individuals and a validation phase with 20 individuals (10 girls and 10 boys).

 

Setting Up

• Open XLSTAT.

• Click on the Modeling data menu and select the Nonparametric regression feature.

• Once you've clicked on the button, the nonparametric regression dialog box appears

In the General tab:

• The Dependent variable corresponds to the variable that needs to be explained (or the variable to model), which is here the "Weight".

• The explanatory variables are the "Height" and the "Age" (quantitative data) and the sex (qualitative data).

• The selection has been done by columns as the data start on the first row. The Variable labels option is activated as the first row corresponds to the name of the variables.

• We have chosen to use the polynomial function with degree 1, using All the data (except the one that is being predicted), with a weighting based on the Gaussian kernel, and a bandwidth based on the standard deviation of the variables. The latter allows to avoid scaling effects during the computations.

Note: we are very close to the ANCOVA model, the difference being that we do not use the observation in the model that is used to do the corresponding prediction, and that the weights of the observations in the model depend on their distance to the observation to predict. 

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results

The goodness of fit coefficients allows to evaluate the performance of the model and to possibly compare several models. The R’² (the determination coefficient) gives an idea of the % of the variability of the Weight variable that is explained by the explanatory variables. The closer the R’² to 1, the better the model.

The table of predictions and residuals allows to visualize for each individual the input data, prediction, and the residual. The residuals vary in absolute values between 0.01 (individual 45) and 40 (individual 195). For the validation data that are displayed in the second part of the table, we notice that the residuals vary also a lot. For individuals 229 and 235 the prediction is excellent. It is a lot worse for the individual 224.

 

Related Articles

non parametric regression (Kernel / Lowess

linear regression