XLSTAT: Lasso & Ridge Regression in Excel

 

Background

In high-dimensional regression, Ridge and Lasso improve coefficient estimation by adding a regularization term to the cost function.

Lasso (L1 norm) penalizes the sum of absolute coefficients, potentially setting some to zero.

Ridge (L2 norm) penalizes the sum of squared coefficients, shrinking them without nullifying.

Elastic Net combines both approaches to address their limitations. Regularization strength is controlled by the parameter Lambda.

These methods, also known as weight decay in machine learning, are useful when predictors outnumber observations. In XLSTAT-R, they are implemented via the glmnet function from the R package glmnet.

Dataset for Example

The attached data were simulated and correspond to 36 biological samples of diseased and healthy individuals belonging to three different genotypes. For each sample, the expression of 1561 genes is measured through RNA quantification. More details  differential expression. We will use Lasso regression followed by Ridge regression and compare the outcomes.

Logistic regression cannot be performed here because the number of variables (genes) is far higher than the number of observations (samples).

 

Setting up (Lasso Regression)

• Open XLSTAT.

• Click on the XLSTAT-Rb glmnet menu and select the Ridge, Elastic net and Lasso GLM (glmnet) feature.

In the General tab:

• Select the status variable (healthy/diseased) as the Y variable.

• Choose Binomial as the response family.

• Select all expression data under Explanatory quantitative variables.

• Set alpha to 1 to apply Lasso penalization.

In the Options tab:

• Activate standardize to apply data standardization.

• Set the number of Lambda values to 100 for k-fold Cross-Validation to find the optimal lambda.

 

• Click on the OK button to launch the computations.

• The results are displayed on a new Excel sheet.

 

Interpreting the Results (Lasso Regression)

The Lambda value minimizing deviance is shown first, followed by the computed coefficients for all genes.

Lasso penalization sets most coefficients to zero, highlighting key features like T106.02. Since this is a binary regression, coefficients represent log-odds. A positive coefficient means T106.02 is more expressed in diseased patients (labeled 1).

The Coefficients plot shows how coefficients shrink as Lambda (log scale) increases, with many reaching zero at high Lambda values. Numbers atop the chart indicate how many coefficients remain non-zero at each Log Lambda.

  

The Cross Validation plot displays deviance (error) versus lambda values, with the optimal lambda at the minimum deviance. Lambda is shown on the natural (Neperian) logarithm scale.

Let’s compare these outputs to the outputs of a Ridge regression.

 

Setting Up (Ridge Regression)

• Go back to the glmnet dialog box and set alpha to 0. Click OK.

 

Interpreting the Results (Ridge Regression) 

Unlike Lasso, Ridge regression assigns non-zero coefficients to all features but shrinks them toward zero. The shrinkage increases with the regularization parameter Lambda, as shown in the coefficients plot.