XLSTAT: Random Forest Regression

Dataset for Example

The dataset used is extracted from the Machine Learning competition entitled "Titanic: Machine Learning from Disaster" on Kaggle the famous data science platform.
The Titanic dataset can be accessed at this address. It refers to the ocean liner the Titanic that sunk in 1912. During this tragedy, more than 1,500 of the 2,224 passengers lost their lives due to an insufficient number of lifeboats.

The dataset is made up of a list of 1309 passengers and their characteristics:

• survived: Survival (0 = No; 1 = Yes)

• pclass: Passenger Class (1 = 1st; 2 = 2nd; 3 = 3rd)

• name: Name

• sex: Gender (male; female)

• age: Age

• sibsp: Number of Siblings/Spouses Aboard

• parch: Number of Parents/Children Aboard

• fare: Passenger Fare

• cabin: Cabin

• embarked: Port of Embarkation (C = Cherbourg; Q = Queenstown; S = Southampton)

 

Setting Up

• Open XLSTAT.

• Click on the Machine Learning menu and select the Classification and regression Random Forest feature.

•  The Classification and regression Random Forest dialog box appears.

In the General tab:

• In the Response type field, select the type of variable you want to predict (here Quantitative).

• In the Response variable, select the Passenger’s fare information.

• Select the Variable labels to take into account the variables names provided in the first row if the data set.

• Select the passengers’ names as the Observation labels.

 

In the Options tab, enable convergence and set it to 100 so XLSTAT checks every 100 trees for algorithm convergence and OOB error stabilization.

  

• Click OK to start the computations. Results appear on a new Excel sheet.

 

Interpreting the Results

The first result shown is the OOB error, representing the average prediction error on each out-of-bag sample in the training set.

The following table displays the response and the predicted value related to each observation of the learning set (prediction made using only the trees in which they are OOB).

The following graph displays the evolution of the OOB error according to the number of trees.

The OOB error rapidly decreases and stabilizes, leading the algorithm to stop at 200 trees since no further improvement occurs. The next table shows each variable’s normalized importance (standard deviation normalized).

A graphical representation of the above table is given below:

The most important variable is the passenger’s class, suggesting a link between fare and class. The table also shows the standard deviation of each variable’s importance measure.