XLSTAT: Repeated Measures ANOVA w/ Unbalanced Data Using Mixed Models in Excel

 

Background

Repeated measures ANOVA is based on the same model as a classical ANOVA.  In our example, the equation of the model will be written as below:

We will have two fixed factors -  group and time AND an interaction factor - group * time.

In contrast to conventional variance analysis, eijk errors can be correlated.

We cannot assume that measurements on the same subject taken at different times are independent.

In mixed models, the model equation is given by:

 where:

• Y is the quantitative variable to be explained

• X collects the factors associated with the fixed effects

• β is a vector of coefficients associated with the fixed effects

• Z is a matrix gathering the random effects

• γ is a vector of coefficients associated with the random effects

• ε is a vector gathering the errors associated with each observation.

In order to reproduce a repeated measures anova with mixed models, one solution is to include the subject factor as a random variable (with a variance component covariance structure) and to declare an error covariance structure with the same properties.

For mixed models with complex covariance structures and/or unbalanced data sets (as in our example), the test statistics for the fixed effects follow unknown distributions and are no longer exact F (Fisher) tests.

Assuming that the test statistics approximately follow an F distribution, XLSTAT implements an approximation of residual errors (Satterthwaite -1946) by finding the appropriate linear combination of random error sources of the error term to test the significance of each fixed effect of the model in question.

 

Dataset for Example

The (attached) data correspond to the evaluation of a depression score measured at different times in 24 patients separated into two groups: a treatment group and a control group that did not undergo treatment.

We have the following 4 variables:

• Subject: This qualitative variable contains the patient's identifier.

• Group: This qualitative variable makes it possible to know the group to which the patient belongs (1: control / 2: treatment).

• Time: This qualitative variable indicates when the measurement of the score was carried out, it includes 5 modalities (0: before the beginning of the treatment, 1: 1 month after the beginning of the treatment, 3: 3 months after and 6: 6 months after).

• dv: The quantitative dependent variable represents a score to assess the state of depression of the patient.

The data are unbalanced - the numbers of at least one of the factors (of all the modalities) are not equal.

Example: subject 1 was NOT measured at time 6; subject 2 WAS measured at time 6.

 

Setting Up

• Open XLSTAT.

• Click on the Modeling data menu and select the Mixed Models feature:

• The Mixed Models dialog box appears.

In the General tab:

• Select the data from the Excel sheet.

• The dependent variable is the one to be explained. In this case, select the score (dv).

• Select the three qualitative explanatory variables: "subject", "time", and "group".

• The Variable Labels option is enabled since the first row contains the variable names.

In the Options tab:

• Select the covariance matrix structure for errors (e) and random effects (g). 

• Choose the variance components structure for the random effects (this is also the default structure for residuals).

• Apply the constraint a1 = 0, meaning that the control group is assumed to have the standard effect on the score. This constraint is theoretically important but does not affect the results or the quality of the analysis.

• Enable the interactions option to account for interactions in the model.

• Enable the Satterthwaite’s t-tests option to use the Satterthwaite approximation for the model’s error term.

• Once you click OK, a new window for selecting fixed effects and random effects is displayed:

In this window:

• Select "time", "group", and their interaction "time*group" as fixed effects.

• Select "subject" as a random effect.

• Once the OK button is clicked, the computation begins.

• The results are then displayed.

 

Interpreting the Results

Goodness of fit coefficients table

The first results displayed by XLSTAT are the goodness of fit coefficients.

Model parameters are obtained using the restricted maximum likelihood (REML) method.

All indexes are used to compare models with different covariance structures.

 

Covariance parameters table

The following table gives the parameters of the covariance matrices of the random effect.

The errors (variance components structures). Z-tests p-values (Pr> Z) given in the tables below indicate a good reliability of standard error estimations for the model random parameters. Note that the parameters standard errors are high (likely because of the small sample size).

 

The type III of fixed effect table

The type III of fixed effect table (below) allows us to check significance of fixed effects using Fisher's F-test. We see here that all the fixed effects are significant.

The elapsed time has the most important effect. The Satterthwaite approximation method was used to calculate the degrees of freedom of the denominator as well as the F statistic in order to calculate a more accurate value (Pr> F) and thereby reduce the Type I error on the tested hypothesis (equality of “Least square means” difference).

In unbalanced cases, the tests for the higher order terms are always the same (here the "group time" interaction effect).

The assumptions differ between types for the lower order terms ("time" & "group"). In our example, the typical hypotheses I and II become dependent on the number of observations (experimental units) at each factor-level combination, so that the assumptions for these types become difficult to interpret.

On the contrary, the Type III hypothesis test always tests the fixed effect considered by controlling the effect of other factors (via orthogonal decomposition) which gives it good properties for studying unbalanced datasets. We can conclude that the treatment (group variable) and the time elapsed have effects on the level of depression of the patients.

 

Model parameters table

The following table summarizes the model parameters with their standard deviation and confidence interval.

Parameters interpretations are similar as in the case of a classical analysis of variance. Student's t-tests on non-nullity of regression coefficients do not exactly follow a Student's law. However, results presented in the table approximate a Student t distribution under the null hypothesis based on the Satterthwaite approximation to the effective degrees of freedom of error term.

When we look at the model parameters, we can see that time 1, 3 and 6 have a negative impact on the depression score. Patients are less depressed as time passes. Being in the treatment group has also a negative impact on the depression score.

Thus, we were able to perform a repeated measures ANOVA with mixed models despite the fact that our dataset does not meet the required conditions (strictly identical number of repetitions per subject).

In addition, Satterthwaite's approximation to model error term ensures even more reliable results for the different computed statistical tests. (Type I, II, III and t-tests).