XLSTAT: CATA Check-All-That-Apply Analysis in Excel

 

Background

CATA (Check-All-That-Apply) analysis is a method used in sensory evaluation and consumer research to gather information about product attributes. Participants are given a list of characteristics and are asked to check all those they find applicable.

 

Dataset for Example

The (attached) data was provided by Ares et al. (2014). They correspond to the assessment of 6 products (5 regular and 1 ideal) by 119 consumers over 15 attributes. Data is recorded in a binary format (0: attribute not checked; 1: attribute checked). Moreover, every product (except the ideal one) is rated overall (0-10) by each consumer.

Data is in a vertical format, which means that we have one row per combination of consumer and product.

We will use the data to characterize products tested by consumers.

 

Setting Up

• Open XLSTAT.

• Click on the Sensory/CATA data menu and select the CATA data analysis feature:

In the General tab:

• Make sure you select the Vertical data format.

• In the CATA data field, select the attribute table.

• Then select the Consumer, Sample and liking columns in the Assessors, Products and Preference data fields, respectively.

• Capture the ideal product identifier in the Ideal product field.

 Important: The dataset must be balanced (One assessor for each product).

In the Options (1) tab:

• Check CATA data validation and Independence of attributes.

• Select the Chi-square distance for the Correspondence Analysis.

• Click on the OK button to launch the computations.

• A dialog box appears to select and validate the axes to be displayed on the graphical representation of the correspondence analysis, just click the  Done button.

 

Interpreting the Results (Part 1)

The initial tables and graphs validate the CATA data. They identify assessors with unusual behavior, such as assessor 27 who checked only 7% of the time, while most checked between 20-35%. An attribute analysis also detects over- or under-used attributes.

An combined analysis shows the percentage of checks per assessor and attribute, assessing consensus. The Juicy attribute varies widely, with some assessors checking it over 80% and others less than 20%.

Cochran’s Q test evaluates whether different products significantly affect the presence of an attribute. A low p-value indicates significant differences. Pairwise comparisons using letters show which products differ: those sharing the same letter are not significantly different, while those without common letters are. Most attributes, except two related to smell, have significant p-values. For example, product 257 is the most checked for crispiness, but not significantly more than 548. Products 992 and 366 are the least crispy and do not differ significantly.

 

An attribute independence test assesses whether attributes are redundant for each product. For product 106, Juicy and Firm are found to be redundant.

The combined attribute table is used for correspondence analysis (CA). The significant p-value indicates likely real differences in sensory profiles between products. The eigenvalues and plot show the analysis is of good quality, explaining 92.17% of total inertia with the first two dimensions.

The analysis suggests an ideal product should be tasty, juicy, crispy, firm, sweet, and apple-flavored, and not too sour, bitter, astringent, or grainy.

Product 548 is closest to this ideal, while 106, 366, and 992 are less ideal due to bitterness, sourness, and astringency.

A correlation matrix shows strong relationships, notably a negative correlation between sweet and sour. Liking scores are weakly positively correlated with attributes like juiciness, tastiness, and apple flavor.

Principal Coordinates Analysis (PCoA) based on correlation coefficients visualizes relationships in a 2D map. The plot shows that the first two dimensions are enough for interpretation, with liking linked to juicy, tasty, and apple flavor attributes.

More information about principal coordinate analysis is available here.

 

Interpreting the Results (Part 2)

Penalty analysis uses liking data to identify must-have attributes by examining incongruence where an attribute is missing in the real product but present in the ideal. A summary table shows frequencies of P(No|Yes) and P(Yes|Yes), with a graph visualizing these frequencies and their percentages.

Mean drops in liking between the two situations are then presented for each attribute and their significances tested. For example, the firm attribute implies an increase of 1.5 Liking points between the tested products and the ideal product. This increase is significant at 0.05 (p < 0.0001).

Note: In the case where there is no ideal product, this analysis is substituted by an analysis of presence and absence of the attributes.

The mean impact chart shows the attributes with a significant mean impact. Mean increases are displayed in blue and are identified as “must-have”, mean decreases are displayed in red.

The mean drops vs. % chart identifies “must-have” attributes.

The Y-axis shows differences in product appreciation when consumers check both the actual and ideal products versus only the ideal, while the X-axis reflects the percentage of cases where only the ideal is checked.

Attributes with high values on both axes, like tasty, sweet, juicy, apple flavor, crispy, and firm, are considered essential.

A second analysis identifies “nice-to-have” attributes by examining when an attribute is missing in the ideal but present in the real product. This is only applicable when an ideal exists.

The mean impact chart shows significant attributes: blue indicates positive impact (“nice to have”) and red indicates negative impact (“must not have”). In this case, only the attribute Sour was analyzed.

The mean drops vs % chart also allows to clearly identify **“must not have” and “nice to have” attributes.

• The Y-axis corresponds to the differences in product appreciation when consumers did not check either the ideal product or the product (cell [0.0] in the "Attribute Analysis" table) and when they checked the product (cell [1.0]).

• The X-axis represents the percentage of inputs including a check for the real product without the ideal product being checked, which corresponds to a situation where the attribute describes the real products well but is relatively unchecked for the ideal product.

Therefore, attributes that are associated to low coordinate on the Y axis (astringent, bitter, grainy, tasteless, soft, mealy, sour) appear here again to be “must not have”. Attributes associated to high coordinates on the Y axis are “nice to have”.

The two previous analyses are finally summarized in one map. Here again, tasty, sweet, apple flavour, firm, crispy and juicy appear as “must have“; and sour appears as “must not have”.

For each attribute, a 2x2 table compares ideal and surveyed products, showing average preferences and record percentages.

If the attribute is checked in the ideal product and preference is higher when checked, it's a must-have.

If unchecked in the ideal and preference is higher when unchecked, it's a must-not-have.

If unchecked in the ideal but preference is higher when checked, it's nice-to-have. When preferences are similar, the attribute does not influence.

XLSTAT analyzes these tables following these rules and considers sample size thresholds for decision-making.

In the final summary table, we can see that 6 out of the 15 attributes are “must have”, 1 is a does not harm and 1 attribute is a must not have. The remaining attributes could not be related to any category.

The graph shows the difference between each product and the ideal for each attribute. Values near zero indicate similarity; negative values mean the attribute is underrepresented, positive values mean overrepresented. The confidence interval determines if the difference is significant.

For example, the difference for Product 106 versus the ideal is calculated as (6/119) – (92/119) = -0.7227.

 

Related Articles

CATATIS Method

CLUSCATA Method

Statistical Test Guide