When Multiple Variables Interact Simultaneously
Beyond single relationships — testing group differences, latent constructs, and complex causal structures.
Set 04 quantified relationships between pairs of variables. Set 05 goes further: multivariate analysis handles situations where multiple dependent variables, multiple groups, or complex theoretical models must be tested simultaneously. This is the set where IBM AMOS and SPSS become the primary tools — because the questions require software capable of handling structural complexity.
for SEM/CFA
technique
One-Way ANOVA
Tests whether the mean of a continuous variable differs significantly across three or more groups. Extension of the t-test to multiple groups.
Two-Way ANOVA
Tests the effect of two categorical factors simultaneously — and whether they interact. Interaction = the effect of one factor depends on the level of the other.
MANOVA
Multivariate ANOVA — tests group differences across multiple dependent variables simultaneously. Protects against inflated Type I error from running multiple ANOVAs.
ANCOVA
ANOVA with a covariate — statistically controls for a continuous variable when testing group differences. Makes groups more comparable by removing a confound.
Exploratory Factor Analysis (EFA)
Discovers latent factors underlying a set of observed variables. Used to reduce dimensionality and identify construct structure — without a prior hypothesis.
Confirmatory Factor Analysis (CFA)
Tests whether a pre-specified factor structure fits the data. Requires a hypothesis about which variables load on which factors. Typically run in IBM AMOS.
Structural Equation Modelling (SEM)
The most powerful technique in this set. Tests complex causal models with latent variables, mediation, moderation, and multiple relationships simultaneously.
Discriminant Analysis
Predicts group membership from multiple continuous variables. Also describes which combination of variables best separates the groups.
Canonical Correlation
Examines the relationship between two sets of variables simultaneously — e.g. a set of predictors and a set of outcomes.
Confirmatory Factor Analysis and Structural Equation Modelling require specialised software. IBM AMOS (part of the SPSS family) provides a visual drag-and-drop interface for building and testing these models. It is the industry standard for CFA and SEM in business research. Tabs 07 and 08 cover SPSS and AMOS in detail.
ANOVA & MANOVA
Are these groups actually different — or is the variation just noise?
One-Way ANOVA
ANOVA (Analysis of Variance) tests whether the means of a continuous variable are significantly different across three or more groups. It compares the variance between groups to the variance within groups. If between-group variance is much larger than within-group variance, the groups are genuinely different — not just randomly scattered around a common mean.
A large F means groups differ more than chance would predict → reject the null hypothesis that all means are equal
A significant ANOVA F-test tells you that at least one group mean is different from the others. It does not tell you which specific pair differs. For that, you need post-hoc tests (Tukey's HSD, Bonferroni, Scheffé) which compare all pairwise combinations while controlling for the inflated error rate of multiple comparisons.
A company trials three training methods (A, B, C) across 90 sales reps (30 per group). After 3 months, mean monthly sales are: Method A = $82K, Method B = $91K, Method C = $78K. One-Way ANOVA tests: are these differences real, or could they arise from random variation?
Result: F(2,87) = 6.43, p = 0.002. The groups are significantly different. Post-hoc Tukey test: Method B vs Method C is significant (p = 0.001); Method A vs Method C is not (p = 0.18). Method B outperforms C significantly. The company should adopt Method B.
Two-Way ANOVA — Testing Two Factors and Their Interaction
Two-Way ANOVA extends this to two categorical factors simultaneously — and tests whether they interact. An interaction effect means the effect of Factor A depends on the level of Factor B. This is often the most interesting finding.
The same company adds a second factor: experience level (Junior vs Senior). Results reveal an interaction: Method B is significantly better than A — but only for Junior reps. For Senior reps, Method A and B perform equally. Recommendation changes completely with the interaction: implement Method B for junior reps, either method for seniors. A simple One-Way ANOVA would have missed this entirely.
MANOVA — Multiple Dependent Variables
MANOVA (Multivariate ANOVA) tests group differences across multiple dependent variables simultaneously. It is used when the outcome has more than one measurable dimension — for example, training effectiveness might be measured by both revenue AND customer satisfaction AND retention.
| Situation | Use | Why |
|---|---|---|
| 3 groups, 1 outcome variable | One-Way ANOVA | Standard group comparison |
| 3 groups, 2 factors, 1 outcome | Two-Way ANOVA | Test interaction between factors |
| 3 groups, multiple outcome variables | MANOVA | Simultaneous testing protects Type I error; outcomes may be correlated |
| 3 groups, 1 outcome, 1 control variable | ANCOVA | Remove the influence of the covariate before testing group differences |
Running 3 separate ANOVAs on 3 outcome variables sets the Type I error rate at 1 − (0.95³) = 14% — not 5%. You would expect one false positive for every 7 analyses even when there is no real effect. MANOVA tests all outcomes simultaneously, maintaining the 5% threshold. Follow up a significant MANOVA with individual ANOVAs to identify which outcomes drove the multivariate result.
ANCOVA — Removing Confounds Before Testing Groups
ANCOVA makes group comparisons fairer by statistically controlling for a variable that differs across groups.
ANCOVA (Analysis of Covariance) combines ANOVA's group comparison with regression's ability to control for a continuous variable. It asks: "After accounting for the covariate, do the groups still differ?"
A company compares two teams' performance after a new initiative. Team A improved from a base of $65K to $78K. Team B improved from $75K to $85K. A simple ANOVA suggests Team B outperforms — but Team B started higher.
ANCOVA uses prior performance as a covariate and adjusts both group means to what they would be if both teams had started at the same baseline. After adjustment: Team A adjusted mean = $81K, Team B adjusted mean = $82K. The difference is no longer significant. The original ANOVA conclusion was misleading — the groups were not comparable to begin with.
Post-Hoc Tests — Finding Which Groups Differ
When ANOVA is significant, post-hoc tests identify which specific group pairs are different while controlling the overall error rate.
| Post-hoc test | Best for | Strictness |
|---|---|---|
| Tukey's HSD | All pairwise comparisons when group sizes are equal. The most commonly used. | Moderate — good balance of power and error control |
| Bonferroni | A small number of planned comparisons. Divides alpha by the number of comparisons. | Conservative — reduces false positives but increases false negatives |
| Scheffé | All possible comparisons including complex contrasts. Most flexible. | Most conservative — appropriate when many comparisons are explored |
| Games-Howell | When group variances are unequal (violating ANOVA's homogeneity assumption). | Moderate — robust to heteroscedasticity |
Before trusting ANOVA results, verify three assumptions: 1. Normality — each group's residuals should be approximately normally distributed (Shapiro-Wilk test; less critical with n > 30 per group). 2. Homogeneity of variance — groups should have similar variances (Levene's test in SPSS). If violated, use Welch's ANOVA or Games-Howell post-hoc. 3. Independence — observations must be independent. If violated (e.g. repeated measures), use Repeated-Measures ANOVA instead.
Exploratory Factor Analysis — Finding Hidden Structure
When you have many variables and want to discover how many underlying dimensions they represent.
Imagine you survey 200 employees with 20 questions about their workplace experience. The 20 questions probably don't each measure something completely unique. Some will correlate strongly with each other because they are measuring the same underlying thing — a latent factor that you cannot observe directly.
EFA uncovers these latent factors. It reduces 20 questions into, say, 4 underlying dimensions: "management quality," "workload fairness," "growth opportunities," and "work environment." Each factor is a hidden construct that explains why a cluster of questions moves together.
Key EFA concepts
| Concept | Plain explanation | What to look for |
|---|---|---|
| Factor Loading | The correlation between a variable and the factor. How strongly does this question represent this underlying dimension? | |loading| > 0.40 is typically considered meaningful. > 0.60 is strong. |
| Eigenvalue | How much variance a factor explains across all variables. An eigenvalue of 2.0 means the factor accounts for as much variance as two original variables combined. | Kaiser criterion: retain factors with eigenvalue > 1.0 |
| Communality | How much of a variable's variance is explained by all retained factors combined. | Low communality (< 0.30) means the variable is poorly explained — consider removing it. |
| Rotation | Mathematical transformation that makes factor loadings easier to interpret. Varimax (orthogonal) makes factors uncorrelated. Oblimin (oblique) allows factors to correlate. | Use Varimax when factors are theoretically independent. Use Oblimin when factors probably correlate (most business constructs do). |
| Scree Plot | A graph of eigenvalues. The "elbow" point suggests how many factors to retain. | Retain factors before the elbow — where the line bends from steep to flat. |
A retailer runs a 12-question customer satisfaction survey. EFA with Varimax rotation reveals 3 factors:
Factor 1 (eigenvalue 3.8, 32% variance): loads on Questions 1,2,3,4 — all about product quality → label: "Product Quality"
Factor 2 (eigenvalue 2.4, 20% variance): loads on Questions 5,6,7 — all about staff interaction → label: "Service Experience"
Factor 3 (eigenvalue 1.6, 13% variance): loads on Questions 8,9,10 — all about store environment → label: "Shopping Environment"
65% of total survey variance is explained by just 3 factors. The company now has 3 actionable dimensions instead of 12 separate metrics to monitor.
CFA & Structural Equation Modelling
Testing a theoretical model: do these constructs exist as specified — and do they relate to each other as hypothesised?
Confirmatory Factor Analysis (CFA)
While EFA discovers factor structure, CFA tests whether a theoretically specified structure fits the data. You state in advance: "I believe these 4 questions measure Factor 1, and these 4 questions measure Factor 2." CFA tests whether this specification is consistent with the data.
CFA is a prerequisite for SEM. Before modelling relationships between constructs, you must first confirm that your constructs are measured properly — that the indicators actually reflect the latent variable they are supposed to represent.
Reliability (Composite Reliability, CR): Are the indicators consistently measuring the same thing? CR > 0.70 is acceptable. Convergent Validity (AVE): Do indicators of the same construct correlate more with each other than with indicators of other constructs? AVE > 0.50. Discriminant Validity: Are the constructs truly distinct from each other? AVE for each construct should exceed the squared correlation between any two constructs.
Structural Equation Modelling (SEM)
SEM combines CFA (the measurement model) with path analysis (the structural model). It tests a complete theoretical framework — simultaneously estimating how latent constructs are measured AND how they relate to each other.
A full SEM model: two latent constructs (Service Quality and Customer Satisfaction), each measured by three indicators, with a structural path testing whether Service Quality predicts Customer Satisfaction (β = .68, p < .001).
SEM Model Fit Indices — What Good Looks Like
| index | What it measures | Acceptable threshold | Good threshold |
|---|---|---|---|
| CFI Comparative Fit index | How much better the model fits vs a null model (no relationships) | > 0.90 | > 0.95 |
| RMSEA Root Mean Sq Error of Approximation | Average discrepancy between model and data per degree of freedom | < 0.08 | < 0.06 |
| SRMR Standardised Root Mean Residual | Average standardised residual between observed and model-implied correlations | < 0.10 | < 0.08 |
| χ² / df Chi-square to df ratio | Overall model discrepancy — but sensitive to sample size | < 5.0 | < 3.0 |
| TLI Tucker-Lewis index | Similar to CFI but penalises model complexity | > 0.90 | > 0.95 |
Discriminant Analysis & Canonical Correlation
Two multivariate techniques for understanding group separation and inter-set relationships.
Discriminant Analysis
Discriminant Analysis (DA) identifies which combination of continuous variables best separates known groups. It simultaneously predicts group membership AND describes the structure of group differences. It is the multivariate complement to logistic regression, but focuses on both classification and description.
| Question type | Technique | Key output |
|---|---|---|
| Which group is this new observation most likely to belong to? | Discriminant Analysis or Logistic Regression | Classification into groups; posterior probabilities |
| Which variables best separate the groups? | Discriminant Analysis (preferred) | Discriminant function coefficients; structure matrix |
| How well does the model classify? Is it better than chance? | Both — check hit rate (% correctly classified) | Classification table; Wilks' Lambda (significance) |
A bank has three customer segments: "Premium," "Standard," and "At-Risk." DA uses 8 financial variables (income, savings balance, transaction frequency, credit utilisation, etc.) to: (1) build a discriminant function that separates the three segments, and (2) classify new customers into the appropriate segment automatically.
Result: Wilks' Λ = 0.34, p < 0.001 (groups are significantly separated). Hit rate = 84% (84% of customers are correctly classified). Structure matrix identifies income and credit utilisation as the strongest discriminating variables.
Canonical Correlation Analysis
Canonical Correlation examines the relationship between two sets of variables simultaneously. Unlike standard correlation (one X, one Y) or multiple regression (many X, one Y), canonical correlation handles many X with many Y — finding the linear combinations of each set that maximise the correlation between them.
An HR team has two variable sets: Input set (training hours, satisfaction score, tenure, work-life balance rating) and Output set (productivity score, absenteeism rate, promotion likelihood). Canonical correlation finds the combination of input variables that best predicts the combination of output variables — revealing the overall pattern linking employee experience to business performance.
Use EFA/CFA/SEM when you are working with survey or psychometric data and want to test theoretical constructs and their relationships. Use ANOVA/MANOVA when you have defined groups and want to test whether they differ. Use Discriminant Analysis when you want to classify new cases AND understand which variables drive the separation. Use Canonical Correlation when you have two defined sets of variables and want to understand the overall relationship between the sets.
IBM SPSS Statistics
ANOVA, MANOVA, ANCOVA, EFA, and Discriminant Analysis — all through SPSS menus.
IBM AMOS
The industry standard for Confirmatory Factor Analysis and Structural Equation Modelling. Visual, GUI-based, no code required.
AMOS (Analysis of Moment Structures) is part of the IBM SPSS family. It provides a visual canvas for drawing path diagrams — rectangles for observed variables, ovals for latent variables, single-headed arrows for directional effects, double-headed arrows for correlations. AMOS estimates all parameters simultaneously using Maximum Likelihood, then provides fit statistics and modification indices to improve the model.
Python — statsmodels, factor_analyzer & semopy
ANOVA, EFA, and SEM — fully implemented in Python with open-source libraries.
Three Simulators
Build intuition for ANOVA, factor analysis, and SEM fit indices through live interaction.
Simulator 1 · ANOVA Group Difference Explorer
Three training groups are shown. Adjust the group means and within-group spread. Watch how the F-statistic and p-value respond — and see when group differences become statistically significant.
Simulator 2 · Factor Loading Interpreter
EFA output is presented with factor loadings for 9 survey items across 2 factors. Identify which items load on each factor and what the factors likely represent.
Simulator 3 · SEM Fit index Evaluator
Four model fit outputs are shown. Apply the criteria (CFI, RMSEA, SRMR, χ²/df) to decide whether each model has acceptable fit, good fit, or poor fit — and what to do next.
Common Mistakes & Knowledge Check
Six mistakes that undermine multivariate analysis — followed by five assessment questions.
Running multiple ANOVAs instead of MANOVA
Five separate ANOVAs on five outcome variables sets the family-wise error rate at 23% — not 5%. When outcomes are correlated (as they usually are), MANOVA is both statistically correct and more powerful.
Using EFA results as if they were CFA results
EFA discovers structure from data. CFA tests a pre-specified theory. Applying EFA and then claiming you have "confirmed" the factor structure is circular — you discovered and confirmed with the same data. EFA and CFA should use different samples or at minimum be reported as distinct stages.
Ignoring ANOVA assumptions
ANOVA assumes normality and equal variances. With very small or unequal group sizes, these violations produce unreliable F-tests. Always run Levene's test first. When variances are unequal, use Welch's ANOVA and Games-Howell post-hoc.
Accepting a SEM model based on a single fit index
No single fit index tells the whole story. CFI can look good while RMSEA is poor; χ² is inflated by large samples. Always report at least CFI, RMSEA, and SRMR together. A model that passes all three is genuinely well-fitting.
Over-applying Modification Indices in AMOS
Modification indices suggest paths that would improve fit — but they are entirely data-driven. Adding paths just because they improve fit, without theoretical justification, produces an overfitted model that will not replicate on a new sample. Every added path must have a theoretical reason.
Treating factor scores as if they have the same precision as raw data
Factor scores are estimates — they carry measurement error. When using factor scores as variables in subsequent analyses (regression, ANOVA), this error is not accounted for. Full SEM, which models latent variables directly, handles this correctly.