Explore the techniques and methods used to analyze data sets with multiple variables simultaneously, including factor analysis, principal component analysis, and canonical correlation analysis.
Multivariate Analysis – Quick‑Reference Notes
| Term | Meaning |
|---|---|
| Variable | Measured attribute (continuous, categorical, ordinal). |
| Dimension | Number of variables (p). |
| Observation / Case | One row of data (n). |
| Covariance Matrix (Σ) | p × p matrix summarizing pairwise covariances; central to many methods. |
| Correlation Matrix (R) | Standardized covariances (values between –1 and 1). |
| Eigenvalues / Eigenvectors | Decompose Σ or R; eigenvalues indicate variance explained, eigenvectors give direction of new axes. |
| Factor Loadings | Correlations between observed variables and latent factors. |
| Score | Projected value of an observation on a derived component/factor. |
| Residual | Difference between observed and model‑predicted values. |
| Family | Goal | Typical Methods | Typical Output |
|---|---|---|---|
| Dimension‑reduction | Summarize many variables with few latent constructs | Principal Component Analysis (PCA), Factor Analysis (FA), Multidimensional Scaling (MDS) | Component/factor scores, loadings, scree plot |
| Classification / Discrimination | Assign observations to predefined groups | Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Logistic Regression (multinomial) | Discriminant functions, classification tables |
| Clustering | Group similar observations without prior labels | K‑means, Hierarchical clustering, Gaussian Mixture Models | Cluster assignments, dendrograms |
| Multivariate Regression | Model several dependent variables simultaneously | Multivariate Linear Regression (MLR), Partial Least Squares (PLS), Canonical Correlation Analysis (CCA) | Coefficient matrices, canonical variates |
| MANOVA / MANCOVA | Test mean differences across groups for multiple outcomes | MANOVA, MANCOVA (covariate adjustment) | Wilks’ λ, Pillai’s trace, Hotelling’s T² |
| Time‑Series / Panel Multivariate | Analyze multiple series over time | Vector Autoregression (VAR), Dynamic Factor Models | Impulse‑response functions, Granger causality tests |
| Survival / Event‑History (multivariate) | Model time‑to‑event with several predictors | Cox proportional hazards with multiple covariates, Competing‑risk models | Hazard ratios, baseline hazard |