Finding what matters when you have few cases and many candidate predictors
A small sample and high dimensionality: too many demographic predictors to model at once.
The problem
Some questions come with a small sample but a long list of candidate predictors. Here, that meant working out which demographic characteristics of a team relate to its outcomes. You can’t put them all in one model without overfitting, and naive variable selection can quietly drop the confounders you have to keep.
The approach
I built a two-stage method. Stage one selects variables with an elastic net while protecting known confounders through the Frisch–Waugh–Lovell theorem: the confounders are partialled out and always retained, so selection acts only on the remaining candidates. Stage two fits a Bayesian Ridge regression with resampling, giving regularised, stable estimates and honest uncertainty from limited data. Between them they can sift a lot of candidate predictors and still give a defensible answer about which factors matter.