In philosophy, a razor is a device which allows one to eliminate unlikely explanations for a phenomenon.
Useful razors include:
- Occam’s razor: when faced with competing hypotheses, select the one that makes the fewest assumptions
- Popper’s falsifiability principle: a theory can be scientific only if it is falsifiable
- Hanlon’s razor: assume incompetence over malice
- Hitchens’ razor: the burden of proof or onus in a debate lies with the claim-maker, and if he or she does not meet it, the opponent does not need to argue against the unfounded claim
- And the ridiculously named… Newton’s flaming laser sword: If something cannot be settled by experiment then it is not worthy of debate
Razors are rules of thumb, often cited in data analysis to minimize overfitting, which can occur when overly complex models are affected by statistical noise instead of the underlying relationship. A simpler model may capture the underlying structure better and may thus have better predictive performance.
Most real world problems have multicollinear effects. However these effects can be decomposed into the 2 or 3 that have the biggest impact. If you can address those then the problem is solved