Evaluating linear regression methods in four scenarios from Zou & Hastie (2005)

In this short analysis, we compare the prediction accuracy of several linear regression in the four simulation examples described in Zou & Hastie (2005). The methods compared are:

• ridge regression;

• the Lasso;

• the Elastic Net;

• Sum of Single Effects regression (SuSiE), described here;

• variational inference for Bayesian variable selection, or “varbvs”, described here; and

• “varbvsmix”, an elaboration of varbvs that replaces the single normal prior with a mixture-of-normals.