Variable selection: empirical Bayes vs. fully Bayes

For the problem of variable selection for the normal linear model, fixed penalty selection criteria such as AIC, Cp, BIC and RIC correspond to the posterior modes of a hierarchical Bayes model for various fixed hyperparameter settings. Adaptive selection criteria obtained by empirical Bayes estimation of the hyperparameters have been shown by George and Foster (2000) to improve on these fixed selection criteria. In this research, we study the potential of alternative fully Bayes methods, which instead margin out the hyperparameters with respect to prior distributions. Several structured prior formulations are considered, and a variety of fully Bayes selection and estimation methods are obtained. Extensive comparisons with their empirical Bayes counterparts suggest that the empirical Bayes methods perform extremely well in spite of their know inadmissibility.