Bayesian Shrinkage with Super Heavy-Tailed Priors

Abstract

Sparsity is often a desired structure for parameters in high-dimensional statistical problems. Within a Bayesian framework, sparsity is usually induced by spike-and-slab priors or global-local shrinkage priors. The latter choice is often expressed as a scale mixture of normal distributions with polynomial-tailed priors on the scale parameters. In general, heavier-tailed distributions are preferred in estimating sparse parameters. This dissertation focuses on investigating a new class of sparsity-inducing priors called super heavy-tailed (SH) priors, which have the heaviest possible tail among all proper priors. We explore SH priors in normal mean estimation, and demonstrate that this class of pri ors achieves sharp minimax posterior contraction rates and effective empirical performance through simulations and real data examples. Furthermore, we propose a spatially dependent global-local shrinkage prior for prediction and region selection in generalized linear models. It is motivated by the hurricane prediction problem, whose goal is to predict the number of hurricanes based on spatially dependent covariates and select the regions with significant contributions. The proposed prior combines the Conditional Autoregressive (CAR) prior and SH prior. The CAR component introduces spatial dependence in the coefficients of spatially dependent covariates and the SH prior leads to appropriate global-local shrinkage effects for selection. A Metropolis-within-Gibbs sampler is designed for computation. Our extensive simulation studies demonstrate that our proposed method has superior performance when the signals are weak and adjacent and the spatial dependence in covariates is strong. When applied to the North Atlantic hurricane prediction, the proposed method outperforms traditional regression-based methods and rivals the benchmark “oracle” model.