Abstract
<p>This article extends the Bayesian regularized variational procedure for partially confirmatory factor analysis to accommodate sparse residual dependence. Existing PCFA-VA assumes a diagonal residual covariance, which may be restrictive in practice. Using continuous spike-and-slab priors, the proposed method jointly selects factor loadings and residual dependence structures while allowing available expert knowledge to be incorporated. It spans a broad confirmatory-exploratory continuum, accommodating fully or partially specified loading structures with residual dependence selection and fully unspecified loading structures with partially specified residual dependence. Rather than relying on stochastic posterior sampling, we develop a variational expectation conditional maximization (VECM) algorithm for posterior approximation and estimation. By varying the scale of the spike densities along a regularization path, the algorithm supports dynamic model exploration and progressively removes negligible loadings and residual dependence relations. Simulations demonstrate the framework's flexibility under diagonal and non-diagonal residual structures. When residual dependence is absent, it retains PCFA-VA as a special case and yields lower estimation error. When residual dependence is present, it achieves structure recovery and estimation accuracy comparable to the MCMC-based PCFA method, at markedly lower computational cost. An empirical application further illustrates that the loading and residual dependence structures selected by VECM are substantively interpretable. }\keywords{factor analysis, residual dependence, spike-and-slab, variational inference</p>