arXiv:1911.01483 [stat.ML]AbstractReferencesReviewsResources
Statistical Inference for Model Parameters in Stochastic Gradient Descent via Batch Means
Published 2019-11-04Version 1
Statistical inference of true model parameters based on stochastic gradient descent (SGD) has started receiving attention in recent years. In this paper, we study a simple algorithm to construct asymptotically valid confidence regions for model parameters using the batch means method. The main idea is to cancel out the covariance matrix which is hard/costly to estimate. In the process of developing the algorithm, we establish process-level function central limit theorem for Polyak-Ruppert averaging based SGD estimators. We also extend the batch means method to accommodate more general batch size specifications.
Related articles: Most relevant | Search more
arXiv:1610.08637 [stat.ML] (Published 2016-10-27)
Statistical Inference for Model Parameters in Stochastic Gradient Descent
arXiv:1710.06382 [stat.ML] (Published 2017-10-17)
Convergence diagnostics for stochastic gradient descent with constant step size
arXiv:1712.07519 [stat.ML] (Published 2017-12-20)
Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients