arXiv Analytics

Sign in

arXiv:2206.11241 [cs.LG]AbstractReferencesReviewsResources

Concentration inequalities and optimal number of layers for stochastic deep neural networks

Michele Caprio, Sayan Mukherjee

Published 2022-06-22Version 1

We state concentration and martingale inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.

Related articles: Most relevant | Search more
arXiv:1905.09803 [cs.LG] (Published 2019-05-23)
How degenerate is the parametrization of neural networks with the ReLU activation function?
arXiv:2008.01173 [cs.LG] (Published 2020-07-31)
An Investigation on Deep Learning with Beta Stabilizer
arXiv:2001.11595 [cs.LG] (Published 2020-01-30)
Concentration Inequalities for Multinoulli Random Variables