arXiv Analytics

Sign in

arXiv:2003.09671 [cs.LG]AbstractReferencesReviewsResources

On Information Plane Analyses of Neural Network Classifiers -- A Review

Bernhard C. Geiger

Published 2020-03-21Version 1

We review the current literature concerned with information plane analyses of neural network classifiers. While the underlying information bottleneck theory and the claim that information-theoretic compression is causally linked to generalization are plausible, empirical evidence was found to be both supporting and conflicting. We review this evidence together with a detailed analysis how the respective information quantities were estimated. Our analysis suggests that compression visualized in information planes is not information-theoretic, but is rather compatible with geometric compression of the activations.

Related articles: Most relevant | Search more
arXiv:1911.12780 [cs.LG] (Published 2019-11-28)
Detection and Mitigation of Rare Subclasses in Neural Network Classifiers
arXiv:2002.01256 [cs.LG] (Published 2020-02-04)
Minimax Defense against Gradient-based Adversarial Attacks
arXiv:1810.07481 [cs.LG] (Published 2018-10-17)
Provable Robustness of ReLU networks via Maximization of Linear Regions