arXiv Analytics

Sign in

arXiv:1910.11585 [cs.LG]AbstractReferencesReviewsResources

Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?

Ali Shafahi, Amin Ghiasi, Furong Huang, Tom Goldstein

Published 2019-10-25Version 1

Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study, we explore the mechanisms by which adversarial training improves classifier robustness, and show that these mechanisms can be effectively mimicked using simple regularization methods, including label smoothing and logit squeezing. Remarkably, using these simple regularization methods in combination with Gaussian noise injection, we are able to achieve strong adversarial robustness -- often exceeding that of adversarial training -- using no adversarial examples.

Related articles: Most relevant | Search more
arXiv:1611.03383 [cs.LG] (Published 2016-11-10)
Disentangling factors of variation in deep representations using adversarial training
arXiv:2006.08403 [cs.LG] (Published 2020-06-15)
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
arXiv:2006.00387 [cs.LG] (Published 2020-05-30)
Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training