arXiv Analytics

Sign in

arXiv:1509.08745 [cs.LG]AbstractReferencesReviewsResources

Compression of Deep Neural Networks on the Fly

Guillaume Soulié, Vincent Gripon, Maëlys Robert

Published 2015-09-29Version 1

Because of their performance, deep neural networks are increasingly used for object recognition. They are particularly attractive because of their ability to 'absorb' great quantities of labeled data through millions of parameters. However, as the accuracy and the model sizes increase, so does the memory requirements of the classifiers. This prohibits their usage on resource limited hardware, including cell phones or other embedded devices. We introduce a novel compression method for deep neural networks that performs during the learning phase. It consists in adding an extra regularization term to the cost function of fully-connected layers. We combine this method with Product Quantization (PQ) of the trained weights for higher savings in memory and storage consumption. We evaluate our method on two data sets (MNIST and CIFAR10), on which we achieve significantly larger compression than state-of-the-art methods.

Comments: Under review as a conference paper at SPL 2015. arXiv admin note: text overlap with arXiv:1412.6115 by other authors
Categories: cs.LG, cs.CV, cs.NE
Related articles: Most relevant | Search more
arXiv:1511.05497 [cs.LG] (Published 2015-11-17)
Learning the Architecture of Deep Neural Networks
arXiv:1605.05359 [cs.LG] (Published 2016-05-17)
Hierarchical Reinforcement Learning using Spatio-Temporal Abstractions and Deep Neural Networks
arXiv:1601.00917 [cs.LG] (Published 2016-01-05)
Distilling Reverse-Mode Automatic Differentiation (DrMAD) for Optimizing Hyperparameters of Deep Neural Networks