arXiv Analytics

Sign in

arXiv:2310.18564 [cs.LG]AbstractReferencesReviewsResources

A General Framework for Robust G-Invariance in G-Equivariant Networks

Sophia Sanborn, Nina Miolane

Published 2023-10-28Version 1

We introduce a general method for achieving robust group-invariance in group-equivariant convolutional neural networks ($G$-CNNs), which we call the $G$-triple-correlation ($G$-TC) layer. The approach leverages the theory of the triple-correlation on groups, which is the unique, lowest-degree polynomial invariant map that is also complete. Many commonly used invariant maps - such as the max - are incomplete: they remove both group and signal structure. A complete invariant, by contrast, removes only the variation due to the actions of the group, while preserving all information about the structure of the signal. The completeness of the triple correlation endows the $G$-TC layer with strong robustness, which can be observed in its resistance to invariance-based adversarial attacks. In addition, we observe that it yields measurable improvements in classification accuracy over standard Max $G$-Pooling in $G$-CNN architectures. We provide a general and efficient implementation of the method for any discretized group, which requires only a table defining the group's product structure. We demonstrate the benefits of this method for $G$-CNNs defined on both commutative and non-commutative groups - $SO(2)$, $O(2)$, $SO(3)$, and $O(3)$ (discretized as the cyclic $C8$, dihedral $D16$, chiral octahedral $O$ and full octahedral $O_h$ groups) - acting on $\mathbb{R}^2$ and $\mathbb{R}^3$ on both $G$-MNIST and $G$-ModelNet10 datasets.

Journal: Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023)
Categories: cs.LG, cs.AI, cs.CV
Related articles: Most relevant | Search more
arXiv:2403.13249 [cs.LG] (Published 2024-03-20)
A Unified and General Framework for Continual Learning
arXiv:2402.01922 [cs.LG] (Published 2024-02-02, updated 2024-05-28)
A General Framework for Learning from Weak Supervision
Hao Chen et al.
arXiv:2312.16560 [cs.LG] (Published 2023-12-27)
Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching