arXiv Analytics

Sign in

arXiv:2206.07802 [cs.CV]AbstractReferencesReviewsResources

What makes domain generalization hard?

Spandan Madan, Li You, Mengmi Zhang, Hanspeter Pfister, Gabriel Kreiman

Published 2022-06-15Version 1

While several methodologies have been proposed for the daunting task of domain generalization, understanding what makes this task challenging has received little attention. Here we present SemanticDG (Semantic Domain Generalization): a benchmark with 15 photo-realistic domains with the same geometry, scene layout and camera parameters as the popular 3D ScanNet dataset, but with controlled domain shifts in lighting, materials, and viewpoints. Using this benchmark, we investigate the impact of each of these semantic shifts on generalization independently. Visual recognition models easily generalize to novel lighting, but struggle with distribution shifts in materials and viewpoints. Inspired by human vision, we hypothesize that scene context can serve as a bridge to help models generalize across material and viewpoint domain shifts and propose a context-aware vision transformer along with a contrastive loss over material and viewpoint changes to address these domain shifts. Our approach (dubbed as CDCNet) outperforms existing domain generalization methods by over an 18% margin. As a critical benchmark, we also conduct psychophysics experiments and find that humans generalize equally well across lighting, materials and viewpoints. The benchmark and computational model introduced here help understand the challenges associated with generalization across domains and provide initial steps towards extrapolation to semantic distribution shifts. We include all data and source code in the supplement.

Related articles: Most relevant | Search more
arXiv:2304.06968 [cs.CV] (Published 2023-04-14)
Domain shifts in dermoscopic skin cancer datasets: Evaluation of essential limitations for clinical translation
arXiv:2212.04613 [cs.CV] (Published 2022-12-09)
Contrastive View Design Strategies to Enhance Robustness to Domain Shifts in Downstream Object Detection
arXiv:2207.10150 [cs.CV] (Published 2022-07-20)
Tackling Long-Tailed Category Distribution Under Domain Shifts
Xiao Gu et al.