arXiv Analytics

Sign in

arXiv:2206.11239 [cs.LG]AbstractReferencesReviewsResources

FedorAS: Federated Architecture Search under system heterogeneity

Lukasz Dudziak, Stefanos Laskaridis, Javier Fernandez-Marques

Published 2022-06-22Version 1

Federated learning (FL) has recently gained considerable attention due to its ability to use decentralised data while preserving privacy. However, it also poses additional challenges related to the heterogeneity of the participating devices, both in terms of their computational capabilities and contributed data. Meanwhile, Neural Architecture Search (NAS) has been successfully used with centralised datasets, producing state-of-the-art results in constrained (hardware-aware) and unconstrained settings. However, even the most recent work laying at the intersection of NAS and FL assumes homogeneous compute environment with datacenter-grade hardware and does not address the issues of working with constrained, heterogeneous devices. As a result, practical usage of NAS in a federated setting remains an open problem that we address in our work. We design our system, FedorAS, to discover and train promising architectures when dealing with devices of varying capabilities holding non-IID distributed data, and present empirical evidence of its effectiveness across different settings. Specifically, we evaluate FedorAS across datasets spanning three different modalities (vision, speech, text) and show its better performance compared to state-of-the-art federated solutions, while maintaining resource efficiency.

Related articles: Most relevant | Search more
arXiv:2007.16149 [cs.LG] (Published 2020-07-31)
HMCNAS: Neural Architecture Search using Hidden Markov Chains and Bayesian Optimization
arXiv:1611.01578 [cs.LG] (Published 2016-11-05)
Neural Architecture Search with Reinforcement Learning
arXiv:2203.14577 [cs.LG] (Published 2022-03-28)
Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?