Federated Learning with Non-iid data

Supervision: Prof. Dr. Nassir Navab, Dr. Shadi Albarqouni, Azade Farshad

Abstract

In recent years, the interest in the field of Federated learning has been increasing. This interest peaks in medical machine learning because of the nature of data and the privacy requirements in this field. Despite of the high efforts in commercial federated learning such as mobile phone keyboard prediction, there is still much to do in the medical field. The medical data has non-iid nature which means the data in different clients (nodes) come from different distributions. Most of the current federated learning methods focus on iid problem. Our goal in this thesis is to improve a neural networks generalization in a non-iid setting. Our solution for this problem is to use meta-information and statistical information for guiding the global model in the network. We take this investigation further by employing meta-learning methods for personalization of the clients models in our federated setting.

Requirements:

• Good understanding of statistics and machine learning methods.
• Very good programming skills in Python & TensorFlow^? / PyTorch^?

Location:

• Garching

Literature

[1] McMahan^?, H.B., Moore, E., Ramage, D., Hampson, S., et al.: Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629 (2016).

[2] Chen, F., Dong, Z., Li, Z., He, X.: Federated meta-learning for recommendation.arXiv preprint arXiv:1802.07876 (2018).

[3] Corinzia, L., Buhmann, J.M.: Variational federated multi-task learning. arXiv preprint arXiv:1906.06268 (2019).

[4] Hsieh, K., Phanishayee, A., Mutlu, O., Gibbons, P.B.: The non-iid data quagmire of decentralized machine learning. arXiv preprint arXiv:1910.00189 (2019).

[5] Jiang, Y., Koneˇcn`y, J., Rush, K., Kannan, S.: Improving federated learning personalization via model agnostic meta learning. arXiv preprint arXiv:1909.12488(2019).

[6] Li, D., Wang, J.: Fedmd: Heterogenous federated learning via model distillation.arXiv preprint arXiv:1910.03581 (2019).

[7] Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127 (2018).

[8] Li, W., Milletari, F., Xu, D., Rieke, N., Hancox, J., Zhu, W., Baust, M., Cheng, Y.,Ourselin, S., Cardoso, M.J., et al.: Privacy-preserving federated brain tumour segmentation. In: International Workshop on Machine Learning in Medical Imaging.pp. 133–141. Springer (2019).

[9] Sheller, M.J., Reina, G.A., Edwards, B., Martin, J., Bakas, S.: Multi-institutionaldeep learning modeling without sharing patient data: A feasibility study on braintumor segmentation. In: International MICCAI Brainlesion Workshop. pp. 92–104.Springer (2018).

Resultant Paper