[Menu]

COMP 6211G: Federated Learning (Spring 2021)

HKUST / Department of Computer Science and Engineering

Announcements

  • (09-Jan-2021) - Welcome to COMP6211G course website! Please note that week #1 starts on 01-Feb-2021 (Monday)

Course Information

Course Descriptions

The goal of this course is to introduce the concept, technologies, systems and applications related to an emerging machine learning field, federated learning (FL). Students will acquire fundamental knowledge of data privacy and security, privacy-preserving machine learning and distributed AI. The course will discuss new research and application trends in federated learning and cover new challenges and open problems in this field. In particular, some of the lectures will be designed to provide real-world implementations and use cases of FL and to encourage students to explore limitations and maturity of FL technologies. Besides the basic FL theories, students are required to read and present latest FL papers and conduct projects in the FL direction.

Zoom Meetings

The Zoom link for this course is here:

Prerequisites

The prerequisite knowledge for this course includes machine learning, basic computer systems and basic programming skills.

Course Outcomes

Students will attain the following on completion of the course:

  • Knowledge of the basic concepts, architecture and applications of FL.
  • Understanding of new research and application trends in FL.
  • Ability to deploy real-world FL projects.
  • hands-on experience in applying FL tools to solve privacy-preserving AI challenges.

Textbook

Textbook cover image.

Federated Learning
Qiang Yang, Yang Liu, Yong Cheng, Yan Kang,
Tianjian Chen, Han Yu Morgan & Claypool Publishers, 2019
ISBN: 978-1681736983

Lectures

Section Instructor Time
Introduction to Federated Learning Qiang Yang 02/02/2020
Privacy Preserving Machine Learning and Distributed Machine Learning Lixin Fan (Guest Lecture) 02/04/2020
Horizontal Federated Learning, Vertical Federated Learning and Federated Transfer Learning Yang Liu (Guest Lecture) 02/09/2020
Federated Learning applications: CV and Recommendation Lixin Fan (Guest Lecture) 02/11/2020
Federated Learning applications: Health Care, Finance and Edge Computing Qiang Yang 02/16/2020
Advances and Open Problems Yang Liu (Guest Lecture) 02/18/2020

Lab Tutorials

Section Time Venue

Academic Integrity

Colleagues may wish to be aware or reminded of a document which lays down a systematic process by which alleged violations of academic integrity are dealt with within the University. Known as A Framework on Academic Integrity, this document is available here.

Grading Scheme

Class Participation 5%
Mid-term Exam 30%
Paper Presentation 15%
Project proposal, final presentation & report 50%

Syllabus and Lecture Notes

Zoom Meetings for Lecture
  • The Zoom meeting links are published on HKUST Canvas
  • After you login, please click the "Zoom Meeting" menu item on the left-hand side
Part I
Date Week Contents Lecturer Recording
02/02/2020 1 Introduction to Federated Learning Qiang Yang Zoom Recording
02/04/2020 1 Privacy Preserving Machine Learning and Distributed Machine Learning Lixin Fan (Guest Lecture) Zoom Recording
02/09/2020 2 Horizontal Federated Learning, Vertical Federated Learning and Federated Transfer Learning Yang Liu (Guest Lecture) Zoom Recording
02/11/2020 2 Federated Learning applications: CV and Recommendation Lixin Fan (Guest Lecture) Zoom Recording
02/16/2020 3 Federated Learning applications: Health Care, Finance and Edge Computing Qiang Yang Zoom Recording
02/18/2020 3 Advances and Open Problems Yang Liu (Guest Lecture) Zoom Recording
Part II Tutorial and Midterm
Date Week Content
02/23/2021 4 Lab Tutorial
02/25/2021 4 Midterm Exam
Part III Paper Presentations
Date Week Paper Defender-Offender
03/02/2021 5 Federated Learning with Only Positive Labels Jiaxin Bai - Su Ying
03/02/2021 5 FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction Tiezheng Yu - Wenliang Dai
03/04/2021 5 Resource Allocation for Wireless Federated Edge Learning based on Data Importance Xiuzhu WANG - Linping QU
03/04/2021 5 Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning Zhenheng Tang - Zhikai Hu
03/04/2021 5 Analyzing Federated Learning through an Adversarial Lens Dohoon Kim - Qingcan Kang
03/09/2021 6 Peer-to-peer Federated Learning on Graphs Qi HU - Haoran Li
03/09/2021 6 FedBN: Federated Learning on Non-IID Features via Local Batch Normalization Weihang Dai - Shuhan Li
03/11/2021 6 Gossip Learning as a Decentralized Alternative to Federated Learning Yui Hong Ng - Yuchang Sun
03/11/2021 6 SecureBoost: A Lossless Federated Learning Framework Tsz Him CHEUNG - Hao Xu
03/16/2021 7 Federated Adversarial Domain Adaptation Junming Chen - Yuxin Pan
03/16/2021 7 FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems Yuxuan Yu - Chak Ming Chan
03/18/2021 7 Enabling Execution Assurance of Federated Learning at Untrusted Participants Zhifeng Jiang - Haoyu Ma
03/18/2021 7 FetchSGD:communication-efficient algorithms for federated learning Ying Su - Jiaxin Bai
03/30/2021 9 Empirical Studies of Institutional Federated Learning For Natural Language Processing Wenliang Dai - Tiezheng Yu
03/30/2021 9 Federated Visual Classification with Real-World Data Distribution Kai Chen - Dohoon Kim
04/08/2021 9 Federated Learning: Strategies for Improving Communication Efficiency Linping Qu - Xiuzhu Wang
04/08/2021 9 Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications Zhikai Hu - Zhenheng Tang
04/13/2021 10 Measure Contribution of Participants in Federated Learning Qingcan Kang - Tze Him Cheung
04/13/2021 10 Differentially Private Federated Learning: A Client Level Perspective Haoran Li - Qi Hu
04/15/2021 10 HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients Yuchang Sun - Junming Chen
04/15/2021 10 Multi-Participant Multi-Class Vertical Federated Learning Hao Xu - Yui Hong Ng
04/20/2021 11 Federated Learning with Non-IID Data Shuhan Li - Weihang Dai
04/20/2021 11 Optimizing Federated Learning on Non-IID Data with Reinforcement Learning Yuxin Pan - Zhifeng Jiang
04/22/2021 11 LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning Haoyu Ma - Kai Chen
04/22/2021 11 FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare Chak Ming Chan - Yuxuan Yu
Part IV Project Proposal & Presentation
Date Week Group Members
03/23/2021 8 Zhikai Hu, Zhenheng Tang
03/23/2021 8 Shuhan Li, Weihang Dai, Tze Him Cheung
03/23/2021 8 Xiuzhu Wang, Dohoon Kim
03/23/2021 8 Yuxin Pan, Yuxuan Yu, Qingcan Kang
03/25/2021 8 Yuchang Sun, Hao Xu, Junming Chen, Chak Ming Chan
03/25/2021 8 Ying Su, Jiaxin Bai, Haoran Li, Qi Hu
03/25/2021 8 Linping Qu, Weiliang Dai, Tiezheng Yu
Final Weeks 12 and 13 Presentation Orders

Lab Tutorials

Zoom Meetings for Lab

Lab # Week # Lab Materials Recorded lab video

Project Assignments

Final Project

The final project is an open project for students to explore federated learning. Students are supposed to conduct a research project. Students are divided into groups and a presentation and short paper is expected.

The following problems in federated learning is offered but students may pick a different one.

Problem list:

Horizontal Federated Learning

  • The protocol in horizontal federated learning to encrypt the gradient sent by parties.
  • Improving the robustness in horizontal federated learning especially in mobile devices.
  • non-iid ( not independently identically distributed ) dataset and its impact/solution in horizontal federated learning.

Vertical Federated Learning

  • The protocol in vertical federated learning to encrypt the data during training.
  • The private set intersection protocol in vertical federated learning.
  • Exploration of new application of vertical federated learning.

Others

  • Improving the performance ( throughput ) of federated learning system
  • The backdoor of federated learning

References

Open source projects:

Papers:

  • Q. Yang, Y. Liu, T. Chen & Y. Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, pp. 12:1–12:19 (2019).
  • P.Kairouz et al, Advances and Open Problems in Federated Learning, arXiv:1912.04977

Paper Lists:

  • Awesome Federated Learning
  • FederatedAI Research
  • Selected Papers:

  • Federated Machine Learning: Concept and Applications
  • Advances and Open Problems in Federated Learning
  • Privacy-Preserving Deep Learning
  • Scaling Distributed Machine Learning with the Parameter Server
  • Federated Learning with Non-IID Data
  • FedBN: Federated Learning on Non-IID Features via Local Batch Normalization
  • SecureBoost: A Lossless Federated Learning Framework
  • Multi-Participant Multi-Class Vertical Federated Learning
  • Secure and Efficient Federated Transfer Learning
  • A Secure Federated Transfer Learning Framework
  • Federated Learning Based on Dynamic Regularization
  • Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
  • FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
  • HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
  • How To Backdoor Federated Learning
  • Deep Leakage from Gradients
  • Analyzing Federated Learning through an Adversarial Lens
  • Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
  • TOWARDS FEDERATED LEARNING AT SCALE: SYSTEM DESIGN
  • FedML: A Research Library and Benchmark for Federated Machine Learning
  • Quantifying the Performance of Federated Transfer Learning
  • Adaptive Federated Optimization
  • FEDERATED LEARNING FOR MOBILE KEYBOARD PREDICTION
  • Training Keyword Spotting Models on Non-IID Data with Federated Learning
  • TA Information

    Name Email Office
    Zhenghang REN zrenak@connect.ust.hk Room 3661
    Yilun Jin yjinas@connect.ust.hk Room 3661
    Tianjian Chen tchenay@connect.ust.hk