Research-Papers

These are the papers I find interesting, mostly focused around the intersection of security, privacy, and ML. I may also list papers relating to the fundamentals of ML/FL infrastructure, or topics involving AI alignment and fairness. There also might be non-papers in here! I am including whatever helps me grasp the concepts the easiest.

See OpenMined for a brief overview of the types of FL.

This list will be organized by topic and attack model (if applicable).

Table of Contents

• Research Papers
  • Table of Contents
  • Privacy
    • Defenses
  • Security
    • Attacks
    • Defenses
  • Vertical FL
  • FL Optimization
  • FL Systems from big tech companies
    • Paper
  • Data Center Architecture
  • Surveys
  • LLMs
  • MLSys
  • Other FL Paper Lists
  • Other Lists

Privacy

Defenses

• IBM (Cloud'22): DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting PDF

Security

Attacks

Model Poisoning

• (ICML'19): Analyzing Federated Learning through an Adversarial Lens PDF Github
  • Attack Model: "Single, non-colluding malicious agent where the adversarial objective is to cause the model to mis-classify a set of chosen inputs with high confidence."

Defenses

Model Poisoning

• Federated Learning based on Defending Against Data Poisoning Attacks in IoT PDF

  • Attack Model: "A group of p<n/2 malicious label-flipping poisoning attackers, where n is the total amount of participants’ clients."

• (NeurIPS'21): FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective PDF Github

  • Attack Model: "Clients mitigate model poisoning attacks that have already polluted the global model"

Vertical FL

• Vertical Federated Learning: Challenges, Methodologies and Experiments PDF

FL Optimization

• Oort: Efficient Federated Learning via Guided Participant SelectionPDF | OSDI 21 🎓
• (ICML'22): Neural Tangent Kernel Empowered Federated Learning PDF
  • Reduces communication rounds, addresses statistical heterogeneity by transmitting update data that is more expressive than simple model weights/gradients
• Fed-SNN: Federated Learning with Spiking Neural Networks PDF Github
  • Optimizes for energy efficiency
• Swan: A Neural Engine for Efficient DNN Training on Smartphone SoCs PDF
• (ICLR 2021): Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms PDF Github

FL Systems from big tech companies

Paper

Cross-device

• Apple: Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications | PDF, PDF
• Google: Towards Federated Learning at Scale: System Design | MLSys21, Github🎓
• Meta: Papaya: Practical, Private, and Scalable Federated Learning | MLSys22 🎓

Data Center Architecture

• Yarn: PDF
  • Also see: GFG, Cloudera
• Omega: PDF
  • Also see: CS294, CSUDH
• Tiresias: A GPU Cluster Manager for Distributed Deep Learning | PDF
• Leap: Effectively Prefetching Remote Memory | PDF, Github (USENIX'20)🎓
  • Two tricks: Prefetching pages wherever possible
  • Using more efficient data paths that allow them to discard the operating system’s irrelevant disk-access features.

Surveys

• A survey on security and privacy of federated learning URL
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges PDF

LLMs

• In AI, is bigger always better? Nature

• Voyager, An Open-Ended Embodied Agent with Large Language Models Website

  • Vector Database of skills (GPT-4 Generated Code). Keys are descriptions, while the Value is the code of "skills"

• MemGPT: Towards LLMs as Operating Systems PDF

  • LLMs are constrained by limited context windows, hindering their utility in tasks like extended conversations and document analysis
  • MemGPT manages different memory tiers to provide the appearance of large memory resources through data movement between fast and slow memory (similar to traditional OS virtual context management)

• Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents arxiv

  • LLMs roleplay as doctors, nurses, patients
  • "After treating around ten thousand patients (real-world doctors may take over two years), the evolved doctor agent achieves a state-of-the-art accuracy of 93.06% on a subset of the MedQA dataset that covers major respiratory diseases."

• (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts arxiv

MLSys

• Hidden Technical Debt in Machine Learning Systems [NeurIPS PDF](https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf

Other FL paper lists

• https://github.com/AmberLJC/FLsystem-paper/
• ***https://github.com/innovation-cat/Awesome-Federated-Machine-Learning
• https://github.com/chaoyanghe/Awesome-Federated-Learning
• https://github.com/weimingwill/awesome-federated-learning#resource-allocation
• https://github.com/youngfish42/Awesome-Federated-Learning-on-Graph-and-Tabular-Data#federated-learning-framework

Other Lists

• UMich EECS 598: Systems for GenAI