Federated Learning Security and Evaluation Framework for Foundation Models
FENIX is a research project developed during the Industrial Immersion Program at AI Sweden.
The project focused on evaluating and improving the robustness of large language models (LLMs) in federated learning environments, with a particular emphasis on:
- Foundation model evaluation
- Federated learning workflows
- Data poisoning attacks
- Model poisoning attacks
- Security and robustness analysis
- Benchmark development
The primary model used in the project was GPT-SW3, one of Sweden's large language models.
As large language models become increasingly distributed across organizations and infrastructure environments, federated learning introduces new opportunities and challenges.
While federated training can improve privacy and collaboration, it also introduces additional attack surfaces:
- Malicious client updates
- Data poisoning
- Model poisoning
- Evaluation inconsistencies
This project explored how these risks affect foundation models and how benchmarking frameworks can be used to identify weaknesses and improve reliability.
Implemented and evaluated federated learning workflows for foundation models.
Topics explored:
- Distributed training environments
- Client-server update aggregation
- Model synchronization
- Federated evaluation pipelines
Developed benchmarking workflows for GPT-SW3-based systems.
Objectives:
- Measure model performance
- Compare evaluation results across training scenarios
- Improve reproducibility of benchmark results
Conducted simulations of adversarial attacks against federated learning systems.
Attack scenarios included:
- Data poisoning
- Model poisoning
- Malicious update injection
The goal was to identify vulnerabilities and better understand failure modes in distributed AI systems.
Fed_simple/
Fed_simple_MP/
Fed_LoRA/
Fed_LoRA_MP/
training_federated_full.py
prompt_generation.py
Core areas:
- Federated training workflows
- LoRA-based fine-tuning experiments
- Prompt generation
- Benchmark evaluation
Through this project, I gained practical experience with:
- Federated Learning
- Foundation Models
- GPT-SW3
- Distributed AI Systems
- AI Security Research
- Benchmark Development
- Model Evaluation Methodologies
This project was conducted as part of the AI Sweden Industrial Immersion Program in collaboration with Dakota State University.
The research contributed to:
- Foundation model benchmarking
- Security evaluation of federated learning systems
- Robustness analysis for distributed AI environments
- Python
- GPT-SW3
- Federated Learning
- LoRA
- Machine Learning Evaluation Pipelines
The long-term objective of FENIX is to better understand how foundation models behave in distributed environments and how security-focused evaluation frameworks can improve the reliability of future AI systems.