---

Docs • Key Features • Installation • Quick Start • Contributing

MM-OpenFGL

📚 Introduction

MM-OpenFGL is a comprehensive open-source benchmark platform for Federated Graph Learning with Multimodal Data. It integrates state-of-the-art algorithms for learning from graphs that incorporate multiple modalities (text, image, audio) in a federated learning setting, enabling privacy-preserving distributed training across multiple parties while handling heterogeneous multimodal data.

Our Contributions

To advance Multimodal Federated Graph Learning research and establish a standardized evaluation framework, MM-OpenFGL provides the following key contributions:

1. Comprehensive Benchmark: MM-OpenFGL integrates 50+ federated graph learning algorithms across multiple paradigms, offering a unified framework supporting diverse multimodal tasks and federated learning scenarios.

2. Multimodal Support: Native support for text, image, and audio modalities with integrated feature extractors (768-dim representations) and multimodal fusion strategies for heterogeneous graph data.

3. Open-source Library: Designed as an extensible benchmark with unified APIs, comprehensive documentation, modular architecture, and user-friendly interfaces, fostering collaboration and innovation in the federated multimodal graph learning community.

🌟 Key Features

1. Rich Algorithm Collections

MM-OpenFGL integrates 40+ federated graph learning algorithms:

Federated Learning Algorithms

The integrated algorithms are grouped into four categories according to their target scenario:

FL for Vision	Architecture-Heterogeneous FL	Modality-Heterogeneous FL	Traditional FGL
FedAvg	FedProto	PepSy	FedGM
FedProx	FedTGP	FedMosaic	FGC
Scaffold	LG-FedAvg	FedMVP	FedSage+
MOON	FML	FedMAC	FedGTA
FedDC	FedKD	FedMM	GCFL+
FedExP	FIARSE	FedILoRA	FedStar
FLASC	ReeFL		FedSPA
GLocalFair	FedTSA		FedIIH
Calibre	MH-pFLID		FedSSP
	HAPFL		FedHERO
	MH-pFedHN		S2FGL
			FedLap
			FedGLS
			FedSheafHN
			FedDEP
			FCGL

Local Graph Learning Models

Traditional GNNs	Multimodal GNNs	Advanced GNNs
GCN	MMGCN	ChebNet
GAT	MGAT	REVGAT
GIN	MGNet	GSMN
GraphSAGE	MMA	MHGAT
		UniGraph2

Graph Foundation Models

Model	Description
GraphClip	Vision-language graph foundation model for cross-modal learning
UniGraph2	Universal graph representation learning framework
GFT	Graph Foundation Transformer for general graph learning
OFA	One-For-All prompt-based graph foundation model with MoE
RAGraph	Retrieval-Augmented Graph learning with adapter mechanisms
GFSE	Graph Foundation model with Structure Encoding
GQT	Graph Query Transformer for multi-task learning

2. Multimodal Data Support

• Text: Language representations (768-dim) via pretrained transformers
• Image: Visual feature extraction (768-dim) via vision models
• Audio: Audio embeddings (planned)
• Multimodal Fusion: Support for early, late, and hybrid fusion strategies

3. Comprehensive Task Support

Task Type	Task Name	Description
Node-level	Node Classification (node_cls)	Classify nodes in graphs
Node-level	Node Clustering (node_clust)	Cluster nodes based on features
Edge-level	Link Prediction (link_pred)	Predict missing edges in graphs
Graph-level	Graph Classification (graph_cls)	Classify entire graphs
Graph-level	Graph Regression (graph_reg)	Regression on graph-level properties
Multimodal	Modal Matching (modal_match)	Match modalities across nodes
Multimodal	Modal Retrieval (modal_retrieval)	Retrieve cross-modal content
Multimodal	Modal Alignment (modal_align)	Align visual and textual representations
Generation	Graph-to-Text (g2text)	Generate text descriptions from graphs
Generation	Graph-to-Image (g2image)	Generate images from graph structures

4. Flexible Federated Learning Scenarios

• Horizontal Federated Learning: Data partitioning across clients
• Vertical Federated Learning: Feature partitioning
• Personalized Federated Learning: Client-specific model adaptation
• Heterogeneous Federated Learning: Support for heterogeneous client models
• Cross-silo and Cross-device: Multiple communication protocols

5. Easy Configuration & Extensibility

• argparse-based configuration with YAML dataset configs
• Modular architecture for adding new algorithms, models, and tasks
• Distributed dataset loaders (FGLDataset) for efficient data handling
• Rich logging and monitoring utilities via Logger
• Extensive preprocessing and simulation tools

🗂️ Dataset Overview

MM-OpenFGL supports diverse multimodal graph datasets across multiple domains:

Multimodal Recommendation Datasets

• E-commerce: Movies, Toys, Grocery, Clothing, Books, Electronics
• Social Media: Bili-Cartoon, Bili-Dance, Bili-Food, Bili-Movie, Bili-Music
• Short Video: Douyin (DY), Kuaishou (KU), Qutoutiao (QB), Tencent News (TN)

Multimodal Social Networks

• Image-Text Networks: Flickr30k, RedditS
• Multi-platform: Fashion, Ads, Twitter, Facebook (MultiMET datasets)

Citation and Academic Networks

• Citation Networks: Cora, Citeseer, PubMed, DBLP, ogbn-arxiv
• Co-author Networks: CS, Physics

Additional Datasets

• Co-purchase Networks: Photo, Computers, ogbn-products
• Art Datasets: SemArt (semantic art understanding)

📥 Installation

Prerequisites

• Python: 3.10 or higher
• CUDA: 12.8 (recommended for GPU acceleration)
• pip or conda package manager

Step 1: Clone the Repository

git clone https://github.com/striker2333/MM-OpenFGL.git
cd MM-OpenFGL

Step 2: Create a Virtual Environment

Using conda (recommended):

conda create -n mm-openfgl python=3.10
conda activate mm-openfgl

Or using venv:

python -m venv mm-openfgl-env
source mm-openfgl-env/bin/activate  # Unix/MacOS
# or
mm-openfgl-env\Scripts\activate  # Windows

Step 3: Install PyTorch with CUDA Support

For CUDA 12.8:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

For other CUDA versions, refer to PyTorch installation guide.

Step 4: Install PyTorch Geometric Dependencies

pip install torch-scatter torch-sparse torch-cluster torch-spline-conv -f https://data.pyg.org/whl/torch-2.2.0+cu128.html
pip install torch-geometric

Adjust the URL for your specific PyTorch and CUDA versions.

Step 5: Install MM-OpenFGL and Dependencies

pip install -r source/requirements.txt

The requirements.txt includes:

• PyTorch Geometric (graph neural network library)
• NumPy, SciPy, Scikit-learn (scientific computing)
• Transformers (for text/image feature extraction)
• PyYAML (configuration management)
• OGB (Open Graph Benchmark datasets)
• Additional utilities for distributed computing and evaluation

Verify Installation

python -c "import torch; import torch_geometric; print('Installation successful!')"

GPU Setup

Check CUDA availability:

python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"

🚀 Quick Start

Basic Usage Example

After installation, run a federated node classification task:

import mm_openfgl.configs.config as config
from mm_openfgl.trainers import load_trainer

# Configure experiment
args = config.args
args.root = "path/to/your/datasets"

# Set federated learning scenario
args.simulation_mode = "subgraph_fl_louvain"
args.task = "node_cls"
args.fl_algorithm = "fedavg"
args.model = ["gcn"]
args.metrics = ["accuracy"]

# Run experiment
for run_id in range(args.num_runs):
    trainer = load_trainer(args)
    trainer.train()

Command Line Usage

cd src
python main.py \
    --dataset Movies \
    --task node_cls \
    --fl_algorithm fedavg \
    --model gcn \
    --num_rounds 100 \
    --num_clients 5

Available Parameters

• --dataset: Dataset name (Movies, Cora, Flickr30k, etc.)
• --task: Task type (node_cls, link_pred, modal_match, modal_align, g2text, g2image, etc.)
• --fl_algorithm: Federated algorithm (fedavg, fedprox, fedgc, fedmm, etc.)
• --model: GNN backbone (gcn, gat, gin, graphsage, mmgcn, mgat, etc.)
• --num_rounds: Number of communication rounds
• --num_clients: Number of federated clients
• --batch_size: Training batch size

For complete parameter documentation, refer to src/mm_openfgl/configs/config.py.

Example: Multimodal Federated Learning

python main.py \
    --dataset Flickr30k \
    --task modal_match \
    --fl_algorithm fedmm \
    --model mmgcn \
    --modalities image text \
    --num_rounds 50

Example: Graph Foundation Model

python main.py \
    --dataset Movies \
    --task node_cls \
    --fl_algorithm fedavg \
    --model graphclip \
    --use_pretrain \
    --num_rounds 30

Example: Modal Alignment Task

python main.py \
    --dataset Flickr30k \
    --task modal_align \
    --fl_algorithm fedavg \
    --model mmgcn \
    --num_rounds 50

Example: Graph-to-Image Generation

python main.py \
    --dataset SemArt \
    --task g2image \
    --fl_algorithm fedavg \
    --model mgnet \
    --num_rounds 100

📖 Documentation

Comprehensive documentation is available at: https://mm-openfgl.readthedocs.io/

Topics covered:

• Tutorials: Quick start guides and configuration
• API Reference: Detailed module documentation
• Examples: Sample code and use cases
• Custom Algorithms: Guide to implementing new methods

🤝 How to Contribute

We welcome contributions from the community!

Contributing Guidelines

1. Fork the Repository: Create a fork on GitHub
2. Create a Branch: Develop on a feature branch
3. Submit a Pull Request: Submit for review when ready
4. Report Issues: Open issues for bugs or suggestions

Please ensure contributions include:

• Appropriate tests
• Documentation updates
• Code following project style

📧 Contact

• Issues: GitHub Issues
• Documentation: ReadTheDocs
• Discussions: GitHub Discussions

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

We thank the open-source community for their invaluable contributions and the researchers whose work forms the foundation of this benchmark platform.