Overview

Propius is a Federated Learning (FL) resource manager. You can define a FL job, specifying number of clients (edge devices) the job requires, and number of estimated job round. Propius can then route all the clients you need back to you! Additionally, you can set constraints about the device specification (OS version, memory size, etc) for your job so that your FL task can run smoothly on the edge devices, and Propius can make sure that your requirement will be satisfied!

Propius uses GRPC protocal for communication, and Redis for data storage.

Scheduler

Scheduler is the componenent that gives a priority score to every FL job, so that multiple jobs active in Propius can be scheduled with edge device resources. Currently, Propius supports FIFO, Random, SRJF, and IRS, our resource-aware algorithm that has the goal to speed up your FL job!

Job manager

Job manager handles incoming job request. It parses the requirement sent from the job provider, and stores the processed job information in the job database so that client manager can later fetch the job and assign it to clients. Also, job manager will inform the scheduler that a new job has arrived, and let the scheduler update the score for this job and other jobs.

At the beginning of a FL job lifetime, job_info should be sent to job manager, specifying:

1. the parameter server address (ip, port) that hosts the FL job
2. public_constraint of the job on clients (edge device). Public constraints should be constraints on client device specifications that are not sensitive in nature, and client devices are able to provide those information to Propius. Propius maintains a list of constraint options for job to selectively choose and define.
3. private_constraint of the job on clients (edge device). Private constraints are constraints on sensitive client specifications. Propius maintains a list of constraint options for job to selectively choose and define.
4. est_demand (estimated job round demand) of the FL job. Though the job provider can change demand at the start of each FL round, Propius asks for a estimated round demand of each job to make better scheduling decisions
5. est_total_round (estimated number of rounds in total) of the FL job. Though the job provider can terminate the FL job earlier (before the actual round number equals to this total round number), Propius acks for this estimation to make better scheduling decisions. Please note that this estimation would also serve as the maximum round service Propius is able to provide to this FL job. If the actual round exceeds this number, Propius will terminate its service to ensure fairness.

Job database

Client manager

Client database