Repository navigation
Overview
Propius is a Federated Learning (FL) resource manager. You can define a FL job, specifying number of clients (edge devices) the job requires, and number of estimated job round. Propius can then route all the clients you need back to you! Additionally, you can set constraints about the device specification (OS version, memory size, etc) for your job so that your FL task can run smoothly on the edge devices, and Propius can make sure that your requirement will be satisfied!
Propius uses GRPC protocal for communication, and Redis for data storage.
Scheduler is the componenent that gives a priority score to every FL job, so that multiple jobs active in Propius can be scheduled with edge device resources. Currently, Propius supports FIFO, Random, SRJF, and IRS, our resource-aware algorithm that has the goal to speed up your FL job!
Job manager handles incoming job request. It parses the requirement sent from the job provider, and stores the processed job information in the job database so that client manager can later fetch the job and assign it to clients. Also, job manager will inform the scheduler that a new job has arrived, and let the scheduler update the score for this job and other jobs.
At the beginning of a FL job lifetime, job_info should be sent to job manager, specifying:
- the parameter server address (
ip,port) that hosts the FL job -
public_constraintof the job on clients (edge device). Public constraints should be constraints on client device specifications that are not sensitive in nature, and client devices are able to provide those information to Propius. Propius maintains a list of constraint options for job to selectively choose and define. -
private_constraintof the job on clients (edge device). Private constraints are constraints on sensitive client specifications. Propius maintains a list of constraint options for job to selectively choose and define. -
est_demand(estimated job round demand) of the FL job. Though the job provider can change demand at the start of each FL round, Propius asks for a estimated round demand of each job to make better scheduling decisions -
est_total_round(estimated number of rounds in total) of the FL job. Though the job provider can terminate the FL job earlier (before the actual round number equals to this total round number), Propius acks for this estimation to make better scheduling decisions. Please note that this estimation would also serve as the maximum round service Propius is able to provide to this FL job. If the actual round exceeds this number, Propius will terminate its service to ensure fairness.
Job manager will return a job_register_ack message to job, indicating its job IF, and register status boolean.
Job should make a request to Propius job manager at the start of each round to inform Propius that the job is ready for new clients. Propius will then start to assign this job to eligible clients until the demand for this round is satisfied. Job should provide:
-
id: the ID of the job assigned by Propius earlier during the register stage -
demand: current round demand number for clients
Job manager will return ack message indicating whether the request is successful.
Job is recommended to make a end request to Propius job manager to terminate job scheduling phase of a round, whether the actual client allocation number hits the pre-requested amount. Job need to provide job_id message indicating the ID of the job. If successful, job manager will return acknowledge message (ack=True) message to job.
Job is advised to make a finish request to Propius job manager at the end of the job. Job will be terminated by this call, even when the round hasn't reached the estimated total round. Job need to provide job_id message indicating the ID of the job.
Client manager handles client traffic. A client checks in to Propius through client manager which will store the client metadata such as public specifications to client database. The most important role of client manager is offering eligible tasks (we call client-side job as task) to clients by performing a filter operation on job database based on client specification. Every client, upon recieving task offer list, will then select the most suitable task locally based on private conditions. Finally, the client will inform client manager about its final choice, and client manager will return the address of the parameter server that handles the task if the task is available.
To handle the large client traffic, Propius can horizontally scale-up client manager and client database. A load balancer will distribute traffic to every client manager evenly.
Client will first check in to client manager, providing its public specifications (public_specification). Client manager will return:
-
client_id: a unique client id -
task_offer_list: a list of available tasks (client spec satisfies task constraint, and task is actively being assigned) -
private_constraint: a list of private constraints of the tasks in task offer list, for clients to select task locally total_job_num
The task list might be empty if there is no available task for the client in the database.
Client will ping to client manager if the CLIENT_CHECKIN returns an empty task offer list, or there is no available task in the list for the client after locally filtering the task based on private conditions such as dataset size. The client needs to provide a id, recieved from CLIENT_CHECKIN call. Client manager will return the same response as CLIENT_CHECKIN call. Note that the client does not need to provide its specs again as the data has already been stored in client database through the CLIENT_CHECKIN call.
Client needs to inform client manager about its task selection after recieving task offer list, by providing its client_id and task_id of the selected task. Upon recieving this call, client manager will increment allocation amount of the corresponding job of its current round, ack, and return job parameter server address (job_ip, 'job_port') if current amount is smaller than the round demand. Otherwise, job allocation amount will not be increased as job demand has already been satisfied by other checked-in clients, and this client fails to be assigned to this task (job). If this is the case, the client may choose to ping again to get a new task offer list, or select the next desirable task from the previous task offer list.