Feature Addition (regarding Modularity / Running Multiple Models Simultaneously / Aggregator Direct-Inference / Docker containerization / Pre-pulling policies)

[ejbrannonwu]

• Modularity for Aggregator/Nodes (commands require extra parameters; init-command now requires index, module, module_file & db_name, all subsequent commands just require index)
  • module must be the name of the class in the module_path file (ex. MnistDataHandler)
  • module_file is appended to the directory path TRAINING_APPLICATION_DIR that is mentioned in the .env's, edgefl/platform_components/data_handlers
    (ex. mnist_data_handler.py)
  • db_name must be what postgres contains, data in the DB must relate to the module (ex. mnist_fl)
• Tested running multiple models simultaneously (same data-handler)
• Tested running multiple models simultaneously (different data-handlers - winniio/mnist)
• Enabled the aggregator to run direct-inference (new endpoint command, example/explanation shown in README)
• Pre-pulling policies in the policy listener function for aggregator
• Docker Containerization
• Minor fixes:
  • Updated README.md with command updates
  • edited logger messages
  • checking for duplicate index
  • fixed sub-directories
  • moved db checking to init
  • connection to specified db_name in init if edgelake node doesn't have a connection to that DB - connect_to_db() (needs to have had data been connected in postgres already)
  • fixed default .env files to work with module_file
  • Add a message at the end of a training process (via logger) to indicate the training has finished
  • check that module_path exists (if not, don't initialize)
  • node checked to see if actually initialized, clarifying message if not (used to show success even if off)

[ejbrannonwu]

edit 05/15:

• added additional feature regarding aggregator running direct-inference and some more minor fixes (bottom 3 above)

[ejbrannonwu]

edit 05/26: added to pre-main

• Pre-pulling policies in the policy listener function for aggregator
• Docker Containerization

[royshadmon]

tested and lgtm