Platform Near Edge Support - PoC Dev Tracker

[devguyio]

Requirements

1. As a MLOps engineer I would like to take a trained model and package it into a container so that it can be deployed to the near edge

   • As a MLOps engineer I would like to take a trained model and use an automated pipeline to package it inside a container so that it can be deployed to the near edge. #2
   • As a MLOps engineer I would like to include all the dependencies required for my model in the container so I can be sure that it will run in a disconnected environment. #3
   • As a MLOps engineer I would like to register my model on a Quay registry so I can version it and make it available for deployment. #4
   • As a MLOps engineer I would like to perform automated tests to test the functional behavior of the container with the trained model. #5

2. As a MLOps engineer I would like to use ACM GitOps tools to deploy my model at the near edge.  #10

   • As a MLOps engineer I would like to use git as my source of truth for the deployment of my containerized model, so I can manage its lifecycle using a GitOps approach. #11
   • As a MLOps engineer I would like to deploy my containerized model to the near edge using ACM pull model, so I can meet the security requirement about the connection to a hub/core data center. #21
   • As a MLOps engineer I would like to see the topology of my applications installed at the edge, so I can have more control over what containerized models are deployed in which edge location. #22

3. As a MLOps engineer I would like to use ACM prometheus based solution to monitor the operational metrics of my containerized model at the near edge.

   • As a MLOps engineer I would like to have a collector of prometheus metrics at my edge node, so I can collect the operational metrics specific to my model. #6
   • As a MLOps engineer I would like to use ACM to send the metrics of my model back to my core data center/hub so I can visualize them and monitor in a centralized way. #7

4. As a MLOps engineer, I would like to expose the inferencing via REST and gRCP.

   • As a MLOps engineer, I would like to use a serving framework (e.g. OpenVINO, MLserver, etc.) so that I don't need to implement gRPC and REST services by myself (and other common functions) #1

[LaVLaS]

Closing this as complete for the August PoC use case