Margo Preview Release 1 Scope: Workload Management Interface

[ajcraig]

Purpose

The goal of this Issue is to establish the Margo Preview Release 1 scope. The scope snippet below covers the Workload Management Interface. The goal of this issue is to gain approval on the proposed Preview Release 1 scope. Additional issues will be released for approval regarding the full scope.

** Updated 11/13/2024 to include details about interoperability scenarios for this section of the specification.

Workload Management Interface Scope

1. The ability for the workload orchestration service to provide deployment specification(s) that follow the Margo standard. Link

   • This features enabled via the deployment specification include:
     • Deploy one to many workloads
     • Delete one to many existing workloads
     • Update workload(s) to a newer version of source code(container)
     • Reconfigure currently running workload(s) via parameters

2. The ability to build a Workload Orchestration Management service for orchestrating workloads to Margo compliant devices. Link

   • Host a API server endpoint that:
     • Receive the workload deployment status updates
     • Receive device capabilities
   • Host a Gitops enabled repository for providing desired state to the edge devices:
     • Deployment specification storage
     • Deployment specification retrieval

3. Device Workload Management Interface

• The ability to build a Rest API Client that enables the features below:
  • Send the workload deployment status updates as defined in the current specification
  • Send device capability information as defined in the current specification
    • Upon device reconfiguration, the interface should update the workload orchestrator of the new or changed capabilities.
• The ability to use a Git client or library to enable enable the following features:
  • Retrieve deployment specification associated with the device
  • Configure a polling schedule

What does interoperability mean to Margo/what personas are impacted?

• Device Vendor
  • Margo compliant devices are compatible with all Margo compliant WOS vendors.
  • Ensures device vendors are not responsible with supporting various edge "agents".
• Application Vendor
  • Applications built via Margo standard shall be supported on Margo compliant devices.
  • Freedom to only support 1 or both Deployment manifests(compose/helm)
• Workload Orch Software
  • Margo compliant WOS vendors are compatible with all Margo compliant devices.
• End User
  • End user has freedom to choose their choice of device and WOS solution based on features and functionality. This is with the knowledge, if they purchase Margo compliant devices they retain the freedom to switch.
  • Avoids vendor lockin. If End user is not happy with either Device or WOS, they are able to change vendors without having to replace all components in the architecture.

* Secure communication is not a priority for this release, although some details might be provided. Secure connectivity will be required in subsequent releases.

Decisions:

• Do we go with the initial implementation of OpenGitops, which is currently detailed in the specification, or work through various technical options before release.
• The following envisioned features are not included within the preview release 1:
  • Auditability and tracking history of deployment specification IDs
  • Rollback functionality
  • Disaster recovery requirements
  • Maintenance of the workload orchestration service components
  • Enabling container runtime configuration changes.

Review/Approval Options:

@margo/approvers Please respond below and consolidate your company's response to a single reply. If additional information is needed to provide your response, please reach out.

Response Options:

• Approve Proposal
• Reject Proposal - Provide reasoning for rejection
• Abstain from Vote - No strong opinion either way

[merrill-harriman-se]

Does this statement imply that GitOps will be required by the device to get access to the deployment spec?
"Host a Gitops enabled repository for providing desired state to the edge devices:
Deployment specification storage
Deployment specification retrieval"

I am not yet comfortable with requiring GitOps at the end device - not until we have fully analyzed and justified this over a simple REST interface. I suspect REST is simpler, less resource requirements, no need to open yet another network path, smaller attack surface, less maintenance, and can very likely provide all the needed functionality.

[effndc]

On behalf of Intel I am submitting an approval.

[phil-abb]

I am not yet comfortable with requiring GitOps at the end device - not until we have fully analyzed and justified this over a simple REST interface.

@merrill-harriman-se Would you consider this a blocker for any preview release, or would you be ok if there was a statement somewhere indicating we are still looking into how the desired state is handled and the use of GitOps may change?

[abbottjd]

From ABB, I approve this proposal for the PR1 scope. For the preview release, I don't see a problem with basing on GitOps but acknowledge that it makes sense to continue exploring alternative methods for storing/retrieving desired state before progressing to a formal specification release.

[srberard]

I have similar concerns regarding GitOps at the end device. While GitOps is a powerful deployment technique/paradigm, I feel that requiring it as part of Margo would add complexity, impede implementation, and be overly prescriptive.

A core strength of systems like Kubernetes is their flexibility. K8s does not, itself, prescribe GitOps but instead provides APIs that allow GitOps workflows via tools such as ArgoCD, FluxCD, etc. For those that desire to use GitOps, they can select the appropriate tool that fits their needs. I feel the Margo should strive to do something similar. That is, provide mechanisms to enable GitOps but not require it directly.

[phil-abb]

I have similar concerns regarding GitOps at the end device.

@srberard Same question to you. Would you consider this a blocker for any preview release, or would you be ok if there was a statement somewhere indicating we are still looking into how the desired state is handled and the use of GitOps may change?

[srberard]

I'd rather this not be part of the Preview. While I don't want to block progress, I think including it in the Preview gives the impression that this will be supported.

[securidee]

There is also no mention of a GitOps in the Release 1 scope for device requirements. That seems like a conflict between the workload management interface scope and the device scope.

[phil-abb]

There is also no mention of a GitOps in the Release 1 scope for device requirements.

@securidee I see this part of 3. The ability to host the Margo management interface service(s) that enable workload deployment and maintenance. because for this to be possible, the implementation the device vendor chooses to use would need to be able to work with a Git repository.

Do you see it differently? What do you feel should be added?

[securidee]

There is also no mention of a GitOps in the Release 1 scope for device requirements.

@securidee I see this part of 3. The ability to host the Margo management interface service(s) that enable workload deployment and maintenance. because for this to be possible, the implementation the device vendor chooses to use would need to be able to work with a Git repository.

Do you see it differently? What do you feel should be added?

You are correct. Sorry, I missed the nested requirements in 3. The ability to host the Margo management interface service(s) that enable workload deployment and maintenance. I mistakenly thought a device vendor could just read that portion of the spec, but the Device Workload Management Interface has more specific requirements that apply to the device.

[stormc]

I do see the need to go forward with a reasonable snapshot/prototype technical incarnation as Preview Release 1, knowing and clearly communicating that all technical decisions/specifications and hence this realization is most likely subject to change in a later (Preview) Release due to the gist of this preview release (from a 10k ft perspective): Give Application developers something to play with in order to provide feedback for the Margo "look+feel".

Following this argumentation, parts of the realization (e.g. GitOps vs. REST) are "just" implementation details that will most likely look/be different in the releases to come. Nonetheless, I think we need to come to a consensus on whether or not we can go forward with GitOps in this Preview Release 1, considering the objections in this thread which I second.

Having said that, the vote is: Abstain from Vote.

In particular for the GitOps vs. REST discussion at hand, I'd like to bring in @Silvanoc into the discussion who is very proficient (not only) in this topic. And I'd like to drop some reading material he shared with me:
https://sestegra.medium.com/gitops-with-flux-leveraging-oci-registry-as-a-state-store-ffe28dd64836
https://fluxcd.io/flux/cheatsheets/oci-artifacts/

[hsinghcg]

Capgemini View: Abstain from vote - no strong opinion either way.
Not opposed to using gitops in the Preview Release 1 though. Would like us to continue with our analysis on whether or not to impose gitops as the standard (thereby imposing some requirements for the device and the WOS).

This is also related to the topic of what 'range of devices' we'd intend to support - and what's the extreme in terms of the smallest device running linux with kubernetes or docker - and possibly to use that as a benchmark when we decide to impose technology (+ related default tooling or sw stack) choices.

[merrill-harriman-se]

I do not agree with a standards-based specification having a requirement for a protocol and API that are not themselves derived from a standard. GitOps is not a standard - it is a tool and even OpenGitOps is, at best, a DeFacto standard. This implies that there is no way to specify an API version of the "GitOps Standard" as it does not exist. It could change underneath our developments later and we cannot do anything to indicate what we support or not. This, on its own merit, prevents the requirement of GitOps in our standard. How will we have any form of conformance testing? Conformant to what?

[srberard]

I concur with @merrill-harriman-se, while we speak of GitOps, it is not a formal standard, it is a methodology and set of principles.

Further, as I stated previously, I am concerned that this will add complexity to Margo and impede progress. GitOps has been a point of discussion and debate for many weeks now and I believe is distracting us from moving forward. As the above proposal already requires a REST API interface, I would suggest we limit this proposal to that only and drop GitOps for now. We can revisit this later.

As such, I reject the proposal as written with the above reasoning.