Taxonomy draft work in progress#53
Conversation
|
Question about the domains. It seems like "orchestration" is a capability of the control plane domain. And "automation" is a capability of the composable infra domain. And the control plane interacts with the compostable infra domain (subject to governance and policy). I think we refactor "orchestration and automation" as capabilities and leave it to four domains. |
"Compostable?" LOL, I make the same mistake when I'm actually talking about it our loud! So here's why I would argue to keep a distinct Orchestration & Automation domain. The DCM architecture calls out the concept of two important core functions, Naturalization and Denaturalization. This is the current definition: Naturalization
Denaturalization is simply converting in the opposite direction and Unified, as I understand it, means abstracted so that Users acting within the Developer Experience domain are insulated from the infrastructure specifics of the Service Providers providing the basic compute, storage, and networking Resources living in the Composable Infrastructure domain. So, let's look at this through the lens of an example of how we might implement the Naturalization and Denaturalization functions of DCM. I think Ansible is an obvious solution for this. Assuming the Resources defined by each Service Provider will be managed by its specific Ansible collection, the modules in its collection would natively implement these functions. Ansible action modules do the Naturalization enabling the Unified API in the Control Plane domain to Provision, Decommission, make State changes, etc., to the Resources defined by a specific Service Provider. Likewise, the Ansible facts and/or info modules in the collection do the Denaturalization, translating Service Provider specifics of Resource discovery and current State back to something consumable for use by the Unified API and data model. If you prefer Terraform for this example, it would be the Terraform provider plugins specific to each Service Provider that implement these functions. My concern with lumping all this under the Control Plane domain is I want to keep it focused on the Unified API and data model. Putting Naturalization and Denaturalization functions within the Unifed world of the Control Plane domain muddies the waters with Service Provider product specifics. Those should be invisible to the Users calling the Control Plane from Developer Experience domain. By the way, all of the capitalized terms above (Naturalization, Denaturalization, Unified, Service Provider, Resource, Provision, Decommission, State, Users) are included in what I'm working on defining very precisely with the vocabulary updates I'll be adding soon to the draft PR. Happy to debate further if you are not convinced? |
|
I think Naturalization and Denaturalization could both live in the composable infra domain (I'm assuming this is where Service Providers live, and I think Naturalization and Denaturalization should occur within those service providers so that the DCM Control Plane has a consistent interface across all Service Providers). I got here by thinking about mapping the data model types to domains:
I'm still struggling with the terms "orchestration & automation" as a distinct domain....both of those capabilities would be used in almost all other domains. |
|
Yes, Composable Infra domain is where Service Providers live. I still think there needs to be something in the middle.
I don't think that is possible. We can't just wave a magic wand to make the BMC of a bare metal server or the API of virtualization platform conform to the Unified API. There needs to be something in between that does that for all the different Service Providers. Much like how I want to keep the Service Provider specific APIs out of the Unified world of the Control Plane, I also want to allow the Composable Infra domain to operate independently of any Unified requirements. While working through more of the vocabulary, I am finding minor changes I want to make the domain definitions more precise. Maybe we just need to tweak the name or definition of the Orchestration & Automation domain? I'm open to ideas if you think that name isn't right, although I still see value in the automation platform living in a distinct domain between Control Plane and Infra. |
|
ok lets work out the responsibilities of Service Providers, i think we differ a bit here. Put my thoughts:
For example the VM Service Provider might use some VMW APIs or maybe some OCP Virt APIs or maybe even an ansible collection that makes it easy to switch between them. But the Service Provider API, which is used by the DCM Control Plane, should hide VMW/OCP-virt/Ansible specific details. For clarity, I don't think any Service Providers exist today. Mostly they will be implemented as adapters between existing infra management technology and the interoperability requirements of the DCM Control Plane. |
|
Maybe the difference is coming from the fact that I think there is another domain (unlisted) called "Data Center" which only contains the actual physical resources. I see in your taxonomy you have: I am thinking that the physical data center (racks, cpus, storage arrays, etc...) is in a different domain than the virtual assets (VMs,db tables, etc...). I am thinking the virtual assets are managed by Service Providers and live in the Composable Infra domain. |
|
Right, I was just talking with Chris and figured out that I was misunderstanding the definition of Service Providers. I incorrectly thought Service Providers were the just the lower level infrastructure and platforms that provision Resources, but I understand now that they are what are doing the Naturalization and Denaturalization, that is, enabling the interface from the Unified API/data model down to the goodies in the Data Center that provision Resources to ultimately realize the fulfillment of Service requests. So, having cleared that up, I now believe that Service Providers actually live in the Orchestration & Automation domain. Now we need a new term and definition for what I was calling Service Providers before which is also what Chris has been calling Widgets. How does this sound?... Infrastructure Platform Infrastructure Platforms refer to the Data Center infrastructure and platforms that provide Resources within the Composable Infrastructure domain. An Infrastructure Platform may be physical hardware or a software platform, for example, bare metal servers, network gear, storage arrays, virtualization platforms, Kubernetes clusters, etc. Each Infrastructure Platform defines the physical and/or virtual Resources it supports, such as compute, storage, network, VM instances, Kubernetes cluster objects, etc. An Infrastructure Platform provides an API used by a Service Provider to provision, decommission, make state changes to and enable discovery of the Resources it supports. Infrastructure Platforms must also expose event logging and metrics collection to the Governance & FinOps domain required to support monitoring, observability and billing. |
|
|
||
| The top-level domains are defined as follows: | ||
|
|
||
| 1. **Developer Experience** - This domain delivers the Unified interface that Developers and Application Owners use to request and manage Services from DCM. It includes the Service Catalog listing the available Services. Multi-tenancy is experienced at this level. |
There was a problem hiding this comment.
question: where can I find more info about multi-tenancy? as far as I can tell we haven't discussed it yet.
comment: for this domain should we consider SDK and UI?
There was a problem hiding this comment.
@pkliczewski @swapdisk - this discuss this more on wednesday.
There was a problem hiding this comment.
@croadfeldt - Need to address multi-tenancy implementation in the architecture and implementation.
There was a problem hiding this comment.
@croadfeldt - Take a look at this in terms of implementing multi-tenancy: osac-project/fulfillment-service#126
There was a problem hiding this comment.
From a taxonomy perspective, I think all we need to do is agree multi-tenancy is required and where in the domain stack it should exist. How exactly multi-tenancy should be implemented is beyond the scope of this PR.
|
|
||
| 1. **Developer Experience** - This domain delivers the Unified interface that Developers and Application Owners use to request and manage Services from DCM. It includes the Service Catalog listing the available Services. Multi-tenancy is experienced at this level. | ||
| 2. **Control Plane** - This is the central nervous system of DCM. It maintains the Unified API and Data Model including the inventory of Fulfilled Services. It also enforces authentication and role-based access control. | ||
| 3. **Orchestration & Automation** - This domain contains the code and workflows for provisioning and managing the Resources required to Fulfill Sefvice request. It provides the engines that execute requests from the Control Plane and translates those into API calls supported by specific Infrastructure Platforms hosted under the Composable Infrastructure domain. |
There was a problem hiding this comment.
question: how should we think about orchestration and automation in relation to service provider?
There was a problem hiding this comment.
@pkliczewski - Nick Meyer is working on this as well. But the goal is to have emergent orchestration based on data, policy, dependency and service providers. We should talk about this on Wednesday as well.
There was a problem hiding this comment.
The Orchestration & Automation domain has been renamed Resource domain to align with the new draw-io picture Adam and Sal made:
As far as under which domain Service Providers live, I'm still a little unclear on that. In the latest core model picture from Chris' miro, it looks like they are almost external to DCM:
My best guess is Service Providers fall under the Resource domain.
|
|
||
| The vocabulary we use to define this architecture demands precision and clarity. By allowing a term to have a distinct, contextual definition within each architectural domain, we can eliminating ambiguity even when using the same words. | ||
|
|
||
| For example, the "resource" a developer requests for their application is a logical entity, for example, a "Web App Service." However, the "resource" that the Infrastructure Operations team manages is a physical asset like a specific VM with a particular IP address. By providing distinct definitions for the same term in each domain, we avoid confusion as we navigate through the architectural domains. |
There was a problem hiding this comment.
I see that we have definition of the application below. I think we are missing a relationship between an application and a "resource" or group of "resources" which are a result of a developer sending a request.
|
|
||
| ### Data Model | ||
|
|
||
| The Data Model defines the scope of Unified data within the Control Plane. The Data Model includes static data as well as event data such as the history of Fulfilled Service requests and time series data such as Workload usage metrics. Data Model does not mean database, but rather is a broader term meaning the scope of any data supporting the DCM architecture. While static data may be stored using a traditional database service, data from Service requests is better sent to an event logging service and usage metrics would be more suitably captured and stored in a time series data platform. |
There was a problem hiding this comment.
question: should we expose metrics data via the api or collect metrics from different service providers?
|
|
||
| The Data Model defines the scope of Unified data within the Control Plane. The Data Model includes static data as well as event data such as the history of Fulfilled Service requests and time series data such as Workload usage metrics. Data Model does not mean database, but rather is a broader term meaning the scope of any data supporting the DCM architecture. While static data may be stored using a traditional database service, data from Service requests is better sent to an event logging service and usage metrics would be more suitably captured and stored in a time series data platform. | ||
|
|
||
| The Data Model also includes code, for example, the YAML data that defines the Resources and other parameters of Service Catalog item. Any data that is maintained as code would be managed and stored using Git repositories. |
There was a problem hiding this comment.
question: what yaml data do we have in mind? is it DCM deployment manifests or something else?
|
|
||
| ### Region | ||
|
|
||
| A Region is a large, geographically distinct area that hosts Resources. It can be a single Data Center or a group of Data Centers in close proximity. Regions should be designed as physically and logically independent from other Regions. This design is crucial for disaster recovery and business continuity, so a failure in one region doesn't affect another. |
There was a problem hiding this comment.
question: is geo distribution relevant to DCM control plane or is it service provider specific or both?
There was a problem hiding this comment.
I think both and also spills over to placement policy managed under the Governance & FinOps domain.
|
|
||
| ### Service | ||
|
|
||
| In the absence of further context, a Service is the specific capability defined by a Service Catalog item within the Developer Experience domain. Services provide the underlying Resources that Applications rely on. They are often designed to be reusable for multiple Applications. A Service could simply define a single underlying Resource, but more capable Services will define a configuration of many connected Resources. Compound Services may be defined by composing a number of simple Services under a single Service Catalog item. |
There was a problem hiding this comment.
it is unclear to me what is the relationship between services, applications and resources as well as which domain (layer) they are part of.
There was a problem hiding this comment.
Same here. I need to completely redo this based on what I learned hanging out with Chris all week in Raleigh. Specifically quoting Chris, "Service Catalog items are defined by Service Providers" whereas I had previously thought they were much more loosely coupled.
@croadfeldt - correct me if I'm misquoting you above"
|
|
||
| ### Service Provider | ||
|
|
||
| Service Providers support provisioning and management of Resources required to Fulfill Service requests from the Control Plane in a way that is agnostic to the implementations of different Infrastructure Platforms. Service Providers implement the Naturalization required to translate Unified API requests from the Control Plane into an Infrastructure Platform’s native API and the Denaturalization required to translate native API responses into responses that comply with the Unified API. |
There was a problem hiding this comment.
question: what is the role of orchestration and automation domain (layer) in relation to control plane and a service provider? it seems like it is missing here
There was a problem hiding this comment.
It's not you; it's me. Service Providers are the glue that connects the Control Plane and Resource domains.
|
|
||
| In the absence of further context, a User is the human consumer of cloud computing resources and services experiencing DCM from the Developer Experience domain. Of course, other humans "use" DCM within the other domains, so consider using Developer or Application Owner instead. | ||
|
|
||
| To avoid confusion, avoid using User when referring to humans acting within the the other DCM domains. Consider more specific persona alternative terms such as Platform Engineer, Infrastructure Operations, Policy Owner, Risk and Compliance Manager, etc. |
There was a problem hiding this comment.
question: should we define each persona and clarify how they interact with DCM?
There was a problem hiding this comment.
I'm hesitant define every possible persona as that gets into specifics that will vary for different organizations using the architecture. The points I'm trying to make here are just (1) the User is the only ever the human acting within the Application domain above the Control Plane and (2) to avoid confusion by never referring to other actors or identities as the User.
|
|
||
| ### Zone | ||
|
|
||
| Also known as an Availability Zone, a Zone is an isolated group of Resources within a Region. Each Region consists of one or more Zones. A Zone is a separate physical facility with its own power, cooling, and networking, ensuring that a problem in one Zone, like a power outage or a localized fire, doesn't impact other Zones in the same Region. By deploying Resources across multiple Zones within a single Region, high availability and fault tolerance can be achieved for Services and Applications. Zones are defined by Infrastructure Platforms and can be namespaces, clusters or anything that results in its Resources being isolated as spelled out above. |
There was a problem hiding this comment.
question: should zones be consider in a context of service providers and exposed to users in the service catalog?
There was a problem hiding this comment.
Yes, they would have to be. Remember: "Service Catalog items are defined by Service Providers."
Seeleyab
left a comment
Seeleyab
left a comment
There was a problem hiding this comment.
Can we change this to the new Draw.io based picture?
|
|
||
| The top-level domains are defined as follows: | ||
|
|
||
| 1. **Developer Experience** - This domain focuses on the unified interface that developers and application owners use to interact with DCM. It includes the self-service catalog, templates, and all user-facing documentation. This is where the DCM abstractions are consumed. Multi-tenancy is experienced at this level. |
There was a problem hiding this comment.
Should have a level 0 that is Value? I want this to be value stream aligned and above the user is the reason the user even exists.
make Value the "zero domain"
Yes, see as refactored with my commit bfc80df.
Yes, same commit adds the Value "zero domain" defined thusly: recognizes the business value achieved through the abstract capabilities enabled by the end-to-end architecture, for example, improved security and compliance, accelerated software deployments and cost saving thanks to reduced operational toil. |
|
I think my latest commit resolves the reviewed points. In any case, this PR has been open for too long. Let's get this merged and we can continue to iterate as needed. |
5 participants
Work in progress for #31.
Opening this a draft PR for now. I still need to add more terms to the vocabulary, but I have a few there just to set the tone. Looking for feedback on the top-level taxonomy and general layout.