Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
What we currently think of as providers is directly mapped to our external management systems (EMS). However, as we start to expand our concept of managing systems, this mapping is limiting. For example, with Openstack, we currently have a cloud manager, but injecting an infrastructure manager for the same Openstack installation gets complicated. So, in order to handle this, I am proposing a change in what we know as providers.
Providers and Managers
From a terminology perspective, EMS will now map to what I am calling a Manager. Above the manager we will introduce a new layer called Provider. Thus, a provider will have one or more managers.
This is the current modeling for some existing EMS types.
This is the proposed modeling for how these existing EMS types will change.
As you can see from the proposed model, for existing "single manager" systems like VMware, nothing really changes except that a layer is introduced above. However, our existing Amazon region support is actually brought together under a single provider, while still keeping the managers separate. An Amazon provider essentially has many cloud managers.
Having this extra layer opens up possibilities that were difficult previously, such as,
Another term I am introducing is what's known as a managed entity. In the past, the team has referred to these as "CIs", amongst other things. However, when new provider types are introduced, this concept becomes murky. Managed entities are the items of importance to a particular type of manager. For example,
One important concept of managed entities is the role they play with respect to the rest of ManageIQ. Managed entities are seen in the UI, are reportable, RBAC-ed, amongst other things. Code-wise, we may want to take this set of features, and move them into a shared mixin.
Looking at Foreman from a particular perspective you can see two distinct managers (even though they are the same endpoint). There is a provisioning manager which is responsible for understanding how to provision any system. Once the system has been provisioned, it is managed by a configuration manager. Thus, a Foreman provider could be modeled as follows.
Keeping the configuration manager separate opens up the possibility to support other configuration management systems like Chef or Puppet directly. Keeping the provisioning manager separate opens up the possibility of consolidating our home grown provisioning via PXE into this new set of models, or possibly use other provisioning systems.
Openstack infrastructure via Triple O can be supported by adding a new infrastructure manager.
As you may have noticed from the two images above, when a manager collects inventory it keeps a separate "copy" of an entity even though it may represent the "same" underlying thing. One of the challenges of having multiple managers of the same entities is overlap, and the question of who "wins". When you dig deep you have to choose on a column-by-column basis, which gets very difficult to deal with at the inventory collection level.
For example, if one manager reports a system's OS as "Windows 7", and another manager reports it as "Windows 7 SP1", you'd probably want the latter to win, however, the second manager may have less detail on other information that you can obtain from the first manager. A second, more difficult example, is storing a unique id from each manager's perspective. Each manager will likely have different values, but there is only one column. You could add a second column, but what happens when you have 3 managers?
So, the clearest way to deal with this problem is to silo the inventory collection. Each manager will retain their own copy of their managed entities. A separate relationship can then be made between these entities to show they are the "same" (what I labeled as "related" in the images). It is through these "sideways" relationships that the interesting connections can be made. Reporting, the UI, and other presentations will no doubt be challenging, but I believe it is easier to combine data at these higher layers than at the lowest data layer during collection.
In the Foreman example, we see a managed entity called a managed system. From Foreman's perspective, they don't know if the system they are managing is bare metal or virtual, nor do they care. Adding this new table has exposed something that we've known about for some time, but have not done. This new type needs many of the exact same attributes and relationships that are attached to Vm and Host. To consolidate this information, we should create a new type called ComputerSystem that consolidates this common information.
This is the current layout. If we add a ManagedSystem type, we'll have to duplicate all of these relationships and properties. This will get very messy, and duplicates the same information. Note that although there are only 5 relationships displayed here, there are closer to 30 relationships in the code.
Here is the new proposed ComputerSystem. As you can see this cleanly consolidates all common elements.
I've been asked if we need that Vm/Host/ManagedSystem layer at all, and I believe we do. We have already struggled with VM and Template living in the same table, so consolidating even more types into a single table will only complicate the matter further. Operations and attributes that are more related to the managed entity and not the computer system would live at the higher layer, as well as identifying information from the manager. In addition, we may be able to further split Vm and Template more easily when this new model is introduced.
Actual implementation of this can be done in phases. I expect that new work, like the Foreman integration, will add the ComputerSystem concept, and the existing vm and hosts table can be rearranged later.
Connections / Endpoints
Since each manager may have different connection endpoints, it makes sense that endpoints would be stored at the manager level. In fact, this is how it is done now for EMS records currently. However, when a provider has multiple managers that all share the same endpoint, then managing those endpoints at the manager level becomes a maintenance problem. If you change one, we have to change multiple records under the covers, or the user is required to enter the same endpoints over and over.
My proposal for endpoints is to allow them to exist at either the manager layer or the provider layer, and the decision about where to store it will be provider specific. For example, for Amazon, where there are multiple cloud managers that all share the same endpoint, it would live at the provider level, and calls to the
The delegation code to handle a single endpoint at the manager level could probably be abstracted into a mixin, so that all communication is done at the manager level, and it takes care of figuring out where to store the endpoint details.
This issue has been automatically marked as stale because it has not been updated for at least 6 months.
If you can still reproduce this issue on the current release or on
Thank you for all your contributions!