Skip to content

Commit

Permalink
docs: metadata 2.0, object extensibility, message broker (#18)
Browse files Browse the repository at this point in the history
Contains also the object extensibility phase payload schemas, as well as the channel extension message envelope schemas. For now it makes sense to add them here, because they are not available anywhere else.

Signed-off-by: Ivan Iliev <iviliev@vmware.com>
  • Loading branch information
iiliev2 committed Feb 16, 2024
1 parent e09e450 commit a2bce39
Show file tree
Hide file tree
Showing 20 changed files with 2,648 additions and 2 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ All components of this repository are validated by VMware or its partners agains
## Documentation

Get familiar with Cloud Director Solution Add-Ons.
* [Cloud Director Extensibility Platform](documentation/extensibility-platform/extensibility-platform.md)
* [Cloud Director Extension SDK](https://developer.vmware.com/sdk/cloud-director-extension)
* [Service Provider Admin Guide for Solution Add-Ons](https://docs.vmware.com/en/VMware-Cloud-Director/10.4/VMware-Cloud-Director-Service-Provider-Admin-Portal-Guide/GUID-4F12C8F7-7CD3-44E8-9711-A5F43F8DCEB5.html)

Expand Down
22 changes: 20 additions & 2 deletions documentation/extensibility-platform/extensibility-platform.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ Through the UI Extensibility framework, developers can create custom plugins tha

More details for all UI Extensibility capabilities and tooling can be found [here](ui-plugins.md).

## [Message broker](message-broker.md)
The backbone of many of the extensibility framework technologies is a message bus. Extension backends communicate with-, or monitor various system processes and events. Currently, there are two message busses - one for the AMQP protocol, which is backed by a provider's own RabbitMQ server, and one for the MQTT protocol which is embedded in Cloud Director. AMQP is being slowly phased out in favour of MQTT and will eventually be completely removed.

## Notifications and Events
Notifications and Events are mechanisms that provide real-time information about activities, changes, and status updates within the VCD environment. These features allow Cloud Director Extensions to monitor and respond to events that occur in the cloud infrastructure and enable various monitoring, alerting, automation and external system integration usecases.

Expand All @@ -24,8 +27,23 @@ Runtime Defined Entities (RDE) allow Extensions to create custom objects through

RDEs additionally provide advanced RBAC and Access Control for each type of object and their instances. These capabilities, combined with behaviors, are a great alternative to a traditional appliance backend that Extensions usually implement. More details for Runtime Defined Entities' management and all Behavior types will be provided in a detailed guide soon.

## Object Extensibility
Cloud Director allows Extensions to participate in, influence, or override the logic that VMware Cloud Director applies to core workflows like vApp/VM instantiation and placement. While [Blocking Tasks](https://docs.vmware.com/en/VMware-Cloud-Director/10.5/VMware-Cloud-Director-Service-Provider-Admin-Guide/GUID-B61D23C2-CCCF-4B33-8692-642C80A24193.html) allow Providers to control the progress of certain tasks in the system, Object Extensions can directly influence the outcome of certain core platform workflows. Extensions have to leverage MQTT behaviors to interact and plug into the core workflows currently exposed by Cloud Director.
## Blocking Tasks(Callouts)
Most built-in long-running operations in Cloud Director can be configured to block and wait to be resumed/cancelled/aborted via an external entity - either manually by a user, or by an application.

## [Object Extensibility](object-extensibility.md)
Cloud Director allows Extensions to participate in, influence, or override the logic that VMware Cloud Director applies
to core workflows like vApp/VM instantiation and placement. While [Blocking Tasks](https://docs.vmware.com/en/VMware-Cloud-Director/10.5/VMware-Cloud-Director-Service-Provider-Admin-Guide/GUID-B61D23C2-CCCF-4B33-8692-642C80A24193.html)
allow Providers to control the progress of certain tasks in the system, Object Extensions can directly influence the
outcome of certain core platform workflows. Extensions have to leverage MQTT behaviors to interact and plug into the
core workflows currently exposed by Cloud Director.

More details will be provided in a detailed guide soon.

## Object Metadata
Many core entity types support a notion of a soft schema extension to their main properties. Object metadata gives cloud operators and tenants a flexible way to associate user-defined properties (name=value pairs) with objects.

There are two separate pools of metadata, which have different management and capabilities:
1. Metadata of objects managed by the legacy Cloud Director api
2. [Metadata](object-metadata-2.md) of objects managed by the Cloud Director `cloudapi`

Objects which can be managed by both kinds of apis(legacy and cloudapi) will have the two separate pools. Metadata of the legacy api is considered deprecated and will receive only bug-fixes. Metadata of the cloudapi provides new features, with the primary goal being - improved _performance_, including a more powerful _search-by-metadata_ mechanism.
45 changes: 45 additions & 0 deletions documentation/extensibility-platform/message-broker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
Cloud Director uses a traditional message broker to support many of the extensibility platform capabilities. Starting from version 10.2, a message broker comes pre-bundled with the product. The protocol used for communication is [mqtt3.1.1](https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html).
> This page is still under development
# Establishing a connection
* web socket at `{vcd.host}/messaging/mqtt`

## Characteristics of the broker capabilities
* embedded brokers in each cell of a cell group are clustered
* brokers "forward" messages between themselves if necessary; for ex. when one client is connected on cell A and publishes to a topic, and there is another client connected on Cell B which is subscribed to the same topic, it will receive the message
* `QoS=0,1,2` are supported, and
* `QoS=1,2` - message loss may occur in case the broker stops
* `QoS=0` - message loss may occur even in case of network hiccups
* `clean start/session=true`
* `clean start=false` is not supported, because the broker does not keep any persisted state, meaning on crash or restart any durable session will be forgotten
* default message expiration is 5 minutes(unless the broker holding the message is stopped before then)
* cannot be used as a generic broker
* messages flow from Cloud Director outward(notifications), or
* request/response on strictly defined topics(extension services)
* authentication
* extension services -> user=extension service triplet, pass=extension token(get from RESTapi `/cloudapi/1.0.0/tokens`, `type=EXTENSION`)
* notifications -> user=user@organisation, pass=session jwt(perform a regular login at RESTapi to obtain jwt)

## Message Schema
* meta information describing how the payload is encoded(`payload` is a json/xml encoded string, etc); the `type` sometimes dictates the payload type(`BEHAVIOR_INVOCATION`=>`InvocationArguments`), but other times it is additionally specified by `headers.contentType`
* the reason for this structure is that payloads with varying schemas may float over the same topic; this means that your application may need to cope with messages which it is not interested in(and ignore them, rather than fail)
```json
{
"type": "EVENT|EXTERNAL_EVENT|TASK|EXTENSION_TASK|API_REQUEST|API_RESPONSE|BEHAVIOR_INVOCATION|BEHAVIOR_RESPONSE",
"headers": {},
"payload": ""
}
```
> **Note:**
>
> This pattern is used throughout the extensibility platform and various features may themselves encode their payloads in terms of meta info+encoded payload in the main `payload`; for ex. `BEHAVIOR_INVOCATION` wraps `HalfDuplexEnvelope` in its `payload`(`InvocationArguments.arguments`) as a json encoded string and the `HalfDuplexEnvelope` itself also encodes an [object extension request/response](object-extensibility.md#implementing-an-object-extension-backend) in its `payload`(where `payloadType` contains the name of the encoded payload schema)
## Configuring a client
* handling connection hiccups(subscriptions must be re-established "manually")
* handling password expiration(token, jwt, etc)
* working around known bugs of popular clients(paho)

## Best practices
* mission critical applications integrating _notifications_ should have a mechanism to periodically check for missed events and should not solely rely on `QoS`(or `clean start`)
* extension developers must find balance between the frequency of the check and the `QoS` level that is used; the higher the `QoS` the less frequent the check may need to be; for most situations `QoS=1` should be good enough, unless the check is very expensive to make; `QoS=2` may be used as last resort, as that is very taxing on the broker and may lower the overall performance of Cloud Director itself
* `QoS` should be viewed as levels of guarantee that a message transfer between broker and client will succeed, but it does not directly guarantee a message transfer between a `publisher` and a `subscriber`
* applications may need to implement additional logic to cope with duplicated messages for `QoS=1` (at least once)
Loading

0 comments on commit a2bce39

Please sign in to comment.