The Infrastructure Manager supports the definition of Cloud topologies using OASIS TOSCA Simple Profile in YAML Version 1.0.
The TOSCA support has been developed under de framework of the INDIGO DataCloud EU project. You can see some input examples at https://github.com/indigo-dc/tosca-types/tree/master/examples.
This TOSCA file describes a cloud topology with 2 VM with at least 2 CPUs and 2 GB of RAM and 40 GB of root disk, connected with a public IP, using an Ubuntu 20.04 image. As outputs, the TOSCA file will return the public IP of the VM and the SSH credentials to access it:
tosca_definitions_version: tosca_simple_yaml_1_0
imports:
- indigo_custom_types: https://raw.githubusercontent.com/indigo-dc/tosca-types/master/custom_types.yaml
topology_template:
node_templates:
simple_node:
type: tosca.nodes.Compute
scalable:
properties:
count: 2
capabilities:
endpoint:
properties:
network_name: PUBLIC
host:
properties:
num_cpus: 2
mem_size: 2 GB
disk_size: 40 GB
os:
properties:
type: linux
distribution: ubuntu
version: 22.04
outputs:
node_ip:
value: { get_attribute: [ simple_node, public_address, 0 ] }
node_creds:
value: { get_attribute: [ simple_node, endpoint, credential, 0 ] }
As in RADL you can set an specific URI identifying the VMI to use in the VM. The URI format is the same used in RADL (radl_system
). In this case the type must be changed to tosca.nodes.indigo.Compute
(the Compute normative type does not support the os image
property), and the image property must be added in the os
capability:
...
simple_node:
type: tosca.nodes.indigo.Compute
capabilities:
endpoint:
properties:
network_name: PUBLIC
host:
properties:
num_cpus: 2
mem_size: 2 GB
os:
properties:
image: one://someserver.com/123
...
The tosca.nodes.indigo.Compute
custom type add a new set of advanced features to the host properties, enabling the request of GPUs and Intel SGX CPU support in the compute node:
...
simple_node:
type: tosca.nodes.indigo.Compute
capabilities:
host:
properties:
num_cpus: 2
mem_size: 2 GB
num_gpus: 1
gpu_vendor: nvidia
gpu_model: Tesla V100
sgx: false
...
The easiest way to specify network requirements of the Compute node is sing the endpoint capability properties. For example the following example the compute node requests for a public IP:
...
simple_node:
type: tosca.nodes.Compute
capabilities:
endpoint:
properties:
network_name: PUBLIC
...
Possible values of the network_name
endpoint property:
- PRIVATE: The Compute node does not requires a public IP. This is the default behavior if no endpoint capability is defined.
- PUBLIC: The Compute node requires a public IP.
- Network provider ID: As the provider_id network property in RADL It defines the name of the network in a specific Cloud provider (see
_radl_network
):
Furthermore the endpoint capability has a set of additional properties to set the DNS name of the node or the set of ports to be externally accesible:
...
capabilities:
endpoint:
properties:
dns_name: slurmserver
network_name: PUBLIC
ports:
http_port:
protocol: tcp
source: 80
remote_cidr: 0.0.0.0/0 # optional
...
In case that you need a more detailed definition of the networks, you can use the tosca.nodes.network.Network
and tosca.nodes.network.Port
TOSCA normative types. In this way you can define the set of networks needed in your topology using the ports to link the networks with the Compute nodes:
...
pub_network:
type: tosca.nodes.network.Network
properties:
network_type: public
server_pub_port:
type: tosca.nodes.network.Port
properties:
order: 1
ip_address: X.X.X.X # optional to request specific IP
requirements:
- binding: simple_node
- link: pub_network
priv_network:
type: tosca.nodes.network.Network
properties:
network_type: private
server_port:
type: tosca.nodes.network.Port
properties:
order: 0
requirements:
- binding: simple_node
- link: priv_network
...
Custom defined Port type tosca.nodes.indigo.network.Port
has a set of additional properties:
- dns_name: DNS name to assing to the network interface.
- additional_ip: (OpenStack specific) Additional IP to be allowed to the network interface.
- additional_dns_names: Additional DNS names.
IM enable to use Ansible playbooks as implementation scripts. Furthermore it enables to specify Ansible roles (tosca.artifacts.AnsibleGalaxy.role
) and collections (tosca.artifacts.AnsibleGalaxy.collections
) to be installed and used in the playbooks:
...
software:
type: tosca.nodes.SoftwareComponent
artifacts:
docker_role:
file: grycap.docker
type: tosca.artifacts.AnsibleGalaxy.role
requirements:
- host: simple_node
interfaces:
Standard:
configure:
implementation: https://raw.githubusercontent.com/grycap/ec3/tosca/tosca/artifacts/dummy.yml
inputs:
some_input: { get_input: some_input }
...
IM enables the definition of BlockStorage volumes to be attached to the compute nodes. In this example we can see how to define a volume of 10GB to be attached to the compute node and mounted in the path /mnt/disk. The device parameter is optional and it is only needed in some cloud providers, in general is better not to add it:
...
simple_node:
type: tosca.nodes.Compute
...
requirements:
- local_storage:
node: my_storage
relationship:
type: AttachesTo
properties:
location: /mnt/disk
device: hdb # optional
my_storage:
type: tosca.nodes.BlockStorage
properties:
size: 10GB
...
IM enables the definition of the specific cloud provider where the Compute nodes will be deployed in an hybrid deployment. For example, in the following code we assume that we have defined three computes nodes (compute_one, compute_two and compute_three). We can create a placement group with two of them (compute_one and compute_two) and then set a placement policy with a cloud_id (that must be defined in the auth-file
), and create a second placement policy where we can set a different cloud provider and, optionally, an availability zone:
...
groups:
my_placement_group:
type: tosca.groups.Root
members: [ compute_one, compute_two ]
policies:
- deploy_group_on_cloudid:
type: tosca.policies.Placement
properties: { cloud_id: cloudid1 }
targets: [ my_placement_group ]
- deploy_on_cloudid:
type: tosca.policies.Placement
properties: { cloud_id: cloudid2, availability_zone: some_zone }
targets: [ compute_three ]
...
IM also enables the definition of container applications to be deployed in a Kubernetes cluster. In the following example we can see how to define a container application (IM) that uses a ConfigMap for a configuration file. The IM application is connected with a MySQL backend using the IM_DATA_DB
environment variable. The MySQL container is defined with a Persistent Volume Claim (PVC) of 10GB. Furthermore the IM application specifies an endpoint to be published that will result in the creation of a Kubernetes Ingress.
...
node_templates:
- im_container:
type: tosca.nodes.Container.Application.Docker properties: environment: IM_DATA_DB: concat: - "mysql://root:" - { get_input: mysql_root_password } - "@" - { get_attribute: [ mysql_container, endpoints, 0 ] } - "/im-db" requirements: - host: im_runtime artifacts: my_image: file: grycap/im type: tosca.artifacts.Deployment.Image.Container.Docker my_config_map: deploy_path: /etc/im/im.cfg file: https://raw.githubusercontent.com/grycap/im/master/etc/im.cfg type: tosca.artifacts.File properties: # when the content is not provided, the file is downloaded from the URL # otherwise, the file is ignored # If the content is base64 encoded, it is assumed to be a K8s Secret content: | [im] REST_API = True
# The properties of the runtime to host the container im_runtime: type: tosca.nodes.Container.Runtime.Docker capabilities: host: properties: num_cpus: 0.5 mem_size: 1 GB publish_ports: - protocol: tcp target: 8800 source: 30880 endpoint: https://im.domain.com/im
# The MYSQL container based on official MySQL image in Docker hub mysql_container: type: tosca.nodes.Container.Application.Docker properties: environment: MYSQL_ROOT_PASSWORD: { get_input: mysql_root_password } MYSQL_DATABASE: "im-db" requirements: - host: mysql_runtime artifacts: my_image: file: mysql:8 type: tosca.artifacts.Deployment.Image.Container.Docker
# The properties of the runtime to host the container mysql_runtime: type: tosca.nodes.Container.Runtime.Docker capabilities: host: properties: num_cpus: 0.5 mem_size: 1 GB expose_ports: - protocol: tcp target: 3306 volumes: - "some_vol:/var/lib/mysql"
- some_vol:
type: tosca.nodes.BlockStorage properties: size: 10 GB # Set the PV name in this field # volume_id: "PV name"
- outputs:
- im_service_endpoint:
value: { get_attribute: [ im_container, endpoints, 0 ] }
The tosca.nodes.indigo.Compute
node type adds a new attribute named: ansible_output
. It is a map that has one element per each IM configuration step, so you can access it by name. The steps have the keyword tasks
that is also a map that has one element per ansible task. In this case it can bes accessed using the task name as defined in the playbook. Finally there is an output
keyword that returns the output of the task. In most of the cases the task is a debug
ansible task that shows anything you want to return.
In the following example the specified task was a debug ansible task that shows the value of a internal defined value:
...
outputs:
node_ip:
value: { get_attribute: [ front, ansible_output, lrms_front_end_front_conf_front, tasks, 'grycap.nomad : nomad_secret_id', output ] }