Azure membership scheme provides features for automatically discovering WSO2 Carbon server clusters on Azure (Read the blog here).
Once a Carbon server starts it will query the IP addresses in the given cluster via Azure API or Azure SDK using the given Azure services. The name of the resource group where the virtual machines are assigned should be provided. Thereafter, Hazelcast network configuration will be updated with the above IP addresses. As a result, the Hazelcast instance will get connected to all the other members in the cluster. In addition, once a new member is added to the cluster, all other members will get connected to the new member.
Azure Membership Scheme is compatible with WSO2 products that are based on Carbon 4.6.1 and onwards. (To determine the Carbon version of a particular product, please refer to the WSO2 Release Matrix).
Following two approaches can be used for discovering Azure IP addresses.
- Using the Azure REST API (Recommended)
An access token will be acquired from Azure using the provided client credentials and using the access token, Azure REST API is called. IP addresses of the virtual machines are parsed from the response and provided to the Hazelcast network configuration.
- Using the Azure SDK
Azure NetworkManager class from the Java SDK is authenticated with the provided client credentials, and it is used to query IP addresses of the virtual machines from the resource group. Then the IP addresses will be provided to the Hazelcast network configuration.
-
Run the following command from the
azure-membership-scheme
directory.mvn clean install
-
Copy the following JAR file from
azure-membership-scheme/target
to the<carbon_home>/repository/components/lib
directory of the Carbon server.azure-membership-scheme-1.0.0.jar
-
Copy the following dependencies from
azure-membership-scheme/target/dependencies
to the<carbon_home>/repository/components/lib
directory of the Carbon server.azure-core-1.23.1.jar content-type-2.1.jar msal4j-1.11.0.jar oauth2-oidc-sdk-9.7.jar
-
Configure the membership scheme as shown in the
<carbon_home>/repository/conf/deployment.toml
file.[clustering] membership_scheme = "azure" local_member_host = "127.0.0.1" local_member_port = "4000" [clustering.properties] membershipSchemeClassName = "org.wso2.carbon.membership.scheme.azure.AzureMembershipScheme" AZURE_CLIENT_ID = "{{client-id}}" AZURE_CLIENT_SECRET = "{{client-secret}}" AZURE_TENANT = "{{tenant}}" AZURE_SUBSCRIPTION_ID = "{{subscription-id}}" AZURE_RESOURCE_GROUP = "{{resource-group}}" AZURE_API_ENDPOINT = "https://management.azure.com" AZURE_API_VERSION = "2021-03-01"
-
When the server is starting, logs related to the cluster initialization can be observed.
AZURE_CLIENT_ID
- Azure Client ID obtained by registering a client application in the Azure Active Directory tenant. The client app needs to have the necessary permissions assigned to perform the action Microsoft.Network/networkInterfaces/read ex:53ba6f2b-6d52-4f5c-8ae0-7adc20808854
AZURE_CLIENT_SECRET
- Azure Client Secret generated for the application, ex:NMubGVcDqkwwGnCs6fa01tqlkTisfUd4pBBYgcxxx=
AZURE_TENANT
- Azure Active Directory tenant name or tenant ID, ex:default
AZURE_SUBSCRIPTION_ID
- Azure Subscription ID, ex:53ba6f2b-6d52-4f5c-8ae0-7adc20808854
AZURE_RESOURCE_GROUP
- Azure Resource Group to discover IP addresses, ex:wso2cluster
AZURE_API_ENDPOINT
- Azure Resource Manager API Endpoint, ex:https://management.azure.com
AZURE_API_VERSION
- Azure API Version, ex:2021-03-01
-
Run the following command from the
azure-membership-scheme
directory.mvn clean install
-
Copy the following JAR file from
azure-membership-scheme/target
to the<carbon_home>/repository/components/lib
directory of the Carbon server.azure-membership-scheme-1.0.0.jar
-
Since the Azure SDK is using the Java service loader, and it is not OSGI compatible, we have used Apache Aries SPI Fly as a resolution. Run the p2-maven-plugin using the following command in the
azure-membership-scheme
directory in order to generate the required dependencies.mvn p2:site
Then copy the following files from
azure-membership-scheme/target/repository/plugins
to the<carbon_home>/repository/components/dropins
directory.com.azure.core_1.22.0.jar com.azure.core-http-okhttp_1.7.6.jar
You should also configure Apache SPI Fly to be used with the dynamic weaving bundle as explained here
-
Copy the following dependencies from
azure-membership-scheme/target/dependencies
to the<carbon_home>/repository/components/lib
directory of the Carbon server.azure-core-management-1.4.3.jar azure-identity-1.4.2.jar azure-resourcemanager-network-2.10.0.jar azure-resourcemanager-resources-2.10.0.jar content-type-2.1.jar jackson-dataformat-xml-2.13.0.jar jackson-datatype-jsr310-2.13.0.jar kotlin-stdlib-1.4.10.jar msal4j-1.11.0.jar oauth2-oidc-sdk-9.7.jar okhttp-4.9.2.jar okio-2.8.0.jar reactive-streams-1.0.3.jar reactor-core-3.4.12.jar
-
Configure the membership scheme as shown in the
<carbon_home>/repository/conf/deployment.toml
file.[clustering] membership_scheme = "azure" local_member_host = "127.0.0.1" local_member_port = "4000" [clustering.properties] membershipSchemeClassName = "org.wso2.carbon.membership.scheme.azure.AzureMembershipScheme" AZURE_CLIENT_ID = "{{client-id}}" AZURE_CLIENT_SECRET = "{{client-secret}}" AZURE_TENANT = "{{tenant}}" AZURE_SUBSCRIPTION_ID = "{{subscription-id}}" AZURE_RESOURCE_GROUP = "{{resource-group}}" USE_SDK = "true"
-
When the server is starting, logs related to the cluster initialization can be observed.
AZURE_CLIENT_ID
- Azure Client ID obtained by registering a client application in the Azure Active Directory tenant. The client app needs to have the necessary permissions assigned to perform the action Microsoft.Network/networkInterfaces/read ex:53ba6f2b-6d52-4f5c-8ae0-7adc20808854
AZURE_CLIENT_SECRET
- Azure Client Secret generated for the application, ex:NMubGVcDqkwwGnCs6fa01tqlkTisfUd4pBBYgcxxx=
AZURE_TENANT
- Azure Active Directory tenant name or tenant ID, ex:default
AZURE_SUBSCRIPTION_ID
- Azure Subscription ID, ex:53ba6f2b-6d52-4f5c-8ae0-7adc20808854
AZURE_RESOURCE_GROUP
- Azure Resource Group to discover IP addresses, ex:wso2cluster
USE_SDK
- Configure the membership scheme to either use Azure REST API (default) or use the Azure SDK for IP resolution, ex:true
. To use the Azure SDK, this value must be set totrue
.