-
Notifications
You must be signed in to change notification settings - Fork 1
Understanding Clusters
Under each workspace, you can create one or more clusters of your choice of the supported products. A cluster is created by running the create_cluster
command which by default adds two (2) members. You can add more members by using the add_member
command.
The create_cluster
command creates a cluster for the specified product. If the -product
option is not specified, then it creates a cluster for the product in the current cluster context. For example, if your current cluster context is Geode, then running create_cluster
will create another Geode cluster. The supported products are coherence
, geode
, hazelcast
, kafka
, redis
, spark
, gemfire
, hadoop
, jet
, mosquitto
, and snappydata
.
# Create a Geode cluster with the default name 'mygeode'.
create_cluster -product geode
# Switch cluster context and create another Geode cluster named 'mygeode2'
switch_cluster mygeode
create_cluster -cluster mygeode2
# Create a Hazelcast cluster by specifying the product
create_cluster -product hazelcast -cluster myhz2
Interally, PadoGrid sets the following environment variables when you switch clusters. They determine the current cluster context.
Environment Variables | Descriptions | Default |
---|---|---|
PRODUCT | Product name. Valid values are coherence , geode , hazelcast , kafka , redis , spark , gemfire , hadoop , jet , mosquitto , and snappydata . |
N/A |
CLUSTER | Cluster name. If the -cluster is not specified then the PadoGrid commands default to this value. |
Coherence: mycoherence , ComputeDB: mysnappy , Geode: mygeode , GemFire: mygemfire , Hazelcast Platform/IMDG: myhz , Hazelcast Jet: myjet , Mosquitto: mymosquitto , Redis: myredis , SnappyData: mysnappy , Spark: myspark
|
POD | Pod name. If the -pod option is not specified then the PadoGrid commands default to this value. |
local |
Setting cluster specific environment variables such as JAVA_OPTS
is done by adding them in <cluster-dir>/setenv.sh
, which is sourced in by all PadoGrid commands after the workspace setenv.sh
is sourced in. This means you can override workspace-level cluster settings from this file.
# Add or override cluster specific settings in the following file:
vi $PADOGRID_WORKSPACE/clusters/$CLUSTER/bin_sh/setenv.sh
# Or change directory and edit from there:
cd_cluster
vi bin_sh/setenv.sh
In addition to JAVA_OPTS
, for SnappyData clusters, the following environment variables are available for overriding the default bootstrap options or including additional bootstrap options. You can set them in the cluster's setenv.sh
file.
Variable | For | Description |
---|---|---|
LOCATOR_OPTS | Locators | An argument list of locator options. |
LEAD_OPTS | Leaders | An argument list of leader options. |
MEMBER_OPTS | Servers | An argument list of server (member) options. |
By default, PadoGrid runs each member using the same default product commands and server classes as shown below.
"$JAVA" -server -showversion $JAVA_OPTS com.tangosol.net.DefaultCacheServer
KAFKA_OPTS=... kafka-server-start -daemon ...
gfsh start locator $GFSH_OPTS $JAVA_OPTS
gfsh start server $GFSH_OPTS $JAVA_OPTS --classpath=$CLASSPATH
# namenode
HADOOP_CONF_DIR=... HDFS_NAMENODE_OPTS=... hdfs --daemon start namenode
# datanode
HDFS_DATANODE_OPTS=... hdfs --config $HADOOP_CONF_DIR --daemon start datanode
"$JAVA" -server $JAVA_OPTS com.hazelcast.core.server.HazelcastMemberStarter
"$JAVA" -server $JAVA_OPTS com.hazelcast.core.server.StartServer
"$JAVA" -server $JAVA_OPTS com.hazelcast.core.server.HazelcastMemberStarter
"$JAVA" -server $JAVA_OPTS com.hazelcast.jet.server.StartServer
"$JAVA" -server $JAVA_OPTS com.hazelcast.jet.server.JetMemberStarter
KAFKA_OPTS=... kafka-server-start.sh -daemon ...
mosquitto -d -p ... -c $CONFIG_FILE
redis-server $CONFIG_FILE --port ...
redis-cli --cluster add-node ...
snappy locator start $SNAPPY_OPTS $JAVA_OPTS
snappy server start $SNAPPY_OPTS $JAVA_OPTS -classpath=$CLASSPATH
snappy leader start $SNAPPY_OPTS $JAVA_OPTS -classpath=$CLASSPATH
SPARK_WORKER_OPTS=... start-daemon.sh start ...
where
GFSH_OPTS
, HADOOP_CONF_DIR
, HDFS_NAMENODE_OPTS
, HDFS_DATANODE_OPTS
, JAVA_OPTS
, KAFKA_OPTS
, SNAPPY_OPTS
, and SPARK_WORKER_OPTS
are bootstrap options provided in the workspace and cluster setenv.sh
files. Note that there are additional Hadoop, Kafka, and Spark specific environment variables not shown in the above example.
You can also use a custom class to start members by creating a script and setting the RUN_SCRIPT
environment variable to the script's path in the setenv.sh
file. You would normally place the script in the same cluster's bin_sh/
directory where the setenv.sh
is located. The custom script inherits all the PadoGrid environment variables.
-
JAVA
- Java executable -
JAVA_OPTS
- Java options -
CLASSPATH
- Java class paths
The script can use the above environment variables to start the application as shown in the example below. Note that the member automatically runs in the background.
#!/usr/bin/env bash
"$JAVA" -server $JAVA_OPTS com.myco.mycluster.MyServer
You would set the script path in the cluster setenv.sh
file as follows:
# In setenv.sh
RUN_SCRIPT=$CLUSTER_DIR/bin_sh/your_script
PadoGrid Manual
Overview
- Home
- PadoGrid in 5 Minutes
- Quick Start
- Introduction
- Bundle Catalogs
- Building PadoGrid
- Supported Data Grid Products and Downloads
- PadoGrid Components
- Installing PadoGrid
- Root Workspaces Environments (RWEs)
- Initializing PadoGrid
- Bash Auto-Completion
- Viewing PadoGrid Summaries
- Updating Products
- Upgrading PadoGrid
- Migrating Workspaces
- PadoGrid Pods
- Kubernetes
- Docker
- Apps
- Software List
Operations
- Workspace Lifecycle Management
- Creating RWE
- Creating Workspace and Starting Cluster
- Managing Workspaces
- Understanding Workspaces
- Understanding Clusters
- Running Clusters
- Default Port Numbers
- Running Clusters Independent of PadoGrid
- Running Apps
- Understanding Groups
- Running Groups
- Understanding Bundles
- User Bundle Repos
- Using Bundle Templates
- Bundle Repo Guidelines
- User Bundle Catalogs
- Private Bundle Repos
- Gitea Repos
- Running Bundles in Container
- PadoGrid Addon Jars
- Understanding PadoGrid Pods
- Tested Vagrant Boxes
- VM-Enabled Pods
- Multitenancy
- Multitenancy Best Practices
- PadoGrid Configuration Files
Tools
Platforms
Clouds
Pado
Geode/GemFire
- Geode CLASSPATH
- Geode Kubernetes
- Geode Minikube
- Geode Minikube on WSL
- Geode Docker Compose
- Geode Grafana App
- Geode
perf_test
App - Geode WAN Example
- Geode Workspaces on VMs
- Geode on AWS EC2
- Reactivating Geode Workspaces on AWS EC2
Hazelcast/Jet
- Hazelcast CLASSPATH
- Creating Jet Workspace
- Configuring Hazelcast Addon
- HQL Query
- Hazelcast Kubernetes
- Hazelcast GKE
- Hazelcast Minikube
- Hazelcast Minikube on WSL
- Hazelcast Minishift/CDK
- Hazelcast OpenShift
- Hazelcast Docker Compose
- Hazelcast Desktop App
- Hazelcast Grafana App
- Hazelcast
jet_demo
App - Hazelcast
perf_test
App - Hazelcast WAN Example
- Hazelcast Workspaces on VMs
- Hazelcast on AWS EC2
- Reactivating Hazelcast Workspaces on AWS EC2
ComputeDB/SnappyData
Coherence
Hadoop
Kafka/Confluent
Mosquitto
- Mosquitto CLASSPATH
- Mosquitto Overview
- Installing/Building Mosquitto
- Clustering MQTT
- Cluster Archetypes
- Enabling Mosquitto SSL/TLS
- Mosquitto Docker Compose
- MQTT perf_test App
Redis
Spark