Skip to content

Remove Cassandra dependencies from universal/scalardl module #232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 6, 2020
Merged

Remove Cassandra dependencies from universal/scalardl module #232

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6499d6b
Rename variables in docker-compose.yml in universal/scalardl module
ymorimo Oct 29, 2020
a34e3b9
Rename variable names in universal/scalardl module
ymorimo Oct 30, 2020
0edf4d5
Rename variables related to Cassandra in scalardl modules
ymorimo Oct 31, 2020
63351b5
Rename replication_factor to cassandra_replication_factor
ymorimo Oct 31, 2020
e7e6e74
Update READMEs
ymorimo Nov 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions examples/aws/scalardl/example.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,20 @@ region = "ap-northeast-1"
base = "default" # bai, chiku, sho

scalardl = {
# resource_type = "t3.medium"
# resource_root_volume_size = "64"
# blue_resource_count = "3"
# blue_image_tag = "2.0.7"
# blue_image_name = "scalarlabs/scalar-ledger"
# blue_discoverable_by_envoy = "true"
# green_resource_count = "0"
# green_image_tag = "2.0.7"
# replication_factor = "3"
# green_image_name = "scalarlabs/scalar-ledger"
# green_discoverable_by_envoy = "false"
# enable_tdagent = "true"
# cassandra_username = ""
# cassandra_password = ""
# resource_type = "t3.medium"
# resource_root_volume_size = "64"
# blue_resource_count = "3"
# blue_image_tag = "2.0.7"
# blue_image_name = "scalarlabs/scalar-ledger"
# blue_discoverable_by_envoy = "true"
# green_resource_count = "0"
# green_image_tag = "2.0.7"
# green_image_name = "scalarlabs/scalar-ledger"
# green_discoverable_by_envoy = "false"
# enable_tdagent = "true"
# database_username = "cassandra"
# database_password = "cassandra"
# cassandra_replication_factor = "3"
}

envoy = {
Expand Down
6 changes: 3 additions & 3 deletions examples/azure/scalardl/example.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ scalardl = {
# blue_enable_accelerated_networking = "false"
# green_resource_count = "0"
# green_image_tag = "2.0.7"
# replication_factor = "3"
# green_image_name = "scalarlabs/scalar-ledger"
# green_discoverable_by_envoy = "false"
# green_enable_accelerated_networking = "false"
# enable_tdagent = "true"
# cassandra_username = "cassandra"
# cassandra_password = "cassandra"
# database_username = "cassandra"
# database_password = "cassandra"
# cassandra_replication_factor = "3"
}

envoy = {
Expand Down
2 changes: 1 addition & 1 deletion modules/aws/monitor/locals.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ locals {
cassandra_resource_count = lookup(var.cassandra, "resource_count", 0)
scalardl_blue_resource_count = lookup(var.scalardl, "blue_resource_count", 0)
scalardl_green_resource_count = lookup(var.scalardl, "green_resource_count", 0)
scalardl_replication_factor = lookup(var.scalardl, "replication_factor", 0)
scalardl_replication_factor = lookup(var.scalardl, "cassandra_replication_factor", 0)
}

### default
Expand Down
14 changes: 10 additions & 4 deletions modules/aws/scalardl/cluster/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,28 @@
# Cluster Module of Scalar DL
The Cluster module deploys a Scalar DL cluster.

## Requirements

| Name | Version |
|------|---------|
| terraform | >= 0.12 |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:-----:|
|------|-------------|------|---------|:--------:|
| bastion_ip | The IP to bastion host used for provisioning | `any` | n/a | yes |
| cassandra_password | The password of cassandra cluster | `any` | n/a | yes |
| cassandra_username | The username of cassandra cluster | `any` | n/a | yes |
| cassandra_replication_factor | The replication factor for the Cassandra schema | `number` | `3` | no |
| custom_tags | The map of custom tags | `map(string)` | `{}` | no |
| database_password | The database password | `any` | n/a | yes |
| database_username | The database username | `any` | n/a | yes |
| enable_tdagent | A flag to install td-agent that forwards logs to the monitor host | `bool` | `true` | no |
| image_id | The image id to initiate | `any` | n/a | yes |
| internal_domain | Internal domain | `any` | n/a | yes |
| key_name | The key-pair name to assign to cluster resources | `any` | n/a | yes |
| network_dns | The ID for the internal DNS zone | `any` | n/a | yes |
| network_name | The name of the network resources: should be generated by provider/universal/name-generator | `any` | n/a | yes |
| private_key_path | The path to the private key for SSH access | `any` | n/a | yes |
| replication_factor | Set the replication factor for schema | `number` | `3` | no |
| resource_cluster_name | The name to assign the resource cluster | `any` | n/a | yes |
| resource_count | The number of resources to create | `any` | n/a | yes |
| resource_root_volume_size | The size of resource root volume size | `any` | n/a | yes |
Expand Down
13 changes: 7 additions & 6 deletions modules/aws/scalardl/cluster/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,11 @@ module "scalardl_provision" {
provision_count = var.resource_count
enable_tdagent = var.enable_tdagent

scalardl_image_name = var.scalardl_image_name
scalardl_image_tag = var.scalardl_image_tag
replication_factor = var.replication_factor
internal_domain = var.internal_domain
cassandra_username = var.cassandra_username
cassandra_password = var.cassandra_password
scalardl_image_name = var.scalardl_image_name
scalardl_image_tag = var.scalardl_image_tag
internal_domain = var.internal_domain
database_contact_points = "cassandra-lb.${var.internal_domain}" # TODO: add to variables
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's going to be done in another PR, can you create a story not to forget?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the variable is going to be added later.
Created as a subtask https://scalar-labs.atlassian.net/browse/DLT-7605
For database_contact_port variable: https://scalar-labs.atlassian.net/browse/DLT-7524

database_username = var.database_username
database_password = var.database_password
cassandra_replication_factor = var.cassandra_replication_factor
}
26 changes: 13 additions & 13 deletions modules/aws/scalardl/cluster/vars.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,6 @@ variable "scalardl_image_tag" {
description = "The docker image tag for Scalar DL"
}

variable "replication_factor" {
default = 3
description = "Set the replication factor for schema"
}

variable "enable_tdagent" {
default = true
description = "A flag to install td-agent that forwards logs to the monitor host"
Expand All @@ -78,16 +73,21 @@ variable "internal_domain" {
description = "Internal domain"
}

variable "custom_tags" {
type = map(string)
default = {}
description = "The map of custom tags"
variable "database_username" {
description = "The database username"
}

variable "cassandra_username" {
description = "The username of cassandra cluster"
variable "database_password" {
description = "The database password"
}

variable "cassandra_password" {
description = "The password of cassandra cluster"
variable "cassandra_replication_factor" {
default = 3
description = "The replication factor for the Cassandra schema"
}

variable "custom_tags" {
type = map(string)
default = {}
description = "The map of custom tags"
}
36 changes: 18 additions & 18 deletions modules/aws/scalardl/locals.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,24 @@ locals {
### default
locals {
scalardl_default = {
resource_type = "t3.medium"
resource_root_volume_size = 64
blue_resource_count = 3
blue_image_tag = "2.0.7"
blue_image_name = "scalarlabs/scalar-ledger"
blue_discoverable_by_envoy = true
green_resource_count = 0
green_image_tag = "2.0.7"
replication_factor = 3
green_image_name = "scalarlabs/scalar-ledger"
green_discoverable_by_envoy = false
target_port = 50051
privileged_target_port = 50052
listen_port = 50051
privileged_listen_port = 50052
enable_tdagent = true
cassandra_username = "cassandra"
cassandra_password = "cassandra"
resource_type = "t3.medium"
resource_root_volume_size = 64
blue_resource_count = 3
blue_image_tag = "2.0.7"
blue_image_name = "scalarlabs/scalar-ledger"
blue_discoverable_by_envoy = true
green_resource_count = 0
green_image_tag = "2.0.7"
green_image_name = "scalarlabs/scalar-ledger"
green_discoverable_by_envoy = false
target_port = 50051
privileged_target_port = 50052
listen_port = 50051
privileged_listen_port = 50052
enable_tdagent = true
database_username = "cassandra"
database_password = "cassandra"
cassandra_replication_factor = 3
}
}

Expand Down
2 changes: 1 addition & 1 deletion modules/aws/scalardl/output.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ output "scalardl_green_resource_count" {
}

output "scalardl_replication_factor" {
value = local.scalardl.replication_factor
value = local.scalardl.cassandra_replication_factor
description = "The replication factor for the schema of scalardl."
}

Expand Down
88 changes: 44 additions & 44 deletions modules/aws/scalardl/scalardl.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,55 +1,55 @@
module "scalardl_blue" {
source = "./cluster"

security_group_ids = aws_security_group.scalardl.*.id
bastion_ip = local.bastion_ip
network_name = local.network_name
resource_type = local.scalardl.resource_type
resource_count = local.scalardl.blue_resource_count
resource_cluster_name = "blue"
resource_root_volume_size = local.scalardl.resource_root_volume_size
triggers = local.triggers
private_key_path = local.private_key_path
user_name = local.user_name
subnet_ids = local.blue_subnet_ids
image_id = local.image_id
key_name = local.key_name
network_dns = local.network_dns
scalardl_image_name = local.scalardl.blue_image_name
scalardl_image_tag = local.scalardl.blue_image_tag
replication_factor = local.scalardl.replication_factor
enable_tdagent = local.scalardl.enable_tdagent
internal_domain = local.internal_domain
custom_tags = var.custom_tags
cassandra_username = local.scalardl.cassandra_username
cassandra_password = local.scalardl.cassandra_password
security_group_ids = aws_security_group.scalardl.*.id
bastion_ip = local.bastion_ip
network_name = local.network_name
resource_type = local.scalardl.resource_type
resource_count = local.scalardl.blue_resource_count
resource_cluster_name = "blue"
resource_root_volume_size = local.scalardl.resource_root_volume_size
triggers = local.triggers
private_key_path = local.private_key_path
user_name = local.user_name
subnet_ids = local.blue_subnet_ids
image_id = local.image_id
key_name = local.key_name
network_dns = local.network_dns
scalardl_image_name = local.scalardl.blue_image_name
scalardl_image_tag = local.scalardl.blue_image_tag
enable_tdagent = local.scalardl.enable_tdagent
internal_domain = local.internal_domain
database_username = local.scalardl.database_username
database_password = local.scalardl.database_password
cassandra_replication_factor = local.scalardl.cassandra_replication_factor
custom_tags = var.custom_tags
}

module "scalardl_green" {
source = "./cluster"

security_group_ids = aws_security_group.scalardl.*.id
bastion_ip = local.bastion_ip
network_name = local.network_name
resource_type = local.scalardl.resource_type
resource_count = local.scalardl.green_resource_count
resource_cluster_name = "green"
resource_root_volume_size = local.scalardl.resource_root_volume_size
triggers = local.triggers
private_key_path = local.private_key_path
user_name = local.user_name
subnet_ids = local.green_subnet_ids
image_id = local.image_id
key_name = local.key_name
network_dns = local.network_dns
scalardl_image_name = local.scalardl.green_image_name
scalardl_image_tag = local.scalardl.green_image_tag
replication_factor = local.scalardl.replication_factor
enable_tdagent = local.scalardl.enable_tdagent
internal_domain = local.internal_domain
custom_tags = var.custom_tags
cassandra_username = local.scalardl.cassandra_username
cassandra_password = local.scalardl.cassandra_password
security_group_ids = aws_security_group.scalardl.*.id
bastion_ip = local.bastion_ip
network_name = local.network_name
resource_type = local.scalardl.resource_type
resource_count = local.scalardl.green_resource_count
resource_cluster_name = "green"
resource_root_volume_size = local.scalardl.resource_root_volume_size
triggers = local.triggers
private_key_path = local.private_key_path
user_name = local.user_name
subnet_ids = local.green_subnet_ids
image_id = local.image_id
key_name = local.key_name
network_dns = local.network_dns
scalardl_image_name = local.scalardl.green_image_name
scalardl_image_tag = local.scalardl.green_image_tag
enable_tdagent = local.scalardl.enable_tdagent
internal_domain = local.internal_domain
database_username = local.scalardl.database_username
database_password = local.scalardl.database_password
cassandra_replication_factor = local.scalardl.cassandra_replication_factor
custom_tags = var.custom_tags
}

resource "aws_security_group" "scalardl" {
Expand Down
2 changes: 1 addition & 1 deletion modules/azure/monitor/locals.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ locals {
cassandra_resource_count = lookup(var.cassandra, "resource_count", 0)
scalardl_blue_resource_count = lookup(var.scalardl, "blue_resource_count", 0)
scalardl_green_resource_count = lookup(var.scalardl, "green_resource_count", 0)
scalardl_replication_factor = lookup(var.scalardl, "replication_factor", 0)
scalardl_replication_factor = lookup(var.scalardl, "cassandra_replication_factor", 0)
}

### default
Expand Down
13 changes: 8 additions & 5 deletions modules/azure/scalardl/cluster/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Cluster Module of Scalar DL
The Cluster module deploys a Scalar DL cluster on Azure.

## Providers
## Requirements

| Name | Version |
|------|---------|
| terraform | >= 0.12 |
| azurerm | =1.38.0 |

## Inputs
Expand All @@ -13,18 +14,19 @@ The Cluster module deploys a Scalar DL cluster on Azure.
|------|-------------|------|---------|:--------:|
| availability_set_id | n/a | `string` | `""` | no |
| bastion_ip | The IP to bastion host used for provisioning | `any` | n/a | yes |
| cassandra_password | The password of cassandra cluster | `any` | n/a | yes |
| cassandra_username | The username of cassandra cluster | `any` | n/a | yes |
| cassandra_replication_factor | The replication factor for the Cassandra schema | `number` | `3` | no |
| database_password | The database password | `any` | n/a | yes |
| database_username | The database username | `any` | n/a | yes |
| enable_accelerated_networking | A flag to enable accelerated networking on network interface | `bool` | `false` | no |
| enable_tdagent | A flag to install td-agent that forwards logs to the monitor host | `bool` | `true` | no |
| image_id | The image id to initiate | `any` | n/a | yes |
| internal_domain | Internal domain | `any` | n/a | yes |
| region | The Azure region to deploy environment | `any` | n/a | yes |
| locations | The Azure availability zones to deploy environment | `list(string)` | n/a | yes |
| network_dns | The ID for the internal DNS zone | `any` | n/a | yes |
| network_name | The name of the network resources: should be generated by provider/universal/name-generator | `any` | n/a | yes |
| private_key_path | The path to the private key for SSH access | `any` | n/a | yes |
| public_key_path | The path to the public key for SSH access | `any` | n/a | yes |
| replication_factor | Set the replication factor for schema | `number` | `3` | no |
| region | The Azure region to deploy environment | `any` | n/a | yes |
| resource_cluster_name | The name to assign the resource cluster | `any` | n/a | yes |
| resource_count | The number of resources to create | `any` | n/a | yes |
| resource_root_volume_size | The size of resource root volume size | `any` | n/a | yes |
Expand All @@ -42,3 +44,4 @@ The Cluster module deploys a Scalar DL cluster on Azure.
| network_interface_ids | A list of network interface IDs associated with the VM. |
| network_interface_private_ip | A list of private IP addresses assigned to scalardl cluster instances. |
| vm_ids | A list of VM IDs of a scalardl cluster. |

13 changes: 7 additions & 6 deletions modules/azure/scalardl/cluster/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,12 @@ module "scalardl_provision" {
private_key_path = var.private_key_path
provision_count = var.resource_count
enable_tdagent = var.enable_tdagent
internal_domain = var.internal_domain

scalardl_image_name = var.scalardl_image_name
scalardl_image_tag = var.scalardl_image_tag
replication_factor = var.replication_factor
cassandra_username = var.cassandra_username
cassandra_password = var.cassandra_password
scalardl_image_name = var.scalardl_image_name
scalardl_image_tag = var.scalardl_image_tag
internal_domain = var.internal_domain
database_contact_points = "cassandra-lb.${var.internal_domain}" # TODO: add to variables
database_username = var.database_username
database_password = var.database_password
cassandra_replication_factor = var.cassandra_replication_factor
}
Loading