Skip to content

Commit

Permalink
[DM] dev branch merge to master (#294)
Browse files Browse the repository at this point in the history
* [DM] cloud_router: refactoring (#64)

#51

- Added version, links to docs
- Added support for "description"
- Switched to using type provider
- Added support for cross-project resource creation
- Added missing fields:
  - bgp (only asn is supported now)
  - nats
  - bgpPeers
  - interfaces
- Added basic schema unit tests

* [DM] project: refactoring (#59)

#57

- Added version, links to docs
- Added "labels" support
- Added resource name prefix
- Updated compute-beta to compute-v1
- Fixed deprecated gcp-types/compute-v1:compute.subnetworks.setIamPolicy
bindings
- Refactored default resources removal: remove code duplication
- Added usageExportBucket name prefix

* [DM] gcs_bucket: refactoring (#83)

#66

- Added version, links to docs
- Added uniqueItems: true to lists
- Switched to using type provider
- Added cross-project creation support
- Added additionalProperties: false for nested object
- Fixed "bindings" schema
- Added support for "requesterPays"
- Added support for "acl", "billing", "cors", "defaultEventBasedHold", "defaultObjectAcl", "encryption", "iamConfiguration", "retentionPolicy"
- Fixed resource name

* [DM] network: refactoring (#61)

#56

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation (both network and
subnetwork)
- Added oneOf check for subnets: autoCreateSubnetworks should be exclusive
with subnet list
- Fixed network & subnetworks resources names
- Added support for "description", "routingConfig" to network
- Fixed "secondaryIpRanges" definition in subnetwork
- Added basic schema unit tests

* [DM] cloud_sql: refactoring (#62)

#54

- Added version, links to docs
- Switched to using type provider
- Added enum for "databaseVersion", "region", "settings.pricingPlan",
"settings.replicationType"
- Added additionalProperties: false for nested object
- Fixed "users" schema
- Added basic schemas unit tests

* [DM] cloud_spanner: refactoring (#63)

#53

- Added version, links to docs
- Added support for "labels"
- Switched to using type provider
- Made "name" optional, correctly handle it
- Added support for cross-project resource creation
- Updated "instanceConfig" enum with new values
- Added basic schema unit tests
- Added additionalProperties: false for nested objects

* [DM] instance: refactoring (#84)

#70

- Added version, links to docs
- Switched to using type provider
- Added cross-project creation support
- Added additionalProperties: false for nested objects
- Added support for "description", "networkInterfaces[].accessConfigs[]",
"networkInterfaces[].aliasIpRanges[]", "disks", "scheduling", "labels",
"minCpuPlatform", "guestAccelerators", "deletionProtection", "hostname",
"shieldedInstanceConfig", "shieldedInstanceIntegrityPolicy":
[docs](https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert)
- Fixed resource name

* [DM] autoscaler: refactoring (#82)

#67

- Added version, links to docs
- Switched to using type provider
- Added cross-project creation support
- Added additionalProperties: false for nested object
- Fixed resource name

* [DM] cloud_function: refactoring (#65)

#52

- Added version, links to docs
- Switched to using type provider
- Upgraded API version: v1beta2 -> v1
- Added cross-project creation support (including upload.py)
- Added support for "labels", "environmentVariables" fields
- Renamed "region" to "location" with backward compatibility + there is
oneOf check
- Added new runtimes, change default to nodejs10
- Fixed function upload code (it is not redeployed after code upload when
using other project)
- Updated source upload logic:

- If "sourceRepository" field is present, use it as-is
- If "sourceRepositoryUrl" field is provided, change it to
"sourceRepository" format
- If "sourceArchiveUrl" is provided, use it as-is
- If "localUploadPath" is present, create cloudbuild build for function
zip file. Use "sourceArchiveUrl" as target or generate it automatically
(providing generated path via ""sourceArchiveUrl")
- If "sourceUploadUrl" is present, use it as-is
- All other parameter combinations should be restricted

* Fixes #55: DM/folder: refactoring
#55

- added version, links to docs
- fixed folder resource names: use base resource name as a template
- added oneOf check for folderId/orgId + fix incorrect schema for array
- fixed "displayName" regex
- made "name" field optional
- using the same format for parent as in project + fixed name prefix to
plural

* Fixes #69: DM/managed_instance_group: refactoring (#90)

#69

- Added version, links to docs
- Switched to using type provider
- Added cross-project creation support
- Added additionalProperties: false for nested objects
- Fixed resource name

* [DM] instance_template refactoring (#89)

#68

- Added version, links to docs
- Switched to using type provider
- Added cross-project creation support
- Added additionalProperties: false for nested objects
- Added support for "networkInterfaces[].accessConfigs[]", "disks",
"scheduling", "minCpuPlatform", "guestAccelerators",
"shieldedInstanceConfig", "sourceInstance", "sourceInstanceParams"
- Fixed resource name

* Fixes #119: DM/route: refactoring (#120)

#119

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Fixed arrays, objects and required fields (check for routeType + make
old scheme optional)
- Added missing fields: "description", "nextHopInstance",
"nextHopInstance", "nextHopNetwork", "nextHopGateway",
"nextHopVpnTunnel"

* Fixes #119: DM/route: refactoring
#119

Small typo, rechecked tests

* CFT build config to point to the correct license file

* [DM][FIX] Project: SA can not be created ("project_id" field is unknown)

#149

* [DM] iam_member: refactoring (#129)

* Fixes #116: DM/iam_member: refactoring
#116

* [DM] firewall: refactoring (#100)

#99
#91

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Upgraded compute-beta to compute-v1
- Fixed resource names
- Added schema to "rules": "name", "description", "priority",
"sourceRanges", "destinationRanges", "sourceTags", "targetTags",
"sourceServiceAccounts", "targetServiceAccounts", "allowed", "denied",
"direction", "logConfig", "disabled"

* [DM] bastion: refactoring (#102)

#101

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Fixed instance.py outputs (it lacks internalIp/externalIP)
- Fixed endless loops in tests

* [DM] backend_service: refactoring (#104)

#103

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Added new fields: "backends[].maxRatePerEndpoint,
.maxConnectionsPerEndpoint", "iap", "customRequestHeaders[]"
- Changed healthCheck to healthChecks[]
- Added checks for INTERNAL/EXTERNAL LB schemes for all relevant fields
- Added checks for backends[].balancingMode

* [DM] bigquery: refactoring (#106)

#105

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Added missing fields to datasets: "friendlyName",
"defaultPartitionExpirationMs", "labels", "access"
- Added missing fields to tables: "description", "labels", "clustering",
"requirePartitionFilter", "externalDataConfiguration",
"encryptionConfiguration"
- Fixed resource names
- Added uniqueItems: true and additionalProperties: false

* [DM] gke: refactoring (#111)

#110
GoogleCloudPlatform/deploymentmanager-samples#436
GoogleCloudPlatform/deploymentmanager-samples#463

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Removed deprecated "nodeConfig", switch to "nodePools[].config"
- Removed deprecated "initialNodeCount", switch to
"nodePools[].initial_node_count"
- Removed deprecated "privateCluster", "masterIpv4CidrBlock" ->
"privateClusterConfig"
- Updated parameters: "masterAuth", "loggingService",
"monitoringService", "addonsConfig",
"ipAllocationPolicy" and others
- Added support for "nodePools[]", "binaryAuthorization", "autoscaling",
"networkConfig", "defaultMaxPodsConstraint",
"resourceUsageExportConfig", "authenticatorGroupsConfig",
"verticalPodAutoscaling", "tierSettings", "workloadIdentityConfig",
"nodeConfig->diskType,sandboxConfig,shieldedInstanceConfig",
"enableTpu", "databaseEncryption"
- Added uniqueItems: true and additionalProperties: false

* [DM] kms: refactoring (#128)

#117

- Added version, links to docs
- Added support for cross-project resource creation
- Fixed resource names
- Added missing algorithms

* [DM] pubsub: refactoring (#125)

#118

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Added topic fields: "labels"
- Added subscription fields: "pushConfig", "retainAckedMessages",
"messageRetentionDuration", "labels", "expirationPolicy"

* Fixes #115: DM/iam_custom_role: refactoring (#130)

#115

- Added version, links to docs
- Added support for cross-project resource creation
- Added support for "stage" field

* [DM] ip_reservation: refactoring (#133)

#132

- Addde version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Added missing fields: "prefixLength", "networkTier"
- Fixed field checks

* [DM] IAM_member folder and org support (#166)

#162

Added support for folders and organizations:
gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding
gcp-types/cloudresourcemanager-v2:virtual.folders.iamMemberBinding
gcp-types/cloudresourcemanager-v1:virtual.organizations.iamMemberBinding

* [DM] healthcheck: refactoring (#131)

#124

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Added support for missing fields: "description", "name"

* [DM] nat_gateway: refactoring (#134)

#123

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation

* [DM] ssl_certificate: refactoring (#152)

#148

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names

* [DM] url_map: refactoring (#155)

#153

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names

* [DM] target_proxy: refactoring (#156)

#154

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Upgraded compute api version
- Fixed fields descriptions, add checks to arrays and objects
- Added support for "sslCertificates"

* [DM] forwarding_rule: refactoring (#157)

#141

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Fixed descriptions
- Added support for fields: "serviceLabel", "networkTier",
"allPorts"
- Added strict properties checking

* [DM] external_load_balancer: refactoring (#159)

#140

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names

* [DM] Internal_load_balancer: refactoring (#168)

#143

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names

* [DM] haproxy: refactoring (#169)

#142

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Fixed imports

* [DM] dataproc: refactoring (#170)

#137

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Added missing fields: "labels", "<nodes>.imageUri",
"<nodes>.isPreemptible", "<nodes>.accelerators",
"softwareConfig.optionalComponents", "encryptionConfig"
- Mergeed master, nodes and secondary nodes

* [DM][BugFix] GKE template examples (#172)

#171

* [DM] dns_managed_zone: refactoring #138 (#176)

* [DM] Project Factory readme extension

Adding Service Usage Admin to prerequisites

* [DM] Patch IAM member to projectID (#187)

* [DM] Patch IAM member to projectID

Project ID was forcefully overwritten to context.env['project']

* [DM] CloudDNS - Backwards compatibility and bugfixes (#192)

Added Backward Compatibility with the old property zoneName. Added Backward Compatibility Integration Test. Fixed Identation of integration tests and added project parameter for gcloud calls. Adjusted schema paterns. Added description for all the tests. Updated README

* Fix couple of typographical errors.

* Fix prerequisites target name in docs and Makefile

* [DM] - IAM needs to depends on SA existance (#222)

#175

* [DM] interconnect_attachment: refactoring (#195)

#183

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Added missing fileds: "pairingKey", "vlanTag8021q", "adminEnabled",
"candidateSubnets", "bandwidth", "partnerMetadata", "partnerAsn"

* [DM] runtime_config: refactoring (#201)

#184

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Standardized naming

* [DM] interconnect: refactoring (#196)

#182

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Updateed/added fields: "linkType", "requestedLinkCount",
"interconnectType"

* [DM] backwards compatibility and styling fix

Fixed Backward Compatibility Support. Fixed pydocstyle D210, D202. Restricted dnsName pattern to comply with RFC. 1035 #138 (#194)

* [DM] Cloud DNS - Fix/162 (#165)

* [DM] CloudDNS - Backwards compatibility and bugfixes (#192)

Added Backward Compatibility with the old property zoneName. Added Backward Compatibility Integration Test. Fixed Identation of integration tests and added project parameter for gcloud calls. Adjusted schema paterns. Added description for all the tests. Updated README

* Fixes #162: [DM] IAM_member folder and org support
#162
#151

Added support for folders and organizations:
gcp-types/cloudresourcemanager-v1:virtual.projects.iamMemberBinding
gcp-types/cloudresourcemanager-v2:virtual.folders.iamMemberBinding
gcp-types/cloudresourcemanager-v1:virtual.organizations.iamMemberBinding

Updated tests to be non-interactive + added tests for folders

Fixed removal of mid-list members

* [DM] Refactored dns_records template #139 (#197)

* [DM] vpn: refactoring (#202)

#186

- Added version, links to docs
- Switched to using type provider
- Added support for cross-project resource creation
- Fixed resource names
- Added missing router property

* [DM] stackdriver_metric_descriptor: refactoring (#207)

#185

- Added version, links to docs
- Added support for cross-project resource creation
- Fixed resource names
- Added support for "launchStage", deprecated "metadata.launchStage"
- Added checks for "unit"

* [DM] cloudbuild: refactoring (#211)

#180

- Added version, links to docs
- Added support for cross-project resource creation
- Fixed resource names
- Added checks to objects and arrays
- Fixed "artifacts.objects" property, add "options.workerPool, env,
secretEnv, volumes" for cloudbuild
- Added "tags", "github", "triggerTemplate.revision->commitSha" for
trigger

* [DM] cloud_tasks: refactoring (#212)

#179

- Added version
- Fixed resource names

* [DM] network_peering: refactoring (#214)

#145

- Added version, links to docs
- Added support for cross-project resource creation
- Fixed resource names
- Added support for "networkPeering" field

* [DM] Shared VPC Subnet IAM refactoring

Refactored shared_vpc_subnet_iam template. Added bindings and policy syntax tests(in addition to legacy syntax check) (#215)

* [DM] Backend_service update example according to schema validation (#221)

#220

* [DM] cloud_tasks: refactoring (#208)

#179

- Added version
- Fixed resource names

* [DM] logsink: refactoring (#228)

#144

- Added version, links to docs
- Added support for cross-project resource creation
- Fixed resource names

* Enable users to specify network reference in the firewall DM template.

* Enable GKE DM template to support 'latest' initial cluster version.

* Change the logic of checking initial cluster version in the GKE DM template.

* [DM] Org_Policy updates(#231) 

- Added version, links to docs
  - Fixed resource names
  - Fixed pydocstyle warnings
  - Added schema validation against the API documentation
  - Added project parameter to gcloud deployment-manager calls

* [DM] forseti: refactoring (#239)

#181

- Added version, links to docs
- Switched to using type provider

* [DM] Enable cloud_router DM template to support networkURL. (#235)

* Enable cloud_router to support networkURL.

* Update cloud_router DM schema for netowrkURL and networkName.

* Continue to use network instead of networkName so that the CL is not a breaking change.

* Update cloud_router DM schema.

* Update cloud_router.py.schema

* [DM] logsink test fix

Following topic rename in test file

* [DM] Small fixes to templates. (#268)

* [DM] fix healthcheck type property

#270

* [DM] fixing CloudSQL example

* HA proxy example remove unneeded import

* [DM] adding schema validation test

* [DM] bastion bats improve

* [DM] Bastion test disk size correction

* [DM] Fixes #269: DM dev branch quality control (#282)

* Fixes #269: DM dev branch quality control
#269

- Fixed templates/bigquery/tests/integration/bigquery.bats

* Fixes #269: DM dev branch quality control
#269

- Fixed templates/haproxy/tests/integration/haproxy.bats

* Fixes #269: DM dev branch quality control
#269

- Fixed templates/instance_template/tests/integration/instance_template_networks.bats

* Fixes #269: DM dev branch quality control
#269

- Fixed templates/instance/tests/integration/instance.bats

* Fixes #269: DM dev branch quality control
#269

- Fixed templates/instance/tests/integration/instance_template/instance_template.bats

* Fixes #269: DM dev branch quality control
#269

- Fixed templates/shared_vpc_subnet_iam/tests/integration/shared_vpc_subnet_iam.bats

* [DM] Fixing External LB test cleanup

* [DM] Updating changelog (#293)

* Updating changelog

* Adding container images to run tests in CI.

* [DM] warning for bastion host sudoers (#297)

Priorities are default to 1000, not incrementing.

Adding debug messages.

* [DM] Isolate a single template from route DM template. (#257)

* Split route DM template into two parts: a route group and a single route.

* Enable single_route DM template to accept VPN tunnel URL.

* Always use context.env name in single_route DM template.

* Allow users to specify single_route real name in DM template.

* [DM] Enable VPN DM template to support static routing. (#234)

* Enable VPN DM template to support static routing.

* remove two string init, recover comments for address API link.

* recover name and project on the adress resource in VPN DM template.

* Update VPN DM template schema.

* Change routerName back to router.

* Enable VPN template to output an URI of the created VPN tunnel.

* VPN DM schema add uniqItems on selectors.

* [DM] Fixed a path error in an Cloud SQL exampl… (#300)

* farisjarrah/cloud-sql-docs: rebased off of cft-dm-dev to submit to upstream dev branch

* farisjarrah/cloud-sql-docs: fixed postgres version number type, and ensured that the templates matched the cft-dm-dev branch schema for cloud sql
  • Loading branch information
ocsig committed Sep 3, 2019
1 parent e4133e7 commit 20d235a
Show file tree
Hide file tree
Showing 242 changed files with 10,606 additions and 4,218 deletions.
2 changes: 1 addition & 1 deletion cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ After build find binary at bin/cft location
Follow cft --help instructions


Google Cloud Formation Toolkit CLI
Google Cloud Foundation Toolkit CLI

```bash
Usage:
Expand Down
4 changes: 2 additions & 2 deletions cli/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ import (
// rootCmd represents the base command when called without any subcommands
var rootCmd = &cobra.Command{
Use: "cft",
Short: "Google Cloud Formation Toolkit CLI",
Long: "Google Cloud Formation Toolkit CLI",
Short: "Google Cloud Foundation Toolkit CLI",
Long: "Google Cloud Foundation Toolkit CLI",
Args: cobra.NoArgs,
Run: func(cmd *cobra.Command, args []string) {
// no params means same as -h flag
Expand Down
6 changes: 6 additions & 0 deletions dm/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.

## CFT Templates


### 23.08.2019

- Adding container images for test automation
- Finalizing 'cft-dm-dev' branch for merge to master

### 21.03.2019

- *Templates/iam_member*: The template is now using virtual.projects.iamMemberBinding which is and advanced
Expand Down
24 changes: 24 additions & 0 deletions dm/CI/cft_base_contianer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
FROM gcr.io/cloud-builders/gcloud

RUN apt-get update && apt-get -y install make \
&& apt-get -y install gettext-base \
&& pip install --upgrade pip \
&& pip install setuptools \
&& git clone https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit \
&& cd cloud-foundation-toolkit/dm \
&& rm -rf templates \
&& make prerequisites \
&& make build \
&& make install \
&& pip install tox \
&& pip install pytest \
&& make cft-venv \
&& make template-prerequisites \
&& make cft-prerequisites \
&& . venv/bin/activate \
&& ./src/cftenv \
&& pwd \
&& cft --version \
&& bats -v

WORKDIR /cloud-foundation-toolkit/dm
13 changes: 13 additions & 0 deletions dm/CI/cft_base_contianer/cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/cft:${_CFT_VERSION}',
'-t', 'gcr.io/$PROJECT_ID/cft',
'--build-arg', 'CFT_VERSION=${_CFT_VERSION}',
'.']
substitutions:
_CFT_VERSION: 0.0.4

images:
- 'gcr.io/$PROJECT_ID/cft:latest'
- 'gcr.io/$PROJECT_ID/cft:$_CFT_VERSION'
tags: ['cft-test-dm']
16 changes: 16 additions & 0 deletions dm/CI/cft_schema_runner/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM gcr.io/cloud-builders/gcloud

RUN apt-get update
RUN apt-get install python-setuptools -y
RUN apt-get install npm -y
RUN apt-get install jq -y
RUN pip install yq
RUN npm install -g ajv-cli
RUN ln -s /usr/bin/nodejs /usr/bin/node

COPY docker-entrypoint.sh /root/
RUN chmod 777 /root/docker-entrypoint.sh

ENTRYPOINT ["/root/docker-entrypoint.sh"]

CMD []
177 changes: 177 additions & 0 deletions dm/CI/cft_schema_runner/cloudbuild-test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# find . -name "*.yam"l | grep examples| sort -n | awk '{print "- name: 'gcr.io/\$PROJECT_ID/cft-schema'\n args: [`"$1"`]"}' | sed "s/\`/'/g"

steps:
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/autoscaler/examples/autoscaler_regional.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/autoscaler/examples/autoscaler_zonal.yaml']
# bug
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/backend_service/examples/backend_service_global.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/backend_service/examples/backend_service_regional.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/bastion/examples/bastion.yaml']
# Skip, complex example
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/bigquery/examples/bigquery.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloudbuild/examples/cloudbuild_reposource.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloudbuild/examples/cloudbuild_storagesource.yaml']
# FAILING
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/cloudbuild/examples/cloudbuild_trigger.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloudbuild/examples/cloudbuild.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_function/examples/cloud_function_upload.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_function/examples/cloud_function.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_router/examples/cloud_router.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_spanner/examples/cloud_spanner.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_sql/examples/cloud_sql_read_replica.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_sql/examples/cloud_sql.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_tasks/examples/cloud_tasks_queue.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/cloud_tasks/examples/cloud_tasks_task.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/dataproc/examples/dataproc.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/dns_managed_zone/examples/dns_managed_zone_legacy.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/dns_managed_zone/examples/dns_managed_zone_private_visibility_config.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/dns_managed_zone/examples/dns_managed_zone_private.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/dns_managed_zone/examples/dns_managed_zone_public.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/dns_managed_zone/examples/dns_managed_zone.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/dns_records/examples/dns_records.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/external_load_balancer/examples/external_load_balancer_https.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/external_load_balancer/examples/external_load_balancer_http.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/external_load_balancer/examples/external_load_balancer_ssl.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/external_load_balancer/examples/external_load_balancer_tcp.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/firewall/examples/firewall.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/folder/examples/folder.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/forseti/examples/forseti.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/forwarding_rule/examples/forwarding_rule_global.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/forwarding_rule/examples/forwarding_rule_regional.yaml']
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/gcs_bucket/examples/gcs_bucket_iam_bindings.yaml']
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/gcs_bucket/examples/gcs_bucket_lifecycle.yaml']
# SCHEMA version issue
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/gcs_bucket/examples/gcs_bucket.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/gke/examples/gke_regional_private.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/gke/examples/gke_regional.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/gke/examples/gke_zonal.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/haproxy/examples/haproxy.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/healthcheck/examples/healthcheck.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/iam_custom_role/examples/iam_custom_role.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/iam_member/examples/iam_member.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/instance/examples/instance.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/instance_template/examples/instance_template.yaml']
# Schema faulty?
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/interconnect_attachment/examples/interconnect_attachment_dedicated.yaml']
# Schema faulty?
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/interconnect_attachment/examples/interconnect_attachment_partner.yaml']
# - name: gcr.io/$PROJECT_ID/cft-schema
# FIXME
# args: ['./templates/interconnect/examples/interconnect_dedicated.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/internal_load_balancer/examples/internal_load_balancer.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/ip_reservation/examples/ip_reservation.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/kms/examples/kms_signkey.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/kms/examples/kms.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/logsink/examples/billingaccount_logsink_bucket_destination.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/logsink/examples/folder_logsink_bq_destination.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/logsink/examples/org_logsink_pubsub_destination.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/logsink/examples/project_logsink_bucket_destination.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/managed_instance_group/examples/managed_instance_group_healthcheck.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/managed_instance_group/examples/managed_instance_group.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/nat_gateway/examples/nat_gateway.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/network/examples/network.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/network_peering/examples/network_peering.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/org_policy/examples/org_policy.yaml']
# FIXME
# - name: gcr.io/$PROJECT_ID/cft-schema
# args: ['./templates/project/examples/project.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/pubsub/examples/pubsub_push.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/pubsub/examples/pubsub.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/route/examples/route.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/runtime_config/examples/runtime_config.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/shared_vpc_subnet_iam/examples/shared_vpc_subnet_iam_bindings.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/shared_vpc_subnet_iam/examples/shared_vpc_subnet_iam_legacy.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/shared_vpc_subnet_iam/examples/shared_vpc_subnet_iam_policy.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/ssl_certificate/examples/ssl_certificate.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/stackdriver_metric_descriptor/examples/stackdriver_metric_descriptor.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/target_proxy/examples/target_proxy_https.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/target_proxy/examples/target_proxy_http.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/target_proxy/examples/target_proxy_ssl.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/target_proxy/examples/target_proxy_tcp.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/url_map/examples/url_map.yaml']
- name: gcr.io/$PROJECT_ID/cft-schema
args: ['./templates/vpn/examples/vpn.yaml']
tags: ['cft-dm-schema-runner']
13 changes: 13 additions & 0 deletions dm/CI/cft_schema_runner/cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/cft-schema:${_CFT_VERSION}',
'-t', 'gcr.io/$PROJECT_ID/cft-schema',
'--build-arg', 'CFT_VERSION=${_CFT_VERSION}',
'.']
substitutions:
_CFT_VERSION: 0.0.4

images:
- 'gcr.io/$PROJECT_ID/cft-schema:latest'
- 'gcr.io/$PROJECT_ID/cft-schema:$_CFT_VERSION'
tags: ['cft-test-dm']
42 changes: 42 additions & 0 deletions dm/CI/cft_schema_runner/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash
set -eu

readonly GIT_URL='https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit'
readonly CLONE_DIRNAME="$(mktemp -d)"
readonly BRANCH_NAME="cft-dm-dev"

readonly COLOR_RESET='\033[0m'
readonly COLOR_BOLD='\033[1m'
readonly COLOR_BG_BLUE='\033[44m'

echo_color() {
echo -e "${COLOR_BOLD}${COLOR_BG_BLUE}$@${COLOR_RESET}"
}

echo_color "Cloning repo"
git clone "${GIT_URL}" "${CLONE_DIRNAME}"
cd "${CLONE_DIRNAME}"
git checkout "${BRANCH_NAME}"

echo_color 'Initializing CFT DM templates'

cd dm/templates

# cat healthcheck/examples/healthcheck.yaml | yq .resources[0].properties > project.json; cat healthcheck/healthcheck.py.schema | yq . > project.py.schema.json; ajv validate -s project.py.schema.json -d project.json

EXAMPLE_COUNT=`cat $@ | yq '.resources | length'`
EXAMPLE_COUNT=$(($EXAMPLE_COUNT-1))

while [ $EXAMPLE_COUNT -ge 0 ];
do
echo_color "Example $EXAMPLE_COUNT"
cat $@ | yq .resources[$EXAMPLE_COUNT].properties > example.json;
cat example.json
export SCHEMA_PATH=`cat $@ | yq -r .imports[0].path | awk '{print $1".schema"}'`
echo_color $SCHEMA_PATH
cat $SCHEMA_PATH | yq . > example.py.schema.json;
echo_color "Schema validation"
ajv validate -s example.py.schema.json -d example.json
EXAMPLE_COUNT=$(($EXAMPLE_COUNT-1))

done
14 changes: 14 additions & 0 deletions dm/CI/cft_test_runner/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
FROM gcr.io/cft-test-workspace-221111/cft:latest

COPY cloud-foundation-tests.conf /etc/cloud-foundation-tests.conf
RUN cat /etc/cloud-foundation-tests.conf
RUN chmod 666 /etc/cloud-foundation-tests.conf
COPY docker-entrypoint.sh /root/
RUN chmod 777 /root/docker-entrypoint.sh


WORKDIR /cloud-foundation-toolkit/dm

ENTRYPOINT ["/root/docker-entrypoint.sh"]

CMD []
5 changes: 5 additions & 0 deletions dm/CI/cft_test_runner/cloud-foundation-tests.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
export CLOUD_FOUNDATION_ORGANIZATION_ID="12345678
export CLOUD_FOUNDATION_PROJECT_ID="project_ID"
export CLOUDDNS_CROSS_PROJECT_ID="project_ID2"
export CLOUD_FOUNDATION_BILLING_ACCOUNT_ID="123456-789ABCD-000111"
export CLOUD_FOUNDATION_USER_ACCOUNT="user@cft.tips"
9 changes: 9 additions & 0 deletions dm/CI/cft_test_runner/cloudbuild-test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
steps:
- name: 'gcr.io/$PROJECT_ID/cft-ci-test'
args: ['${_BATS_TEST_FILE}']

substitutions:
_BATS_TEST_FILE: ./templates/autoscaler/tests/integration/autoscaler.bats # default value

tags: ['cft-dm-test-runner']
timeout: '7200s'
13 changes: 13 additions & 0 deletions dm/CI/cft_test_runner/cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/cft-ci-test:${_CFT_VERSION}',
'-t', 'gcr.io/$PROJECT_ID/cft-ci-test',
'--build-arg', 'CFT_VERSION=${_CFT_VERSION}',
'.']
substitutions:
_CFT_VERSION: 0.0.4

images:
- 'gcr.io/$PROJECT_ID/cft-ci-test:latest'
- 'gcr.io/$PROJECT_ID/cft-ci-test:$_CFT_VERSION'
tags: ['cft-test-dm']
Loading

0 comments on commit 20d235a

Please sign in to comment.