How to use VCS, CLI and API driven workflows together on same workspace

[kspradheep]

Feature description 💡

Hello,

1. We use hosted bitbucket server and configured a workspace with VCS workflow but don't see plans get trigger automatically when pullreuqest opened or merged. Everytime I had to run job from UI. How to configure triggers? And how to run job from feature branches?

2. Is it possible to run CLI driven workflow for a workspace that was configured VCS workflow already? Basically I would like to test my changes before commit and push changes to remote branch.

Thanks!

Anything else?

No response

[alfespa17]

Hello @kspradheep.

1. Right now there is no support to handle changes when you open/close a pull request, if you want to trigger a job you will have to write some pipeline in bitbucket to trigger the job using the Terrakube API.
2. You should not be mixing VCS and CLI driven worflow, I am not even sure if you can do that when using TFC according to this documentation

I guess that to support pull request automation we will have to handle some webhook from bitbucket and trigger the job, at least from my side I don't have the bandwidth to do all the research to check how it should work.

If someone would like to help with us to add that feature any contribution is welcome in this project

Regards.

[alfespa17]

Maybe you could check atlantis, that project support pull request automation

[kspradheep]

Hi @alfespa17

Thanks for the quick reply.

Understood. How to run terraform plan from different branch like feature branch? Ideally we want to run dry-runs (terraform run excludes apply) from local machine through CLI driven workflow or run from feature branch. After git changes merged to main/master branch VCS workflow to run plan and apply.

Would be nice to have something like this https://docs.scalr.io/docs/workspaces-runs

[alfespa17]

Hello @kspradheep

If you want to use a feature branch, you will have to override the default branch in the workspace using the API, something like the following:

PATCH /organization/{organizationId}/workspace/{workspaceId}
{
    "data": {
        "type": "workspace",
        "attributes": {
            "branch": "feature-branch"
        }
}

After you have updated the value you could run the new job.

By the way you could build custom jobs, we do that using templates and extensions, that would be like the equivalent for workspace runs in scalr to build like a custom flow

[kspradheep]

@alfespa17 workspace branch update using API throws 415 response code.

Also, trying to configure terrakube cli but login failing on tenant id. https://docs.terrakube.io/user-guide/terrakube-cli/getting-started#authentication doc has Azure example. I'm using Dex OIDC connector. What is TERRAKUBE_TENANT_ID?

[alfespa17]

Your are missing the content type

application/vnd.api+json

The CLI won't work it hasn't been update in a long time, it is no even compatible with Dex, there is an issue about that

#306

[kspradheep]

Hi @alfespa17

I configured cli-driven workflow and noticed some issues.

1. To view CLI run in a browser, the provided url is not working. Is this expected?

2. The run status still shows waiting for approval and stuck when Terraform apply CLI approval entered a value other than "yes"

How to cancel this run?

Thanks!

[alfespa17]

For the first one there is an open issue check here

For the second issue I will have to check the code, which terrakube version are you using?

[kspradheep]

Chart: terrakube-3.10.2
App version: 2.17.2

[alfespa17]

Ok, let me check because discard logic should work

[alfespa17]

I think I found the issue I created a small fix, maybe you could override the API version and use 2.17.3-beta.1 to test

[kspradheep]

Okay, do I need to override just api component or all (api, executor, ui and registry)?

[alfespa17]

Just the API

[kspradheep]

okay, it worked and can't reproduce it for new runs however it didn't work for the old run. How to kill old run?

2023-11-28 00:08:00.044  INFO 1 --- [ryBean_Worker-4] o.t.api.plugin.scheduler.ScheduleJob     : Checking Job 57 Status waitingApproval
2023-11-28 00:08:00.044  INFO 1 --- [ryBean_Worker-4] o.t.api.plugin.scheduler.ScheduleJob     : Job 57 Status waitingApproval

2023-11-28 00:10:50.302  INFO 1 --- [nio-8080-exec-4] o.t.a.p.s.c.TerraformOutputController    : Reading output from storage
2023-11-28 00:10:50.302  INFO 1 --- [nio-8080-exec-4] o.t.a.p.s.aws.AwsStorageTypeServiceImpl  : Searching: tfoutput/00cc49f5-4f9f-432c-b296-e74402819a5a/57/e77e6776-953f-4b4f-ba49-afd9c586a37c.tfoutput
2023-11-28 00:10:55.172  INFO 1 --- [nio-8080-exec-3] o.t.a.p.s.aws.AwsStorageTypeServiceImpl  : Searching: tfstate/00cc49f5-4f9f-432c-b296-e74402819a5a/dc1736ec-5291-4874-a951-26cdd03e0fc8/state/44765038-77e8-4b9f-aa9d-82edfcb2a5d9.json
2023-11-28 00:10:55.234  INFO 1 --- [nio-8080-exec-7] o.t.a.p.s.aws.AwsStorageTypeServiceImpl  : Searching: /tfoutput/context/57/context.json
2023-11-28 00:10:55.247 ERROR 1 --- [nio-8080-exec-7] o.t.a.p.s.aws.AwsStorageTypeServiceImpl  : S3 Not found: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: ZMH5FQF0D38ZZB9Q; S3 Extended Request ID: kSOIEbSBQETy6vm1dQ1eIb7dy8KwQEemw8O3I6STLpxJiWwt7sD6a7jBiMyN4gYLA6BJ7H05EbM=; Proxy: null)

[alfespa17]

I think I can add some logic so jobs gets cancelled after some period of time like 6 hours, I will do that tomorrow, other option is to go to the database and change the job status to failed

[kspradheep]

Hello @kspradheep

If you want to use a feature branch, you will have to override the default branch in the workspace using the API, something like the following:

PATCH /organization/{organizationId}/workspace/{workspaceId}
{
    "data": {
        "type": "workspace",
        "attributes": {
            "branch": "feature-branch"
        }
}

After you have updated the value you could run the new job.

By the way you could build custom jobs, we do that using templates and extensions, that would be like the equivalent for workspace runs in scalr to build like a custom flow

Assume I created a new template and wanted to use for terraform CLI operations. How do I change job template for workspace?

[alfespa17]

Not sure what you are trying to do, but you can go to the organization settings and you will find an option for templates, there you will find some special templates that are used specifically for CLI driven workflow.

You could change those templates if you want, but you will have to be very carefull because you could break the internal logic for the CLI driven worlkflow.

Maybe you could try to explain why you want to change the templates logic so I can get more context and help you a little bit

[kspradheep]

My use case: Run terraform operations from both UI (VCS) and CLI for same workspace.
I created a workspace with VCS workflow and can run jobs from UI but also wanted to run jobs from terraform cli. This helps to test my changes locally before push to master/main branch.

[alfespa17]

Not sure if you can do that with terraform cloud, I will have to check the terraform CLI code to validate if that is possible

[kspradheep]

@alfespa17 any updates?

[alfespa17]

Sorry I have been a little busy I will try to check next week.

[alfespa17]

I added this feature you can now run a job using the terraform CLI with a workspace using a private VCS connection, maybe you can test it to see if you can find some bug.

You could test version "2.19.0-beta.1"

[kspradheep]

@alfespa17 It's triggering a job from CLI but not taking locally changed terraform files. It's using VCS repo code scripts. This not helps to test my terraform changes locally before push to master/main branch.

[kspradheep]

@alfespa17 is above behavior expected?

[alfespa17]

Hello @kspradheep

I was testing but I am not facing your issue:

I have this workspace using a VCS github connection that is working correctly:

I did some changes in my local directory to simulate changes that I could test adding the backend.tf with the following:

terraform {
  cloud {
    organization = "simple"
    hostname = "8080-azbuilder-terrakube-srvv8ms68ej.ws-us107.gitpod.io"

    workspaces {
      name = "simple-terraform"
    }
  }
}

Then I can run:

terraform init
terraform plan

And it is working as expected running my local changes remotely:

user@pop-os:~/git/simple-terraform$ terraform plan

Running plan in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://8080-azbuilder-terrakube-srvv8ms68ej.ws-us107.gitpod.io/app/simple/simple-terraform/runs/3

Waiting for the plan to start...

***************************************
Running Terraform PLAN
***************************************
module.time_module.random_integer.time: Refreshing state... [id=4]
null_resource.previous: Refreshing state... [id=2524947301361236107]
time_sleep.wait_30_seconds: Refreshing state... [id=2024-01-03T00:09:15Z]
null_resource.next: Refreshing state... [id=6339220006141915436]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.next2 will be created
  + resource "null_resource" "next2" {
      + id = (known after apply)
    }

  # null_resource.next3 will be created
  + resource "null_resource" "next3" {
      + id = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Could you test with version "2.19.0-beta.2"??

[kspradheep]

Hello @alfespa17 I'm using "2.19.0-beta.2". The test case I did was changing input variable values. Example:

1. With VCS driven, i didn't supply a value to input variable and run should fail - it's working as expected
2. With CLI driven, I supply a value to input variable and run shouldn't fail - it's not working as expected

[alfespa17]

I think it should work if you use a terraform.auto.tfvars file locally to put the variables

[kspradheep]

I tried with terraform.auto.tfvars and terraform.tfvars but not working.

[alfespa17]

Maybe adding a var could help to reproduce your issue, I will check tomorrow

[alfespa17]

I did some tests and I was able to add variables using terraform.auto.tfvars and also adding the terraform variable from the UI and trigger the job using the CLI driven workflow. I have no idea how to reproduce your issue.

If you could create a sample repository with your test case I will be happy to help you, but right not I am out of ideas :(

[kspradheep]

these are the steps I performed:

• Create a workspace using VCS ssh git connection, master/main branch
• Git VCS main branch terraform code has input variable "input_x" but not defined default value or passed it from terraform.auto.tfvars
• Run job from UI and it fails. In Terrakube ui, workspace runs tab shows "commitId b0dbdc | triggered via UI"
• Clone the repo locally and switch to different branch other than master/main branch and define value for "input_x" in terraform.auto.tfvars
• Run terraform init and terraform plan CLI commands in terminal. In Terrakube ui, workspace runs tab shows "commitId b0dbdc | triggered via CLI". It tells it is using git code instead of local dir code.

Did you perform these steps in your tests?

[kspradheep]

@alfespa17 are you able to reproduce the issue with above steps?

[kspradheep]

@alfespa17 any updates?