Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raw output when using wrapper with $() in bash #20

Closed
allejo opened this issue Jun 8, 2020 · 14 comments · Fixed by #367
Closed

Raw output when using wrapper with $() in bash #20

allejo opened this issue Jun 8, 2020 · 14 comments · Fixed by #367
Labels
question Further information is requested

Comments

@allejo
Copy link

allejo commented Jun 8, 2020

I was in the process of migrating our workflows to use this new action for setting up terraform when I ran into this... behavior? bug? I'm not sure what to call it.

When terraform_wrapper is set to true (the default behavior) in this action, it breaks what would be considered normal behavior in bash.

Take the following example from a step in a workflow:

- run: echo "credentials: $(terraform output my_arn)" > my_file.txt

The contents of the file created would be this:

credentials: [command]/home/runner/work/_temp/<some UUID>/terraform-bin output my_arn

Instead of the expected:

credentials: my_arn_value

The only way to fix this problem is to set terraform_wrapper to false. I'm not sure if this is something that can be fixed/changed? Otherwise, I think it would be a good idea to put this somewhere in the README.

/cc @petersin0422

@benc-uk
Copy link

benc-uk commented Jul 31, 2020

Same
This had me pulling my hair out for about 3 hours why the output terraform output always contained a load of gibberish!

It renders $(terraform output foo) impossible to use 😥

@NillsF
Copy link

NillsF commented Aug 24, 2020

Came to this comment since I am dealing with the same issue. I was a little confused by @allejo's comment on wrapper, but was able to figure it out:

If you want to be able to use terraform output, use the following terraform setup step in your action:

    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v1
      with:
        terraform_wrapper: false

@volkorny
Copy link

If disabling the wrapper is not good for your case you can use this horrible workaround:

     - name: Terraform Output
        id: get_ip
        run: terraform output instance_ip

      - name: Trim Output 
        uses: frabert/replace-string-action@v1.1
        id: trim
        with:
          pattern: '/(\n+)/'
          string: '${{ steps.get_ip.outputs.stdout }}'
          replace-with: ''

      - name: Add url to file
        run: echo "http://${{ steps.trim.outputs.replaced }}:${{ env.APP_PORT }}${{ env.APP_URI }}" >> file.txt

@tony84727
Copy link

Run into this too. And I need the wrapper so disabling it is not an option.
I found when creating the wrapper, the action actually renames the original terraform to terraform-bin:

// Rename terraform(.exe) to terraform-bin(.exe)
try {
source = [pathToCLI, `terraform${exeSuffix}`].join(path.sep);
target = [pathToCLI, `terraform-bin${exeSuffix}`].join(path.sep);
core.debug(`Moving ${source} to ${target}.`);
await io.mv(source, target);
} catch (e) {

So another workaround will be changing terraform to terraform-bin to use the terraform binary directly.

In the context of @allejo 's issue, it should be
echo "credentials: $(terraform-bin output my_arn)" > my_file.txt

@skeggse
Copy link

skeggse commented Jun 11, 2021

This ate up a good chunk of time for me. IMO this is surprising/unexpected behavior, and I'd love it if we disabled the wrapper by default. That said, it sounds like plenty of folks do derive value from the wrapper, though if you just throw setup-terraform in your workflow and expect it to give you a normal terraform binary you'll be sorely disappointed.

@mattmichal
Copy link

I have a workflow with some steps that leverage the features of the wrapper but I also needed the clean JSON plan output to use with infracost. This is the workaround that I used:

- name: Terraform show
  id: show-json
  if: github.ref != 'refs/heads/main'
  # terraform_wrapper adds GH-related text to stdout (debug, outputs, etc.)
  # the wrapper is required for other steps (fmt, init, validate) so it cannot be disabled
  # use sed to pull the second line of the file which, for this command, is the only line that is valid
  run: terraform show -json terraform.plan | sed -n 2p > terraform.plan.json

This will have to be adjust if the wrapper ever changes but this is good enough for now.

@NathanielRN
Copy link

I also ran into this issue, but for some reason doing the terraform output -raw my-value in another step seemed to work. Would be nice if we could have a command line flag to disable the wrapper temporarily so we can do manipulations on the data in one step!

An example of what I had to do:

- name: Extract SDK layer arn
  id: extract-arn
  run: terraform output -raw arn
- name: Set SDK layer version output
    # NOTE: If I try to do this and the above step in the same step,
    # I get garbage wrapper text in my variable!
  run: |
    arn_version=$(echo "${{ steps.extract-arn.outputs.stdout }}" | cut -d : -f 8)
    echo $arn_version

@eclosson
Copy link

eclosson commented Dec 8, 2021

I ran into this issue too, disabled the wrapper to work around it. I don't use the wrapper so wasn't an issue. Posting to help keep this issue alive as this was unexpected behavior that took a bit to sort out.

My vote would be to disable terraform_wrapper by default. If not a good option for most, next best thing would be a flag to the wrapper that disables the stdout capture/mod.

Example:

website_uri=$(terraform --nowrap output -raw website_uri)

@solarmosaic-kflorence
Copy link

When handling expected failures with terraform ... || terraform ... I wanted to ignore the core.setFailed due to the first command failing. The solution in #20 (comment) worked great (using terraform-bin instead of terraform just for the case that could potentially fail). Would be great to have the ability to run commands without the wrapper become an actual feature so it doesn't break at some point.

@slalomark
Copy link

I think it's rather incredible that this isn't mentioned in the Readme at all as well. Generating Terraform Outputs for use in later steps is a very likely use case in a CICD situation. Dumping an entire stdout and filtering through that to get your Output value is not really practical.

@schlomo
Copy link

schlomo commented Dec 28, 2022

FYI, to temporarily disable - unwrap - the wrapper, I use this Bash script fragment:

if type -p terraform-bin ; then
    echo "Unwrapping terraform wrapper :-)"
    shopt -s expand_aliases
    unalias -a
    alias terraform=terraform-bin
fi

@air3ijai
Copy link

#167 (comment)

terraform-bin output instance_id

ishuar added a commit to ishuar/terraform-ansible-azure that referenced this issue Aug 12, 2023
ishuar added a commit to ishuar/terraform-ansible-azure that referenced this issue Aug 12, 2023
- refer to hashicorp/setup-terraform#20

Signed-off-by: ishuar <ishansharma887@gmail.com>
ishuar added a commit to ishuar/terraform-ansible-azure that referenced this issue Aug 13, 2023
- refer to hashicorp/setup-terraform#20

Signed-off-by: ishuar <ishansharma887@gmail.com>
OJFord added a commit to OJFord/setup-terraform that referenced this issue Oct 25, 2023
Presently using a command such as `terraform output -json | jq` does not
work with the wrapper enabled, as it is by default.

In order to consume terraform's output having set it up with this
Action, it is necessary either to disable the wrapper (`with:
terraform_wrapper: false`) or run it in its own Actions step with an
explicit `id` (e.g. `id: foo`) so that it can be referred to and consumed
(`${{steps.foo.outputs.stdout}}` et al.) in later steps.

This seems to be the result of much confusion (issues passim) and is not
at all easy (hashicorp#338) to debug/diagnose and come to the realisation that
it's due to the wrapper, or even that such a thing exists.

This commit aims to address the issue by passing through stdout &
stderr, so that they're available in Unix pipelines and variable
assignment etc. as expected within a single step, while still retaining
the wrapper's listening behaviour to provide them as Actions outputs.

Closes hashicorp#20, hashicorp#80, hashicorp#85, hashicorp#149, hashicorp#338, and probably more.
OJFord added a commit to OJFord/setup-terraform that referenced this issue Oct 25, 2023
Presently using a command such as `terraform output -json | jq` does not
work with the wrapper enabled, as it is by default.

In order to consume terraform's output having set it up with this
Action, it is necessary either to disable the wrapper (`with:
terraform_wrapper: false`) or run it in its own Actions step with an
explicit `id` (e.g. `id: foo`) so that it can be referred to and consumed
(`${{steps.foo.outputs.stdout}}` et al.) in later steps.

This seems to be the result of much confusion (issues passim) and is not
at all easy (hashicorp#338) to debug/diagnose and come to the realisation that
it's due to the wrapper, or even that such a thing exists.

This commit aims to address the issue by passing through stdout &
stderr, so that they're available in Unix pipelines and variable
assignment etc. as expected within a single step, while still retaining
the wrapper's listening behaviour to provide them as Actions outputs.

Closes hashicorp#20, hashicorp#80, hashicorp#85, hashicorp#149, hashicorp#338, and probably more.
OJFord added a commit to OJFord/setup-terraform that referenced this issue Oct 26, 2023
Presently using a command such as `terraform output -json | jq` does not
work with the wrapper enabled, as it is by default.

In order to consume terraform's output having set it up with this
Action, it is necessary either to disable the wrapper (`with:
terraform_wrapper: false`) or run it in its own Actions step with an
explicit `id` (e.g. `id: foo`) so that it can be referred to and consumed
(`${{steps.foo.outputs.stdout}}` et al.) in later steps.

This seems to be the result of much confusion (issues passim) and is not
at all easy (hashicorp#338) to debug/diagnose and come to the realisation that
it's due to the wrapper, or even that such a thing exists.

@austinvalle identified the issue as being due to the `@actions/exec`
package writing the spawned command to stdout (along with then its
actual stdout). This has previously been reported upstream in
actions/toolkit#649; I've proposed actions/toolkit#1573 to fix it.

This commit aims to address the issue for `setup-terraform` in the
meantime by silencing `@actions/exec` and then writing out to stdout &
stderr from the listener buffers, which it writes to without this
additional logging.

Closes hashicorp#20, hashicorp#80, hashicorp#85, hashicorp#149, hashicorp#338, and probably more.
OJFord added a commit to OJFord/setup-terraform that referenced this issue Oct 26, 2023
Presently using a command such as `terraform output -json | jq` does not
work with the wrapper enabled, as it is by default.

In order to consume terraform's output having set it up with this
Action, it is necessary either to disable the wrapper (`with:
terraform_wrapper: false`) or run it in its own Actions step with an
explicit `id` (e.g. `id: foo`) so that it can be referred to and consumed
(`${{steps.foo.outputs.stdout}}` et al.) in later steps.

This seems to be the result of much confusion (issues passim) and is not
at all easy (hashicorp#338) to debug/diagnose and come to the realisation that
it's due to the wrapper, or even that such a thing exists.

@austinvalle identified the issue as being due to the `@actions/exec`
package writing the spawned command to stdout (along with then its
actual stdout). This has previously been reported upstream in
actions/toolkit#649; I've proposed actions/toolkit#1573 to fix it.

This commit aims to address the issue for `setup-terraform` in the
meantime by silencing `@actions/exec` and then writing out to stdout &
stderr from the listener buffers, which it writes to without this
additional logging.

Closes hashicorp#20, hashicorp#80, hashicorp#85, hashicorp#149, hashicorp#338, and probably more.
austinvalle pushed a commit to OJFord/setup-terraform that referenced this issue Oct 27, 2023
Presently using a command such as `terraform output -json | jq` does not
work with the wrapper enabled, as it is by default.

In order to consume terraform's output having set it up with this
Action, it is necessary either to disable the wrapper (`with:
terraform_wrapper: false`) or run it in its own Actions step with an
explicit `id` (e.g. `id: foo`) so that it can be referred to and consumed
(`${{steps.foo.outputs.stdout}}` et al.) in later steps.

This seems to be the result of much confusion (issues passim) and is not
at all easy (hashicorp#338) to debug/diagnose and come to the realisation that
it's due to the wrapper, or even that such a thing exists.

@austinvalle identified the issue as being due to the `@actions/exec`
package writing the spawned command to stdout (along with then its
actual stdout). This has previously been reported upstream in
actions/toolkit#649; I've proposed actions/toolkit#1573 to fix it.

This commit aims to address the issue for `setup-terraform` in the
meantime by silencing `@actions/exec` and then writing out to stdout &
stderr from the listener buffers, which it writes to without this
additional logging.

Closes hashicorp#20, hashicorp#80, hashicorp#85, hashicorp#149, hashicorp#338, and probably more.
austinvalle added a commit that referenced this issue Oct 27, 2023
* Fix output malformed when wrapper enabled

Presently using a command such as `terraform output -json | jq` does not
work with the wrapper enabled, as it is by default.

In order to consume terraform's output having set it up with this
Action, it is necessary either to disable the wrapper (`with:
terraform_wrapper: false`) or run it in its own Actions step with an
explicit `id` (e.g. `id: foo`) so that it can be referred to and consumed
(`${{steps.foo.outputs.stdout}}` et al.) in later steps.

This seems to be the result of much confusion (issues passim) and is not
at all easy (#338) to debug/diagnose and come to the realisation that
it's due to the wrapper, or even that such a thing exists.

@austinvalle identified the issue as being due to the `@actions/exec`
package writing the spawned command to stdout (along with then its
actual stdout). This has previously been reported upstream in
actions/toolkit#649; I've proposed actions/toolkit#1573 to fix it.

This commit aims to address the issue for `setup-terraform` in the
meantime by silencing `@actions/exec` and then writing out to stdout &
stderr from the listener buffers, which it writes to without this
additional logging.

Closes #20, #80, #85, #149, #338, and probably more.

* add test for stdout with jq

* update test name

* remove debug lines and add changelog

* add additional note about the bug fix to wrapper

---------

Co-authored-by: Austin Valle <austinvalle@gmail.com>
@austinvalle
Copy link
Member

This issue will be fixed in an upcoming v3.0.0 release next week. Please see this comment for more info

Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.