ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar #62970

pietroalbini · 2019-07-25T12:17:49Z

This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new TOOLSTATE_PUBLISH environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk.

For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load:

- bash: echo foo
  env:
    SECRET_VAR: $(SECRET_VAR)

This works fine when the variable is present but when it's missing, instead of setting SECRET_VAR to an empty string or just not setting it at all, Azure Pipelines puts the literal $(SECRET_VAR) as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing:

# At the top:
variables:
  - name: MAYBE_SECRET_VAR
    value: $[ variables.MAYBE_SECRET_VAR ]

# In the step:
- bash: echo foo
  env:
    SECRET_VAR: $(MAYBE_SECRET_VAR)

While that could've worked it was ugly and messy, so I just opted to add yet another non-secret variable.

r? @alexcrichton
fixes #62811

Unfortunately due to an Azure quirk the TOOLSTATE_REPO_ACCESS_TOKEN is not suitable to gate whether to push new commits to the repo, as if it's not defined on the Azure side it will actually be set to the literal `$(TOOLSTATE_REPO_ACCESS_TOKEN)`, which screws everything up. This instead adds another, non-secret environment variable to gate publishing: TOOLSTATE_PUBLISH. As non-secret environment variables behave correctly this fixes the issue.

alexcrichton · 2019-07-25T14:06:20Z

@bors: r+ rollup

I'm saying rollup b/c it's probably not necessary to get this in ASAP, but feel free to tweak priority as you see fit.

bors · 2019-07-25T14:06:21Z

📌 Commit b01b5b9 has been approved by alexcrichton

@alexcrichton

…lexcrichton ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk. For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load: ```yaml - bash: echo foo env: SECRET_VAR: $(SECRET_VAR) ``` This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing: ```yaml # At the top: variables: - name: MAYBE_SECRET_VAR value: $[ variables.MAYBE_SECRET_VAR ] # In the step: - bash: echo foo env: SECRET_VAR: $(MAYBE_SECRET_VAR) ``` While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable. r? @alexcrichton fixes rust-lang#62811

@alexcrichton

…lexcrichton ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk. For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load: ```yaml - bash: echo foo env: SECRET_VAR: $(SECRET_VAR) ``` This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing: ```yaml # At the top: variables: - name: MAYBE_SECRET_VAR value: $[ variables.MAYBE_SECRET_VAR ] # In the step: - bash: echo foo env: SECRET_VAR: $(MAYBE_SECRET_VAR) ``` While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable. r? @alexcrichton fixes rust-lang#62811

@ghost

Rollup of 14 pull requests Successful merges: - #62084 (allow clippy::unreadable_literal in unicode tables) - #62421 (Introduce `as_deref` to Option) - #62692 (rustc: precompute the largest Niche and store it in LayoutDetails.) - #62801 (Remove support for -Zlower-128bit-ops) - #62828 (Remove vector fadd/fmul reduction workarounds) - #62862 (code cleanup) - #62897 (Attempt to fix backtrace tests on i686-msvc) - #62904 (Disable d32 on armv6 hf targets) - #62907 (Initialize the MSP430 AsmParser) - #62956 (Implement slow-path for FirstSets::first) - #62963 (Allow lexer to recover from some homoglyphs) - #62970 (ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar) - #62983 (Remove needless indirection through Rc) - #62985 (librustc_errors: Support ui-testing flag in annotate-snippet emitter) Failed merges: r? @ghost

@alexcrichton

…lexcrichton ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk. For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load: ```yaml - bash: echo foo env: SECRET_VAR: $(SECRET_VAR) ``` This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing: ```yaml # At the top: variables: - name: MAYBE_SECRET_VAR value: $[ variables.MAYBE_SECRET_VAR ] # In the step: - bash: echo foo env: SECRET_VAR: $(MAYBE_SECRET_VAR) ``` While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable. r? @alexcrichton fixes rust-lang#62811

@alexcrichton

…lexcrichton ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar This PR fixes toolstate failing to push on the LinuxTools PR builder by gating the pushes on the new `TOOLSTATE_PUBLISH` environment variable, which is set on prod credentials but not on the PR ones. The old code checked whether the access token was set, but that doesn't work due to an Azure quirk. For a bit of background, secret environment variables are not available by default, but each step needs to explicitly declare which secret vars to load: ```yaml - bash: echo foo env: SECRET_VAR: $(SECRET_VAR) ``` This works fine when the variable is present but when it's missing, instead of setting `SECRET_VAR` to an empty string or just not setting it at all, Azure Pipelines puts the literal `$(SECRET_VAR)` as the content, which completly breaks the old check we had. I tried almost every thing to make this work in a sensible way, and the only conclusion I reached is to set the variable at the top level with the runtime expression evaluation syntax, which sets the variable to an empty string if missing: ```yaml # At the top: variables: - name: MAYBE_SECRET_VAR value: $[ variables.MAYBE_SECRET_VAR ] # In the step: - bash: echo foo env: SECRET_VAR: $(MAYBE_SECRET_VAR) ``` While that *could've worked* it was ugly and messy, so I just opted to add yet another non-secret variable. r? @alexcrichton fixes rust-lang#62811

@ghost

Rollup of 22 pull requests Successful merges: - #62084 (allow clippy::unreadable_literal in unicode tables) - #62120 (Add missing type links in documentation) - #62310 (Add missing doc links in boxed module) - #62421 (Introduce `as_deref` to Option) - #62583 (Implement Unpin for all raw pointers) - #62692 (rustc: precompute the largest Niche and store it in LayoutDetails.) - #62801 (Remove support for -Zlower-128bit-ops) - #62828 (Remove vector fadd/fmul reduction workarounds) - #62862 (code cleanup) - #62904 (Disable d32 on armv6 hf targets) - #62907 (Initialize the MSP430 AsmParser) - #62956 (Implement slow-path for FirstSets::first) - #62963 (Allow lexer to recover from some homoglyphs) - #62964 (clarify and unify some type test names) - #62970 (ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar) - #62980 (std: Add more accessors for `Metadata` on Windows) - #62983 (Remove needless indirection through Rc) - #62985 (librustc_errors: Support ui-testing flag in annotate-snippet emitter) - #63002 (error_index_generator should output stdout/stderr when it panics.) - #63004 (Add test for issue-54062) - #63007 (ci: debug network failures while downloading awscli from PyPI) - #63009 (Remove redundant `mut` from variable declaration.) Failed merges: r? @ghost

rust-highfive assigned alexcrichton Jul 25, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 25, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 25, 2019

Centril mentioned this pull request Jul 25, 2019

Rollup of 10 pull requests #62991

Closed

Centril mentioned this pull request Jul 26, 2019

Rollup of 14 pull requests #63001

Closed

Centril mentioned this pull request Jul 26, 2019

Rollup of 17 pull requests #63011

Closed

Centril mentioned this pull request Jul 26, 2019

Rollup of 22 pull requests #63015

Merged

bors merged commit b01b5b9 into rust-lang:master Jul 26, 2019

pietroalbini deleted the fix-tools-builder branch August 1, 2019 20:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar #62970

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar #62970

pietroalbini commented Jul 25, 2019 •

edited

Loading

alexcrichton commented Jul 25, 2019

bors commented Jul 25, 2019

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar #62970

ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar #62970

Conversation

pietroalbini commented Jul 25, 2019 • edited Loading

alexcrichton commented Jul 25, 2019

bors commented Jul 25, 2019

pietroalbini commented Jul 25, 2019 •

edited

Loading