Some env vars updates #175
Conversation
🙇🏼♂️ Let me know when you want feedback! |
|
Looks pretty good to me, another way to cut it would be:
|
|
What did you have planned for interpolating? |
|
Those headings sound great 👌🏻 For interpolating, I meant not interpolating at all 😅 I'm planning for that to actually be a section about how env vars are merged together, explaining how they override each other depending on what level you set them at. I've just added in a big lot of notes on how this works after talking with @sj26. Will clean those up next week ⚡️ |
Fixed merge conflict in env vars file: - added the new item to the vars table from the third party pr - alphabetised the env vars, as per existing pr - added section on how env vars work, as per existing pr
|
Merged in master and new env var table item from 3rd party pr 🎉 |
|
@sj26 could you please have a quick read over the 'How environment variables do the thing' section, and let me know if it reflects your ideas about the process? I didn't change a whole lot from our conversation the other day, but it would be good to double check the overall thing 😊 |
| </tr> | ||
| <tr> | ||
| <th><code>BUILDKITE_BUILD_ID</code></th> | ||
| <td>The UUID of the original build this was rebuilt from. <p class="Docs__api-param-eg"><em>Example:</em> <code>"4735ba57-80d0-46e2-8fa0-b28223a86586"</code>, or <code>""</code> if not a rebuild.</p></td> |
There was a problem hiding this comment.
Oops, maybe BUILDKITE_BUILD_ID and BUILDKITE_REBUILT_FROM_BUILD_ID have been mixed up here?
There was a problem hiding this comment.
Hah, some copy pasta most likely. Should it just be 'The UUID of the build'?
| </tr> | ||
| <tr> | ||
| <th><code>BUILDKITE_PULL_REQUEST_REPO</code></th> | ||
| <td>The repository URL of the third-party pull request. <p class="Docs__api-param-eg"><em>Example:</em> <code>"git://github.com/acme-inc/my-project.git"</code>, or <code>""</code> if not a pull request.</p></td> |
There was a problem hiding this comment.
The repository URL of the third-party pull request
I'm pretty sure this is present for all pull requests, not just third-party ones
| </tr> | ||
| <tr> | ||
| <th><code>BUILDKITE_REBUILT_FROM_BUILD_NUMBER</code></th> | ||
| <td>The UUID of the original build this was rebuilt from. <p class="Docs__api-param-eg"><em>Example:</em> <code>"1514"</code>, or <code>""</code> if not a rebuild.</p></td> |
There was a problem hiding this comment.
Hmm I'm not sure I understand. What's the difference between a build number and a build UUID?
| </table> | ||
|
|
||
| Bootstrap vars: | ||
| BUILDKITE_DISABLE_GIT_SUBMODULES | ||
| BUILDKITE_REFSPEC |
There was a problem hiding this comment.
BUILDKITE_REFSPEC can be supplied from the api, ui, or wherever triggers the build, or any hook before checkout, to override how a build's source code is fetched in the bootstrap using git fetch origin $BUILDKITE_REFSPEC.
https://github.com/buildkite/agent/blob/master/bootstrap/bootstrap.go#L648-L658
There was a problem hiding this comment.
So is this purely for the branch name? Or can you put anything in that var?
There was a problem hiding this comment.
Um, not really for plain branch names, no — that's a more common scenario and takes a different code path. This variable was added for a specific customer who was using phabricator or something and wanted very specific control over what git fetched code to run builds against, so we let them specify it using this variable. It could be just a branch name (but you'd probably just use BUILDKITE_BRANCH for that) but is more likely for custom refspecs like +refs/weird/123abc:refs/local/weird/456 or something. I'm almost tempted to say this variable shouldn't be documented — it was only for a very specific use case.
| </table> | ||
|
|
||
| Bootstrap vars: | ||
| BUILDKITE_DISABLE_GIT_SUBMODULES |
There was a problem hiding this comment.
BUILDKITE_DISABLE_GIT_SUBMODULES can be supplied from the api, ui, or wherever triggers the build, or any hook before checkout to disable fetching any git submodules during checkout.
| - more details on setting vars in a pipeline.yml: | ||
| - there are two places in a pipeline.yml file that you can set env vars: | ||
| - in the `env` attribute of command or trigger steps | ||
| - at the top of the yaml file, before you define your pipeline's steps (this will set the defined variable on each of the command steps in the pipeline, and is equivalent to setting the `env` attribute on every step) |
There was a problem hiding this comment.
There's one caveat to this — shared env (at the top level of the pipeline.yml) currently overrides the step level env. So:
env:
- FOO=bar
steps:
- ...
env:
- FOO=bazIntuitively it looks like the job should be run with FOO=baz, but currently it is run with FOO=bar. We're thinking about changing it from this to to what you'd expect so that FOO=baz wins.
There was a problem hiding this comment.
Oh! Right. Is there a vague eta for the decision about that possible change? If it's going to be a while off, I'll publish this telling people about how it overrides now. If it's imminent, I'll hold off until after we've changed it or we decide we don't want to 👌🏻
There was a problem hiding this comment.
Maybe @lox would have a better idea?
|
|
||
| - these are merged together in the following way: | ||
| (maybe put an image or example code here) | ||
| bk env (most important, not overridden by the following bits) > build > step > project (least important, can be overridden by everything else) |
There was a problem hiding this comment.
worth noting here maybe that step = step or pipeline.yml, even if it feels like pipeline uploads should override the build env
|
|
||
| - more details on setting vars in a Trigger step: | ||
| - env vars set in a trigger step end up as build-level env on the newly created build | ||
| - NOTE: the 'all steps env' from the top of a pipeline.yaml isn't applied to trigger steps, the only env that applies to trigger steps is what you choose to set in that trigger step's `env` attribute (which ends up as build env in the new build) |
There was a problem hiding this comment.
Maybe worth adding that you can copy env to the trigger env using env interpolation from agent 3:
env:
- FOO=bar
steps:
- trigger: some-pipeline
build:
env:
- FOO=$FOO # triggered build will have `FOO=bar`and that trigger env becomes "build env" in the triggered build
| - any environment variables that were inherited from how you started the process (i.e. systemd sets some env vars for you) | ||
|
|
||
| For a list of variables/flags you can set on your agent, see the [`buildkite-agent start` page](/docs/agent/cli-start). | ||
| For a deeper look at what's involved in starting the agent, you can see the the [`buildkite-agent start` code on github](https://github.com/buildkite/agent/blob/master/clicommand/agent_start.go). |
There was a problem hiding this comment.
Maybe worth explicitly mentioning that the agent environment influences how the agent runs your job, but does not get directly copied into the job's environment.
There was a problem hiding this comment.
Oh, I see it's kinda covered in the next section.
| - some vars are created fresh at this point | ||
| - describe the agent that the job is running on | ||
| - vars to tell the bootstrap and other prebaked commands what to do | ||
| - e.g. we don't give the job environment the agent's registration token, we instead create a new access token for this specific job |
There was a problem hiding this comment.
👍 this is something which often trips folks up
|
Looking good, @harrietgrace! |
|
@keithpitt Could I please get you to have a quick squiz at this? Sam helped me sort out what was going on in each section, I just wanted to get an extra set of eyes on it before I merge it 😊Mostly looking for feedback on formatting, information organisation, and spelling. But content feedback is welcome too! Screenshots of the new section from dev (I've also alphabetised the vars table at the top, but this is the main bit of the pr):
|
|
@harrietgrace I don't think we use the word |
|
@keithpitt yep it should totally be pipeline! How did that even sneak in... 😅 |
Alphabetising the env vars, because it kills me every time I look at this page that they're not in order. Also, makes it much easier for people to find what they're looking for 💁🏻
Possibly adding some more things to this page, so leaving this PR as a WIP for the moment.
Later: