Breakup builds.adoc into subdir #3739

adellape · 2017-02-13T19:42:28Z

Trello card: https://trello.com/c/sG2J1yry/413-builds-section-improvements

Latest WIP preview:

http://file.rdu.redhat.com/~adellape/021417/buildsfinally/dev_guide/builds/

Open PRs to consolidate:

Built image digest is stored in the build status #3533 (Handled via Built image digest stored in build status #3772)

bfallonf · 2017-02-14T02:00:41Z

_topic_map.yml

+        File: build_strategies
+      - Name: Triggering Builds
+        File: triggering_builds
+      - Name: Secrets During Builds


Can I suggest something like "Using Secrets with Builds"?

Yeah "Secrets During Builds" is weird. "Using Secrets with Builds" is what it was originally, was trying to come up with something different... Placeholder for now. I bet it'll just end up back at "Using Secrets During Builds" ultimately.

bfallonf · 2017-02-14T02:02:40Z

_topic_map.yml

+        File: how_builds_work
+      - Name: Basic Build Operations
+        File: basic_build_operations
+      - Name: Build Inputs and Output


I'm still not sure what this section is about... Maybe something like "Build Plug-ins" or "Using Builds with Outside Sources" or something. It'd help if I fully understood what the topic was about. Sorry :|

It's about the type of inputs that you can feed into a build and the output that result from it.

bfallonf · 2017-02-14T02:03:44Z

@adellape I agree with this all. I made some suggestions for the titles of the topics. I know it's largely copied over from the existing stuff, but it'd be a good chance to make them a bit more descriptive.

adellape · 2017-02-14T17:19:43Z

Updated preview:

http://file.rdu.redhat.com/~adellape/021417/buildsfinally/dev_guide/builds/

@bparees Can you take a look at how this is shaping up?

adellape · 2017-02-14T17:21:07Z

Builds section improvements

bparees · 2017-02-14T17:36:47Z

setting a maximum duration seems closely related to setting build resources, so i'd consider moving the build resources section out of "how builds work" and into "basic build operations". Both topics could also be moved into the "advanced usage" section instead.

on inputs/outputs i'd like to see the proxy+secrets subsections linked in the index at the top

build-secrets are really just another build input, so that content should probably be refactored from here:
http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_secrets.html#using-secrets-during-build
to here:
http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_inputs_output.html

of course that leaves just the docker push/pull secrets content in the "build secrets" topic which leaves me thinking that topic needs to be refactored.

there's really 3 separate discussions of secrets in the build docs:

using a source secret to access a git repo
using a secret to push + pull images
using a secret within your dockerfile/assemble logic (using a secret as an input, essentially)

You've put (1) under build inputs and 2+3 under "build secrets" but 2+3 aren't really related to each other any more than they are to (1).

(there's actually a 4th discussion of secrets too, in the custom strategy options section. That use of secrets is most closely related to (3), but the mechanism is slightly different)

adellape · 2017-02-14T21:30:23Z

@bparees Thanks for looking. Inline:

setting a maximum duration seems closely related to setting build resources, so i'd consider moving the build resources section out of "how builds work" and into "basic build operations". Both topics could also be moved into the "advanced usage" section instead.

Both moved to "Advanced Build Operations".

on inputs/outputs i'd like to see the proxy+secrets subsections linked in the index at the top

I've now set :toclevels: 3 for that file so "Using a Proxy" and "Source Secrets" are now listed in the top TOC. Is that what you meant? Alternatively, I could pull all the input/source sections up a level and re-title the "Build Inputs" section to something like "Build Inputs Overview".

build-secrets are really just another build input, so that content should probably be refactored from here:
http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_secrets.html#using-secrets-during-build
to here:
http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_inputs_output.html

of course that leaves just the docker push/pull secrets content in the "build secrets" topic which leaves me thinking that topic needs to be refactored.

there's really 3 separate discussions of secrets in the build docs:

using a source secret to access a git repo
using a secret to push + pull images
using a secret within your dockerfile/assemble logic (using a secret as an input, essentially)
You've put (1) under build inputs and 2+3 under "build secrets" but 2+3 aren't really related to each other any more than they are to (1).

(there's actually a 4th discussion of secrets too, in the custom strategy options section. That use of secrets is most closely related to (3), but the mechanism is slightly different)

Would it be fine to move all of what's currently in the "Build Secrets" topic into "Inputs and Output" (therefore 1, 2, and 3 in your list are all in that topic at that point)? Maybe pull the "* Source" sections up a section-level... The reason I put "Inputs and Output" together was because it didn't seem like there was enough for output to warrant its own topic, but at this point it's seeming like there will be a lot of content in here for inputs, so probably worth splitting them apart after all.

bparees · 2017-02-14T21:56:56Z

I've now set :toclevels: 3 for that file so "Using a Proxy" and "Source Secrets" are now listed in the top TOC. Is that what you meant?

yup that's what i meant.

Alternatively, I could pull all the input/source sections up a level and re-title the "Build Inputs" section to something like "Build Inputs Overview".

i think it's fine as is for now, but splitting inputs and outputs into separate top level pages is certainly something to consider in the future if things continue to grow.

Would it be fine to move all of what's currently in the "Build Secrets" topic into "Inputs and Output"

yes. Incidentally while skimming that Build Secrets section I see we have a terminology problem. We use "source secrets" elsewhere to mean "a secret used to access your source(git repo)". Then in the Build Secrets section we use "source secrets" to mean "a secret that is provided as a source input".

We should disambiguate that. When referring to a secret that's being used a source input, maybe we should call it an input secret or something. Not sure.

It's extra difficult since the first one is specified via spec.source.sourceSecrets(used to access git) and the second one is in spec.source.secrets(provided to the build to be used for whatever the build wants to do with it)

The reason I put "Inputs and Output" together was because it didn't seem like there was enough for output to warrant its own topic, but at this point it's seeming like there will be a lot of content in here for inputs, so probably worth splitting them apart after all.

i'm ok either way.

adellape · 2017-02-14T22:24:49Z

dev_guide/builds/index.adoc

+`BuildConfig` is typically generated automatically for you if you use the web
+console or CLI, and it can be edited at any time. Understanding the parts that
+make up a `BuildConfig` and their available options can help if you choose to
+manually tweak your configuration later.


@bparees I added this 2nd paragraph here and renamed the section from "Defining a BuildConfig" -> "What Is a BuildConfig?" because it initially seemed potentially like we were expecting you to go craft a BuildConfig from scratch before doing anything w/ builds. Hopefully this makes it a little clearer what the expectations are. Any suggestions on this?

works for me.

bparees · 2017-02-14T22:30:35Z

dev_guide/builds/index.adoc

+xref:build_inputs_output.adoc#binary-source[how it is specified to the system].
+
+[[defining-a-buildconfig]]
+== What Is a BuildConfig?


lowercase i or uppercase a?

This is how title case says to do it, apparently. 🤷‍♂️

ick. but ok :)

adellape · 2017-02-14T23:22:31Z

@bparees How's these?

http://file.rdu.redhat.com/~adellape/021417/buildsfinally/dev_guide/builds/build_inputs.html
http://file.rdu.redhat.com/~adellape/021417/buildsfinally/dev_guide/builds/build_output.html

adellape · 2017-02-14T23:23:37Z

dev_guide/builds/build_inputs.adoc

+endif::[]
+
+[[source-clone-secrets]]
+=== Source Clone Secrets


@bparees I noticed in dev_guide/secrets.adoc we referred to these as "source clone secrets". That alright? Or is the clone part of it too narrow of a name.

Other maybes: Git Source Secrets, Git Repo Secrets...

clone is reasonable, that's really what we're using it for...the git clone operation. it's just a bit confusing since the api doesn't mention "clone".

adellape · 2017-02-14T23:23:52Z

dev_guide/builds/build_inputs.adoc

+
+
+[[using-secrets-during-build]]
+== Input Secrets


@bparees Went with "Input Secrets" here.

adellape · 2017-02-14T23:24:54Z

dev_guide/builds/build_inputs.adoc

+[[using-secrets-s2i-strategy]]
+=== Source-to-Image Strategy
+
+When using a `Source` strategy, all defined source secrets are copied to their


@bparees In this and the next 2 subsections, the instances of "source secrets" should become "input secrets", yes?

bparees · 2017-02-14T23:40:40Z

input secrets needs to be mentioned in the bullet list in the overview of the build inputs topic (and i guess that whole paragraph should be reworked to include what happens w/ input secrets).

on the output side, http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_output.html#output-image-environment-variables should also make mention of the fact that any user defined env vars from things like http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_strategies.html#configuring-the-source-environment and http://file.rdu.redhat.com/%7Eadellape/021417/buildsfinally/dev_guide/builds/build_strategies.html#docker-strategy-environment will also be part of the output image environment variable list.

(the strategy sections do already note this, saying " The environment variables defined there are visible during the assemble script execution and will be defined in the output image, making them also available to the run script and application code." and "The variables are defined during build and stay in the output image, therefore they will be present in any container that runs that image as well.")

adellape · 2017-02-14T23:44:08Z

@bfallonf @ahardin-rh @bmcelvee @aheslin @mburke5678 I think I've touched enough of this by now to call it a rough first draft, feel free to peer review. Be warned I haven't done a comprehensive link check yet (I've updated all links inside these files but haven't gone through the rest of the repo to fix everything that links back to the old builds.adoc).

Preview build (may need to refresh if you've linked here previously):

http://file.rdu.redhat.com/~adellape/021417/buildsfinally/dev_guide/builds/

bparees · 2017-02-14T23:46:25Z

this is great stuff @adellape

bfallonf · 2017-02-15T00:47:54Z

dev_guide/builds/index.adoc

+      generic:
+        secret: "secret101"
+    -
+      type: "ImageChange"


The "type" stuff is a little screwy here, format-wise.

I think this YAML formatting come up before with @bparees... IIRC, this is how stuff gets reformatted in OpenShift. Or at least, OpenShift will accept this form. Think I'll leave any potential changes along these lines to another PR.

mburke5678 · 2017-02-15T01:42:08Z

Much better than the original @adellape ! Will take a closer look.

bfallonf · 2017-02-15T03:43:30Z

dev_guide/builds/build_troubleshooting.adoc

+----
+$ oc describe quota
+----
+|===


I know this is like this currently in the docs, but would it be better to take this out of a table and into an actual section? I think a table with just one thing is a little redundant, plus, having it like this isn't really consistent with the rest of the docs.

yeah it does look fairly weird today.

Agreed, though I couldn't remember if someone had already taken it from section -> table, and I'd be putting it back into sections again. But I do think the table is weird, I'll change it.

adellape · 2017-02-15T20:45:44Z

dev_guide/builds/build_inputs.adoc

+toc::[]
+
+[[how-build-inputs-work]]
+== How Build Inputs Work


Re-titled "Providing Source for a Build" -> "How Build Inputs Work", since "Providing" made it seem like there was actually a task in here, which there is not.

adellape · 2017-02-15T20:50:32Z

dev_guide/builds/build_inputs.adoc

+Input secrets are useful for when you do not want certain resources or
+credentials used during a build to be available in the final application image
+produced by the build. External artifacts outside of a source repository can
+also be used to pull in additional files during a build.


@bparees I added "Input secrets" and "External artifacts" to the above list. Some questions:

Is it problematic to try adding these 2 to the list since it's supposed to be "In order of precedence"? Where would they fall, or is that not really helpful?

I reordered the topic's later section order to match the precedence list above. I assume we had Git Source first previously because it's the most common. Any issues w/ reordering?

I added this above paragraph to touch briefly on input secrets and external artifacts since they were new additions and the other 4 had gotten touched on in the preceding paragraph. Any issues w/ the summary?

(1) slightly. using precedence was already problematic though... (eg git source and binary source are mutually exclusive, so there's no applicable "precedence" there).

(2) it's not just the most common but also the most universally applicable (having an inline dockerfile w/ an s2i build doesn't usually make sense. having a git repo always makes sense), but i think keeping the ordering symmetric is sensible.

(3) slight rewordings/augmentation:
"Input secrets are useful for when you do not want certain resources or
credentials used during a build to be available in the final application image
produced by the build or want to consume a value that is defined in a Secret resource. External artifacts can be used to pull in additional files that are not available as one of the other input types."

adellape · 2017-02-15T20:50:56Z

dev_guide/builds/build_inputs.adoc

+. The content from the current directory is provided to the build process
+for reference by the Dockerfile, *_assemble_* script, or custom builder logic.
+This means any input content that resides outside the `contextDir` will be
+ignored by the build.


@bparees Do we need to try adding in mention of build inputs and external artifacts into this numbered list?

no. external artifacts wouldn't be relevant here and i think "input content" covers the rest sufficiently.

adellape · 2017-02-15T20:51:23Z

dev_guide/builds/build_inputs.adoc

+`$BUILD` environment variable, which includes the full build object.
+
+[[using-external-artifacts]]
+== Using External Artifacts


Dropped the "During a Build" part of this section title since it was redundant in context.

adellape · 2017-02-15T20:52:00Z

dev_guide/builds/build_inputs.adoc

+
+
+[[using-docker-credentials-for-private-registries]]
+== Using Docker Credentials for Private Registries


@bparees Any issues with this re-title? It was previously "Using Docker Credentials for Pushing and Pulling Images", which seemed overlong.

Plus, given the [NOTE] we have about how it's not required for internal registry use, it seemed to make sense to clarify in the section title that this is for private registry use.

looks fine.

adellape · 2017-02-15T20:54:58Z

dev_guide/builds/index.adoc

+- xref:build_inputs.adoc#binary-source[Binary]
+- xref:build_inputs.adoc#image-source[Image]
+- xref:build_inputs.adoc#using-secrets-during-build[Input secrets]
+- xref:build_inputs.adoc#using-external-artifacts[External artifcats]


Added input secrets and external artifacts here.

adellape · 2017-02-16T17:24:01Z

@bparees I'm trying to land these general changes for Origin + 3.4, but also making sure that merged 3.5-content doesn't get into 3.4 too early.

I went back through the dev_guide/builds.adoc commit history back through to October, and I've identified two PRs (and their follow-ups) for 3.5 content currently merged in master:

Using URLs with start-build --from-file/--from-dir (#3184)
Add documentation for 'build.openshift.io/source-secret-match-uri-' annotation (#3465)

I've removed that related content from this PR, and will make a follow-up PR after this one merges containing the 3.5+ stuff, merge that to master, then label that new PR for inclusion in OCP starting with 3.5 GA. 😵

So my question:

#3184 was easy enough to cut out. Can you confirm that what I've cut out for the #3465 changes is fine? That's currently in this separate commit: 9249ed6. Basically I removed the "Automatically Adding Source Clone Secrets" subsection, and left the "Manually Adding Source Clone Secrets" as-is, which still seemed accurate AFAICT.

bparees · 2017-02-16T18:21:27Z

@adellape yup that looks right.

adellape · 2017-02-16T21:53:42Z

@bmcelvee @ahardin-rh @mburke5678 @bfallonf @vikram-redhat @aheslin @bparees Please be aware that this has merged to master, so any new work going forward related to "Builds" should now go into a dev_guide/builds/ topic.

This will show up in the Origin docs after the nightly auto-builds later tonight, and I'll take care of cherry-picking/publishing to OCP 3.4, Online, and Dedicated in the morning (RDU time). I'll send a note out to some lists when those are published and redirects from the old builds.adoc are in place.

bparees · 2017-02-16T21:56:11Z

@openshift/devex fyi, our docs have been significantly refactored.

adellape · 2017-02-16T21:58:23Z

Preview build can be found here:

https://ci.openshift.redhat.com/openshift-docs-master-testing/latest/dev_guide/builds/

adellape · 2017-02-17T16:37:38Z

There is some Customer Portal maintenance going on today which is blocking doc publishing there, so I'm holding off publishing this in OCP 3.4 docs (on both docs.openshift and the Customer Portal) until that's settled down.

adellape added the branch/enterprise-3.4 label Feb 13, 2017

adellape added this to the Next Release milestone Feb 13, 2017

adellape changed the title ~~Breakup builds.adoc into subdir~~ [WIP] Breakup builds.adoc into subdir Feb 13, 2017

bfallonf reviewed Feb 14, 2017

View reviewed changes

adellape mentioned this pull request Feb 14, 2017

Built image digest is stored in the build status #3533

Closed

adellape force-pushed the buildsfinally branch from 076d083 to c9972a6 Compare February 14, 2017 17:16

adellape force-pushed the buildsfinally branch from c9972a6 to 3972db7 Compare February 14, 2017 17:35

adellape force-pushed the buildsfinally branch from 3972db7 to 1d5bf64 Compare February 14, 2017 21:22

adellape commented Feb 14, 2017

View reviewed changes

bparees reviewed Feb 14, 2017

View reviewed changes

adellape force-pushed the buildsfinally branch from da35102 to ebfb0aa Compare February 14, 2017 23:21

adellape commented Feb 14, 2017

View reviewed changes

adellape force-pushed the buildsfinally branch from ebfb0aa to 68cf83f Compare February 14, 2017 23:39

adellape force-pushed the buildsfinally branch from 68cf83f to b479e17 Compare February 14, 2017 23:43

bfallonf reviewed Feb 15, 2017

View reviewed changes

adellape force-pushed the buildsfinally branch 2 times, most recently from 0fb8cec to 114f936 Compare February 15, 2017 20:43

adellape commented Feb 15, 2017

View reviewed changes

adellape force-pushed the buildsfinally branch 3 times, most recently from 754ad24 to 9249ed6 Compare February 16, 2017 17:12

adellape force-pushed the buildsfinally branch from 9249ed6 to 7bfe83b Compare February 16, 2017 19:02

ahardin-rh approved these changes Feb 16, 2017

View reviewed changes

adellape force-pushed the buildsfinally branch from 7bfe83b to a024e92 Compare February 16, 2017 20:48

Breakup builds.adoc into subdir

79cae27

adellape force-pushed the buildsfinally branch from a024e92 to 79cae27 Compare February 16, 2017 21:03

adellape added branch/dedicated branch/online labels Feb 16, 2017

adellape merged commit 6e40699 into openshift:master Feb 16, 2017

adellape mentioned this pull request Feb 16, 2017

Temp rm PR#3184/3657 #3766

Merged

adellape modified the milestones: Builds Re-org, Next Release Feb 16, 2017

adellape mentioned this pull request Feb 16, 2017

Re-add 3.5+ builds content (start-build URLs + source-secret-match-uri) #3767

Merged

adellape deleted the buildsfinally branch November 9, 2017 20:19



		[[using-docker-credentials-for-private-registries]]
		== Using Docker Credentials for Private Registries

Breakup builds.adoc into subdir #3739

Breakup builds.adoc into subdir #3739

Conversation

adellape commented Feb 13, 2017 • edited Loading

Choose a reason for hiding this comment

adellape Feb 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bfallonf commented Feb 14, 2017

adellape commented Feb 14, 2017

adellape commented Feb 14, 2017

bparees commented Feb 14, 2017

adellape commented Feb 14, 2017 • edited Loading

bparees commented Feb 14, 2017

adellape Feb 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adellape commented Feb 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bparees commented Feb 14, 2017

adellape commented Feb 14, 2017 • edited Loading

bparees commented Feb 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mburke5678 commented Feb 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adellape commented Feb 16, 2017

bparees commented Feb 16, 2017

adellape commented Feb 16, 2017

bparees commented Feb 16, 2017

adellape commented Feb 16, 2017 • edited Loading

adellape commented Feb 17, 2017

adellape commented Feb 13, 2017 •

edited

Loading

adellape Feb 14, 2017 •

edited

Loading

adellape commented Feb 14, 2017 •

edited

Loading

adellape Feb 14, 2017 •

edited

Loading

adellape commented Feb 14, 2017 •

edited

Loading

adellape commented Feb 16, 2017 •

edited

Loading