Adding draft of blog post for SCM-API 2.0.x #540

Merged
merged 9 commits into from Jan 16, 2017

Projects

None yet

8 participants

+:author: stephenc
+---
+
+=== The back-story
@recampbell
recampbell Jan 11, 2017

This reads more like a History of Literate plugin than an advisory about potential upgrade problems.

I think we should start with something like "We've made some great improvements to the GitHub Branch Source & BitBucket branch source plugins, but there are some gotcha's that Jenkins administrators will need to be aware of.

We want to give you the whole story, but the TL;DR; is: When updating your plugins, please be sure to upgrade the GitHub Branch Source and/or Bitbucket branch source plugins. Doing partial upgrades is bad news!"

@reviewbybees

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

@PaddyOH
Contributor
PaddyOH commented Jan 11, 2017

Should the "tombstone" in "tombstone release" be lowercase T? It sounds like a proper noun. Also, I don't have an issue with deliberately European spelling, but "behaviour" with a "u" is European spelling; on this side of the pond, 'Mericans drop the u.

@stephenc
Contributor

@PaddyOH well it's a regular noun: http://www.dictionary.com/browse/tombstone and we are using that release as a marker that the plugin is dead... it would be different if the version actually included the word "tombstone"... at least IMHO.

I have a strong preference for spelling English in accordance with the OED. I have switched to a different word entirely to humour you and may you turn an off colour from the smells of sulphur leaking out of an aluminium can if you disagree ;-)

@bitwiseman

For the entire document:

  • s/https:/link:https:/
  • Put each link at the start of a new line.
  • split long lines into shorter ones for easier diff-ing and feedback.
  • replace === with ==
  • replace all current uses of ==== for blocks with ----
+:author: stephenc
+---
+
+=== TL;DR
@bitwiseman
bitwiseman Jan 11, 2017 Contributor

Remove this header. This is the introduction. You have the TL;DR in a tip below, that is enough.

+We are announcing the SCM API 2.0.x and Branch API 2.0.x release line.
+These plugin releases include:
+
+* https://github.com/jenkinsci/scm-api-plugin/blob/master/docs/implementation.adoc[documentation on how plugin developers are supposed to implement the SCM API]
@bitwiseman
bitwiseman Jan 11, 2017 Contributor
* https://github.com/jenkinsci/scm-api-plugin/blob/master/docs/implementation.adoc[documentation on how plugin developers are supposed to implement the SCM API]

to

* documentation on how developers are supposed to
link:https://github.com/jenkinsci/scm-api-plugin/blob/master/docs/implementation.adoc[implement the SCM API]
@bitwiseman
bitwiseman Jan 11, 2017 Contributor

Repeat for each item in the list.

+Ok, that was the good news... Here is the bad news...
+
+We found out that the GitHub Branch Source and BitBucket Branch Source plugins had made invalid assumptions about how to implement the SCM API.
+To be clean, this was not the plugin developers fault: at the time there was no documentation on how to implement the SCM API.
@bitwiseman
bitwiseman Jan 11, 2017 Contributor

s/clean/clear/

+We found out that the GitHub Branch Source and BitBucket Branch Source plugins had made invalid assumptions about how to implement the SCM API.
+To be clean, this was not the plugin developers fault: at the time there was no documentation on how to implement the SCM API.
+
+But fixing the issues that we found means that you have to be careful about the matrix of plugin versions that you have installed.
@bitwiseman
bitwiseman Jan 11, 2017 Contributor

s/about the matrix of plugin versions/about which specific combination of plugin versions/

@stephenc
Contributor

@bitwiseman said

replace all current uses of ==== for blocks with ----

those are Admonition blocks which are supposed to be ==== not ---- which is a Verbatim block

@stephenc
Contributor

@bitwiseman said

split long lines into shorter ones for easier diff-ing and feedback.

The recommended practice in Asciidoc is One Sentence Per Line as that gives the easiest diffing.
Spreading a longer sentence across multiple lines will only make the diff larger if you have to change the sentence as the wrapping will have side-effects when you have to correct

@bitwiseman
Contributor
bitwiseman commented Jan 11, 2017 edited

Admonitions: oops, my bad. Thanks for the correction.
Line length: Yeah, I've seen that guidance. That's good enough, but github comments go per line and so if you have a long sentence giving point-wise feedback becomes difficult.

+
+We want to give you the whole story, but the <<tldr,TL;DR>> is this
+
+TIP: When updating your plugins, please be sure to upgrade the GitHub Branch Source and/or Bitbucket branch source plugins.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

Everyone knows what tl;dr means, except those who don't. How about "the key points are:"

For the key points, be imperative. s/please be sure to//

+
+TIP: When updating your plugins, please be sure to upgrade the GitHub Branch Source and/or Bitbucket branch source plugins.
+
+WARNING: Doing partial upgrades is bad news!
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

What is a partial upgrade? What is bad news?
Be explicit and imperative:
"Do NOT upgrade some of these plugins but not others! Doing so may cause your Jenkins server to fail to start."

+* Subversion plugin
+* Mercurial plugin
+
+While we had some interest in the literate plugin, it did not gain much traction - there are only 39 Jenkins instances running the literate plugin as of December 2016.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/we had/there was/

+
+In terms of the reusable components, we had only made a minimal implementation with some limitations:
+
+* Very basic event support - every event just triggers a re-scan of the entire repository.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/triggers/triggered/

@stephenc
stephenc Jan 12, 2017 Contributor

No. The tense is correct... when running SCM API 1.0 every event will just trigger a re-scan. I have rephrased to clarify why the tense is correct

+
+* Very basic event support - every event just triggers a re-scan of the entire repository.
+This was acceptable at the time because the only three implementations use a local cache of the remote state so re-scanning is quick.
+* No implementation of the `SCMFileSystem` API for performing deep inspection of the remote repository without needing to checkout the repository into a workspace.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor
No implementation of the `SCMFileSystem` API for performing deep inspection of the remote repository without needing to checkout the repository into a workspace.

to

No implementation of the `SCMFileSystem` API - performing deep inspection of the remote repository required a checkout of the repository into a workspace.
+* Very basic event support - every event just triggers a re-scan of the entire repository.
+This was acceptable at the time because the only three implementations use a local cache of the remote state so re-scanning is quick.
+* No implementation of the `SCMFileSystem` API for performing deep inspection of the remote repository without needing to checkout the repository into a workspace.
+* No documentation on how plugin developers are supposed to implement the SCM API
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/are/were/
(The reset of this list should be past tense as well.)

@stephenc
stephenc Jan 12, 2017 Contributor

No it should be present tense SCM API 1.0 currently has all these issues

+* No documentation on how plugin developers are supposed to implement the Branch API to create their own multi-branch project types
+* No documentation on for users on how the Branch API based project types are expected to work.
+
+Roll forward to November 2015 and Jenkins Pipeline got a release of the link:https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Multibranch+Plugin[Pipeline Multibranch Plugin].
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

link on new line

+* GitHub Branch Source Plugin
+* BitBucket Branch Source Plugin
+
+Unlike the previous implementations of the SCM API, however, these plugins do not maintain a local cache of the repository state.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/do not/did not/

+
+== Enter SCM API 2.0.x and Branch API 2.0.x
+
+We are announcing the SCM API 2.0.x and Branch API 2.0.x release line.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

This (or something similar) should be your opening sentence for the article. You can repeat it here, but it needs to be in the opening as well.

+link:https://github.com/jenkinsci/scm-api-plugin/blob/master/docs/implementation.adoc[implement the SCM API]
+* documentation on how plugin developers are supposed to
+link:https://github.com/jenkinsci/scm-api-plugin/blob/master/docs/consumer.adoc[consume the SCM API (if they wanted to do something like Branch API but not the same was as Branch API)]
+* documentation on how plugin developers are supposed to link:https://github.com/jenkinsci/branch-api-plugin/blob/master/docs/implementation.adoc[implement the Branch API]
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

link on newline.

+* link:https://github.com/jenkinsci/scm-api-plugin/tree/master/src/test/java/jenkins/scm/api[lots]
+link:https://github.com/jenkinsci/scm-api-plugin/tree/master/src/test/java/jenkins/scm/impl[and]
+link:https://github.com/jenkinsci/branch-api-plugin/tree/master/src/test/java/integration[lots]
+of new automated tests added
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/added//

+* Git Plugin
+* Mercurial Plugin
+
+Ok, that was the good news... Here is the bad news...
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

Ok, that was the good news... Here is the bad news...
to
Ok, that was the good news. Here is the bad news.

+When you are ready to upgrade you must ensure that you upgrade all the required plugins.
+If you miss some, just upgrade them and restart to fix the issue.
+
+TIP: Always take a backup of your `JENKINS_HOME` before upgrading any plugins.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

Put this tip at the top as well (with the other two key points). Telling people twice to backup before upgrading is better than them not seeing it.

+GitHub Plugin:: 1.25.0 or newer
+Folders Plugin:: 5.16 or newer
+
+NOTE: After an upgrade you will see the data migration warning
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

"After an upgrade you will see the data migration warning. "

+
+So when releasing the initial version of the
+link:https://wiki.jenkins-ci.org/display/JENKINS/Literate+Plugin[literate plugin]
+we also released some reusable components upon which the multi-branch functionality was built.
@HRMPW
HRMPW Jan 12, 2017 Contributor

s/multi-branch/Pipeline Multibranch and Org Folder/

We introduced those plugins above and that is what people are using and where we want to lead them.

@stephenc
stephenc Jan 12, 2017 Contributor

how about

So when releasing the initial version of the
link:https://wiki.jenkins-ci.org/display/JENKINS/Literate+Plugin[literate plugin]
we also separated the literate specific bits from the SCM specific conepts and multi-branch concepts.
These were lower level concepts were organized into the following plugins:
+we also released some reusable components upon which the multi-branch functionality was built.
+Those components were:
+
+* link:https://wiki.jenkins-ci.org/display/JENKINS/SCM+API+Plugin[SCM API] -
+
+* link:https://wiki.jenkins-ci.org/display/JENKINS/SCM+API+Plugin[SCM API] -
+which was intended to be a plugin to hold a next generation API for interacting with source control systems.
+* link:https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin[Branch API] -
+:tags:
+- development
+- plugins
+:author: stephenc
@rtyler
rtyler Jan 12, 2017 Member

I don't believe you have authored a blog post since we switched to the static site. Would you mind adding a stephenc.adoc to content/_data/authors?

The existing files should provide a good example

+
+There are some gotcha's that Jenkins administrators will need to be aware of.
+
+We want to give you the whole story, but the <<tldr,TL;DR>> is this
@rtyler
rtyler Jan 12, 2017 Member

The use of these admonitions has me confused, what is the actual TL;DR here? The formatting of this makes it very unclear to me 😦

@HRMPW

in the bullet above this line.

s/same was/same way/

@HRMPW

Line 109-123 replace the word "hacks" with "...plugins have hard-coded assumptions about internal implementations."

@HRMPW
HRMPW commented on 421691d Jan 12, 2017

Overall I don't get a sense of why I would want to upgrade to this new thing. If I'm not a developer using the SCM API or the Branch API what's in it for me. We should talk more about the benefits of the Event System and link in the JENKINS tickets on the API rate limit problems. We should also talk about how certain events weren't trigger like a new repository, etc.

-* SCM API
-* Pipeline Multibranch
-* GitHub Organization Folders
+Downstream of this there are also made some great improvements to a number of popular plugins including:
@HRMPW
HRMPW Jan 12, 2017 Contributor

remove "made"

@HRMPW
Contributor
HRMPW commented Jan 12, 2017

🐝 this looks good to me now.

+link:https://plugins.jenkins.io/github-branch-source[GitHub Branch Source]
+and/or
+link:https://plugins.jenkins.io/cloudbees-bitbucket-branch-source[BitBucket branch source]
+plugins installed then you *must* upgrade them at the same time or Bad Things™ will happen.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/them at/them *all* to 2.0.x at/

"tm" - asciidoc has a thing for this: (TM).

+Unlike the previous implementations of the SCM API, however, these plugins do not maintain a local cache of the repository state.
+Rather they make queries via the GitHub / BitBucket REST APIs on demand.
+
+The above design decision exposed one of the initial MVP compromises of the SCM API plugin: _Very basic event support_.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/Very/very/

+Under the SCM API 1.x model, the only event that an `SCMSource` can signal is _something changed, go look at everything again_.
+When you are accessing an API that only allows 5,000 API calls per hour, performing a full scan of the entire repository just to pick up a change in one branch does not make optimum usage of that 5,000 calls/hour rate limit.
+
+So we decided that perhaps the SCM API and Branch API plugins have left their Minimum Viability Experiment state and the corresponding limitations should be addressed.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/have/had/

@stephenc
stephenc Jan 12, 2017 Contributor

This is plural and present tense so should be have. Putting "had" here completely sounds wrong to my ear

+Rather they make queries via the GitHub / BitBucket REST APIs on demand.
+
+The above design decision exposed one of the initial MVP compromises of the SCM API plugin: _Very basic event support_.
+Under the SCM API 1.x model, the only event that an `SCMSource` can signal is _something changed, go look at everything again_.
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

s/can/could/

@stephenc
stephenc Jan 12, 2017 Contributor

The 1.x model has not disappeared into the past. It is still there.

Under that model, the only event that you can (today) signal is scan everything.

it's not a past conditional tense. Putting "could" here completely sounds wrong to my ear

+
+== Summary for busy SCM plugin developers
+
+You should read the new https://github.com/jenkinsci/scm-api-plugin/blob/master/docs/implementation.adoc[documentation on how plugin developers are supposed to implement the SCM API]
@bitwiseman
bitwiseman Jan 12, 2017 Contributor
/n
link:https
+
+The persistent reader may be wondering what happens now to the Literate plugin.
+
+For me, the logical heir of the Literate Plugin is the https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Model+Definition+Plugin[Pipeline Model Definition plugin].
@bitwiseman
bitwiseman Jan 12, 2017 Contributor
/n
link:https
+== The back-story
+
+Way back in September 2013 we announced the
+link:https://jenkins.io/blog/2013/09/23/literate-builds-wtf/[literate plugin],
@bitwiseman
bitwiseman Jan 12, 2017 Contributor

At the end of blog you capitalize "Literate".
Do you want to do that here as well, or make them all lowercase?

@bitwiseman
Contributor

@stephenc - still a few more tweaks.

@bitwiseman
Contributor

🐝

@HRMPW
Contributor
HRMPW commented Jan 13, 2017

🐝 I'm good with it. are we ready to publish?

@stephenc
Contributor

@reviewbybees done... depends on whether we want this post out now or on monday when I cut the releases...

@HRMPW
Contributor
HRMPW commented Jan 13, 2017

We need to update the file name to reflect the proper date when we publish, too.

@bitwiseman
Contributor
bitwiseman commented Jan 13, 2017 edited

@stephenc - We don't usually release posts after Friday morning. Monday seems right to me. Can you rename the file to Monday's date?

@daniel-beck daniel-beck merged commit 41d535b into jenkins-infra:master Jan 16, 2017

1 check passed

continuous-integration/jenkins/pr-merge This commit looks good
Details
@stephenc stephenc deleted the stephenc:scm-api-2.x branch Jan 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment