First pass at 1.8 schedule #305

grodrigues3 · 2017-06-22T21:11:33Z

Approximate timeline and schedule for the Kubernetes 1.8 release

@kubernetes/sig-release-misc

idvoretskyi · 2017-06-22T21:23:14Z

release-1.8/release_team_absence_tracker.md

@@ -0,0 +1,13 @@
+### Release Team absence tracker


Do we really need this file?

Probably not, I can delete

luxas

A couple of comments...
Otherwise LGTM

luxas · 2017-06-24T07:49:15Z

release-1.8/release-1.8.md

+  * Ensure docs and release notes are written
+  * Identify all features going into the release, and make sure alpha, beta, ga is marked in features repo
+- Sept 6 (Weds): v1.8.0-beta.0
+- Jun 7 (Weds): v1.8.0-beta.1 cut


should be Sept 13?

luxas · 2017-06-24T07:50:29Z

release-1.8/release-1.8.md

+  * Focus on bugfix, test flakes and stabilization
+  * Ensure docs and release notes are written
+  * Identify all features going into the release, and make sure alpha, beta, ga is marked in features repo
+- Sept 6 (Weds): v1.8.0-beta.0


Clarify that the release-1.8 branch is created at this point and v1.9.0-alpha.0 is created

luxas · 2017-06-24T07:51:01Z

release-1.8/release-1.8.md

+  * Begin regular fast-forwards (at least daily to keep CI up-to-date).
+
+### Sept 1 (Fri) - Sept 14 (Thurs)
+- Sept 1 Begin Feature Freeze


Be consistent and say Code Freeze to be clear that this isn't the Features Repo Freeze

luxas · 2017-06-24T07:51:41Z

release-1.8/release-1.8.md

+  have an associated issue in the features repo
+- Aug 9 (Weds): v1.8.0-alpha.3
+- Aug 23 (Weds): v1.8.0-alpha.4
+- Sept 6 (Weds): v1.8.0-beta.0


This seems misplaced

luxas · 2017-06-24T07:52:28Z

release-1.8/release-1.8.md

+- Release 1.8 alphas every 2 weeks
+- July 12th (Weds): v1.8.0-alpha.1
+- July 26th (Weds): v1.8.0-alpha.2
+- Aug 1 (Tues): features repo freeze: all features going into release should


Sometimes 12th, 26th, etc. sometimes 1, 9, 23.
Please be consistent

luxas · 2017-06-24T07:53:37Z

release-1.8/release-1.8.md

+- July 5 (Weds) coding start (7w)
+- July 12th (Weds): v1.8.0-alpha.1
+- July 26th (Weds): v1.8.0-alpha.2
+- Aug 1 (Tues): features repo freeze


suggest, just say "Feature Freeze" here and define later down in the doc.
The term has been used for the past 5-ish releases so it's not new anyway

idvoretskyi · 2017-06-28T20:39:52Z

release-1.8/release-1.8.md

+
+
+# Key features
+[Feature tracking spreadsheet


Spreadsheet link to be provided by @idvoretskyi

pwittrock · 2017-06-30T14:44:39Z

/assign pwittrock

pwittrock · 2017-06-30T14:45:38Z

release-1.8/release-1.8.md

+
+### July 5 - Sept 1
+- 8 week coding period
+- Release 1.8 alphas every 2 weeks


i suggest we require all release tests to be passing to cut an alpha

Agreed. The more the alpha releases are like treated like real releases, the better.

To clarify:

By "all release tests", do you mean all the jobs displayed on the release-master-blocking testgrid dashboard? AFAIK this what kubernetes/release uses

By "be passing" do you mean their most-recent "Overall" results are green? Some noise I've heard in the past: human explanations for why we could ignore red results in some mostly green suites; a practice of waiting for "3 in a row" prior to calling a final release good

By "all release tests", do you mean all the jobs displayed on the release-master-blocking testgrid dashboard? AFAIK this what kubernetes/release uses

I think this makes sense at least until we can get better ownership of non-blocking suites. If we do include others at this point we should explicitly list them since the status quo is currently having some bad jobs. This would make sense for important suites that are likely non-blocking for test duration reasons like serial and soak.

By "be passing" do you mean their most-recent "Overall" results are green? Some noise I've heard in the past: human explanations for why we could ignore red results in some mostly green suites; a practice of waiting for "3 in a row" prior to calling a final release good

The "3 in a row" is the standard I've seen for the past few releases but it's generally been up to human discretion. We should decide on an exact criteria and ideally have it machine enforced.

I believe we need to look at overall test stability when it comes to making the call. Philosophically speaking, a disruptive release is far worse than a disruptive wait.

Decision: don't cut alpha until master-blocking-tests are passing
Outstanding Question: GKE tests should be release blocking? Failing 25+% clouds the release signal.

pwittrock · 2017-06-30T15:28:07Z

release-1.8/release-1.8.md

+- Aug 23rd (Weds): v1.8.0-alpha.4
+
+### Sept 1 (Fri) - Sept 14 (Thurs)
+- Sept 1st Begin Code Freeze


Should this be called 'Code Slush' or 'Code Freeze'?

Both terms have been used in the past...

Let's pick one and be consistent. I have no strong opinions, so I consulted a search of kubernetes-dev. "Code freeze" returned ~58 results, "code slush" returned ~9, so maybe we should just stick with "code freeze" even if it doesn't strictly represent what happens during this period.

+1 for freeze

I am +1 for freeze, but wanted to point out that "code slush" usually implies some changes can/will continue to be made at the discretion of the release team. "Code freeze" from an auditing perspective means that zero code changes above those required for the release mechanics will occur. If we look at code freezes from the past, it might look more slushy than hard frozen. If we want to introduce a period of time were select changes were possible, we could use slush for that to differentiate.

pwittrock · 2017-06-30T15:28:34Z

release-1.8/release-1.8.md

+  * v1.9.0-alpha.0 is cut
+  * Test-infra lead to set up CI for branch.
+  * Begin regular fast-forwards (at least daily to keep CI up-to-date).
+- Sept 13th (Weds): End of Code Slush


I was thinking we should shoot for making code slush just 1 week. If the tests are all passing, and the issues are auto-managed, this should be possible.

We should only accept cherry picks for release blockers starting 1 week after code slush starts.

pwittrock · 2017-06-30T15:31:42Z

release-1.8/release-1.8.md

+
+### Sept 15 - Sept 27
+- Sept 18th (Mon): v1.9.0-alpha.1
+- Sept 20th (Weds): v1.8.release-candidate


This is effectively saying, have all release blocking issues resolved 1 week before we release. Historically, we have never come close to this, or even been able to resolve release blocking issues by the actual release date. I suggest either scrap the release candidate until we are able to consistently meet our release timeline.

I think the issue is this hasn't been treated as the actual release date in the past so we always blow right through it. I do think there's value in trying to target this because we still haven't released on time and this gives a buffer and the way we backload code changes means hitting any specific date is unlikely.

Isn't the release candidate supposed to provide a preview for external consumer validation, and other out-of-band actions? If it's not as close as possible to the actual release, I am not sure what value it has to the community. If it's a dry run for the actual release, I am all for it. If it's just "one more thing" to worry about at release crunch time with no demonstrable value, then it should be obsoleted.

We need to to treat RC date as the release date if we want the RC to be meaningful, which is something we haven't done well yet.

I'm in favor of keeping the RC but only if we commit to a mindset that Sept. 20 is when we're releasing.

Decision: keep it but drive harder to

have all tests passing (blocking, upgrades, skew)

pwittrock · 2017-06-30T15:34:05Z

release-1.8/release-1.8.md

+  * Only accept cherrypicks for release-blockers.
+  * Announce on mailing lists, Twitter, etc. to ask people to build and test the RC and submit feedback
+  * Perhaps managed k8s providers make rc.1 available on early access channels.
+  * More RCs as needed to verify release-blocking fixes.


We need to have something about release notes and docs on here. When they are started, and what the dates for milestones are. I suggest release notes for completed issues must be done by SIGs 1 week before the release date.

By now, we're 1 week out from release. From a release notes perspective, I'm in full agreement we should basically have everything documented, all we should be changing by now are structural edits or last minute Known Issues. We should have enough in place that the release branch manager could cut a copy of CHANGELOG.md with all the release notes incorporated (assuming that's where they still land)

IMO what should not be happening by now is the addition of features or large structural changes in the release notes. I feel like I saw both happen the final week of the previous release.

From a docs perspective, I would hope we're basically at a live copy of all of the docs, with grammar edits and spell checking at this point. Historically that has not been the case though. What is reasonable to expect by now? @kubernetes/sig-docs-maintainers

Honestly, at 1 week out for 1.7, half the 22 docs were merged and good to go. A couple days later (that Fri), we were at 80%. Even the day of the original release date, we were scrambling to get a couple docs reviewed and merged because one snuck in last minute and another one hadn't been completed and ready for review until that day. It follows sort of an exponential decay pattern. We can set the deadlines earlier, but there will still be a few stragglers at the end.

pwittrock · 2017-06-30T15:35:20Z

release-1.8/release-1.8.md

+## Operational Changes 
+1. For the 1.8 release, the [release team](https://github.com/kubernetes/features/blob/master/release-1.8/release_team.md)
+  will use the following procedure to identify release blocking issues
+  1. Any issues listed in the [v1.8 milestone](https://github.com/kubernetes/kubernetes/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20milestone%3Av1.8)


FYI, my proposal is that issues are automatically moved out of the milestone unless they are labeled critical-urgent and have a SIG owner.

See link here
kubernetes/community#752

Let the 1.8 release team decide on this

pwittrock · 2017-06-30T15:35:37Z

release-1.8/release-1.8.md

+     milestones **or the release will not ship**
+
+# Contact us
+- [via slack](https://kubernetes.slack.com/messages/k8s-release/)


sig-release?

pwittrock · 2017-06-30T15:36:40Z

release-1.8/release_team.md

+|  Test Infra | | | |
+|  Docs || |
+|  Bugs | | | |
+|  CI Signal | | | |


For 1.8 I think we should merge CI Signal and Upgrade Testing, but the role should start immediately and require keeping the tests passing throughout the release.

+1 for having roles start immediately

My only concern with merging would be spreading leads thin.

pwittrock · 2017-06-30T15:37:31Z

release-1.8/release-1.8.md

+- Sept 6th (Weds): v1.8.0-beta.0
+  * Branch Manager to create v1.8 release branch.
+  * v1.9.0-alpha.0 is cut
+  * Test-infra lead to set up CI for branch.


Note that we will still use the master tests as the source of truth until the end of code slush, since the master tests are more up-to-date

Is there a case where this practice could lead to a false green or red? Just wondering if there are any risks in this.

idvoretskyi · 2017-06-30T15:38:47Z

release-1.8/release_team.md

@@ -0,0 +1,12 @@
+|  **Role** | **Name** (**GitHub/Slack ID**)  | **Shadow Name(s) (GitHub/Slack ID), ...**


Is the "Shadow role" still actual? For 1.7 we had only Lead release manager shadowed.

Personally I would like to see it remain present. I can shadow someone this release.

+1 I like how it provides a lower commitment avenue to be involved in a release

It might be moot if we don't find someone to fill it.

jdumars · 2017-07-05T22:35:37Z

With the requested changes, LGTM.

spiffxp

I'd like to see requirements for docs spelled out a bit more, and some explicit definition of passing tests. Couple other tiny nits.

spiffxp · 2017-07-06T01:08:40Z

release-1.8/release-1.8.md

+  * Ensure docs and release notes are written
+  * Identify all features going into the release, and make sure alpha, beta, ga is marked in features repo
+- Sept 6th (Weds): v1.8.0-beta.0
+  * Branch Manager to create v1.8 release branch.


release-1.8 branch

spiffxp · 2017-07-06T01:10:09Z

release-1.8/release-1.8.md

+  * Identify all features going into the release, and make sure alpha, beta, ga is marked in features repo
+- Sept 6th (Weds): v1.8.0-beta.0
+  * Branch Manager to create v1.8 release branch.
+  * v1.9.0-alpha.0 is cut


kubernetes/release#342 @timothysc does this schedule adequately address the concern you had here?

@spiffxp Yes, this is much better than v1.7, where we cut the branch basically the same day as the code freeze.
As we know there's a lot of movement when everyone is trying to get their thing into the next release, the SQ was 50-ish deep and things flaked violently.

Cutting the branch that day or the next wasn't the best thing to do, as folks had to fast-forward it very frequently in order to keep things somewhat green. This combined with all the other churn wasn't ideal. On top it this, it also made things harder for consumers of release artifacts like kubeadm.

Waiting a couple of days before cutting the branch/beta.0/alpha.0 makes a lot of sense, thanks @grodrigues3

spiffxp · 2017-07-06T01:30:20Z

release-1.8/release-1.8.md

+
+### July 5 - Sept 1
+- 8 week coding period
+- Release 1.8 alphas every 2 weeks


To clarify:

By "all release tests", do you mean all the jobs displayed on the release-master-blocking testgrid dashboard? AFAIK this what kubernetes/release uses

By "be passing" do you mean their most-recent "Overall" results are green? Some noise I've heard in the past: human explanations for why we could ignore red results in some mostly green suites; a practice of waiting for "3 in a row" prior to calling a final release good

spiffxp · 2017-07-06T01:40:48Z

release-1.8/release-1.8.md

+- Aug 23rd (Weds): v1.8.0-alpha.4
+
+### Sept 1 (Fri) - Sept 14 (Thurs)
+- Sept 1st Begin Code Freeze


Let's pick one and be consistent. I have no strong opinions, so I consulted a search of kubernetes-dev. "Code freeze" returned ~58 results, "code slush" returned ~9, so maybe we should just stick with "code freeze" even if it doesn't strictly represent what happens during this period.

spiffxp · 2017-07-06T01:47:18Z

release-1.8/release-1.8.md

+- Sept 13th (Weds): End of Code Slush
+  * v1.8.0-beta.1 cut
+  * Final fast-forward of release branch.
+  * After this, all changes for the release must now be cherrypicked in batches by branch


@pwittrock I believe at this point, we no longer look at master tests, but at release-branch tests for our CI signal, right?

@spiffxp yup, after we start cherrypicking master no longer duplicates CI

spiffxp · 2017-07-06T01:58:34Z

release-1.8/release-1.8.md

+  * Add milestone restriction on submit queue.
+  * Community Feature Burndown Meetings held two or three times until release week (then every day). For those interested in joining please join [Kubernetes Milestone Burndown Group](https://groups.google.com/forum/#!forum/kubernetes-milestone-burndown)
+  * Focus on bugfix, test flakes and stabilization
+  * Ensure docs and release notes are written


@pwittrock so we mention release notes and docs up here at the start of code freeze, but agree we're not spelling out enough

In terms of release notes, we should now have enough to collate one-line descriptions of high-level features that are being added, deprecations or removals that are happening, and improvements that are being made (eg: scrape the one-line release description from features repo issues, or copy-pasta from a manually managed spreadsheet)

This should be enough to answer "but why should I care about the beta?" with a bite-sized blog or e-mail of "you can try out X, we've made Y way more usable, and you can can get a jump on the deprecation of Z"

In terms of docs, I'm guessing by now there should be enough content for a tech review? @kubernetes/sig-docs-maintainers

@spiffxp this all seems spot on. Those three criteria must be satisfied. We should have a document go/no-go date set as well. Documentation (the one-liner) should be a blocker for matriculation IMO since the burden is low and the risk of end-user "feature discovery" is very high. Without this documentation, we lose our most valuable asset: transparency.

how would we feel about making "Release note deadline" two weeks after feature freeze?

+1 @calebamiles. But I'd propose to set this deadline for docs as well, and at the same time, when the code freeze should happen.

I think it makes sense collect the full release notes at the time of code freeze.

Collecting and announcing what features are gonna be in the release on a high level should be done as soon as possible after the feature freeze. It will serve as the base for the release notes, which will be enhanced at the time of the code freeze with important PRs, action required/deprecation/etc items etc.

@calebamiles I am for that being when they are finalized, but not necessarily due. I acknowledge that the burden of getting the feature AND docs over the line at the same time can be difficult. But, SIGs should have that as part of the planned workstream as much as the feature itself. Would a one-week grace period serve the purpose? After that, it would be time to have discussions with SIGs and individuals to ensure it gets done.

Decision: Relnotes for feature due after feature freeze (see dates above)
Decision: Docs out (must be a PR) for tech review by one week after code freeze. A week after that should be lgtm'd and ready for merge.

spiffxp · 2017-07-06T02:02:34Z

release-1.8/release-1.8.md

+  * Test-infra lead to set up CI for branch.
+  * Begin regular fast-forwards (at least daily to keep CI up-to-date).
+- Sept 13th (Weds): End of Code Slush
+  * v1.8.0-beta.1 cut


We've just lifted code freeze, cut our second beta, and are one full business week and change out from our first pass at docs and release notes. What is reasonable progress to expect by now? @kubernetes/sig-docs-maintainers

I'm curious about this as well.

spiffxp · 2017-07-06T02:06:01Z

release-1.8/release-1.8.md

+  * Only accept cherrypicks for release-blockers.
+  * Announce on mailing lists, Twitter, etc. to ask people to build and test the RC and submit feedback
+  * Perhaps managed k8s providers make rc.1 available on early access channels.
+  * More RCs as needed to verify release-blocking fixes.


By now, we're 1 week out from release. From a release notes perspective, I'm in full agreement we should basically have everything documented, all we should be changing by now are structural edits or last minute Known Issues. We should have enough in place that the release branch manager could cut a copy of CHANGELOG.md with all the release notes incorporated (assuming that's where they still land)

IMO what should not be happening by now is the addition of features or large structural changes in the release notes. I feel like I saw both happen the final week of the previous release.

From a docs perspective, I would hope we're basically at a live copy of all of the docs, with grammar edits and spell checking at this point. Historically that has not been the case though. What is reasonable to expect by now? @kubernetes/sig-docs-maintainers

spiffxp · 2017-07-06T02:08:50Z

release-1.8/release_team.md

@@ -0,0 +1,12 @@
+|  **Role** | **Name** (**GitHub/Slack ID**)  | **Shadow Name(s) (GitHub/Slack ID), ...**


Personally I would like to see it remain present. I can shadow someone this release.

jdumars · 2017-07-20T15:25:06Z

release-1.8/release-1.8.md

+
+# Contact us
+- [via slack](https://kubernetes.slack.com/messages/k8s-release/)
+- [via email](mailto:kubernetes-release@googlegroups.com)


kubernetes-sig-release@googlegroups.com

calebamiles

We'll need to add in the dates for the release notes and docs but it all looks reasonable to me

grodrigues3 · 2017-07-21T21:21:20Z

release-1.8/release-1.8.md

+
+### July 5 - Sept 1
+- 8 week coding period
+- Release 1.8 alphas every 2 weeks


Decision: don't cut alpha until master-blocking-tests are passing
Outstanding Question: GKE tests should be release blocking? Failing 25+% clouds the release signal.

grodrigues3 · 2017-07-21T21:30:23Z

release-1.8/release-1.8.md

+- July 26th (Weds): v1.8.0-alpha.2
+- Aug 1 (Tues): Feature Freeze
+- Aug 9 (Weds): v1.8.0-alpha.3
+- Aug 23 (Weds): v1.8.0-alpha.4


Aug 8: release notes draft for features due
Aug 15: hard deadline for feature related relnotes

Consequence: your feature not included in release notes

grodrigues3 · 2017-07-21T21:34:47Z

release-1.8/release-1.8.md

+  * Add milestone restriction on submit queue.
+  * Community Feature Burndown Meetings held two or three times until release week (then every day). For those interested in joining please join [Kubernetes Milestone Burndown Group](https://groups.google.com/forum/#!forum/kubernetes-milestone-burndown)
+  * Focus on bugfix, test flakes and stabilization
+  * Ensure docs and release notes are written


Decision: Relnotes for feature due after feature freeze (see dates above)
Decision: Docs out (must be a PR) for tech review by one week after code freeze. A week after that should be lgtm'd and ready for merge.

grodrigues3 · 2017-07-21T21:42:03Z

release-1.8/release-1.8.md

+
+### Sept 15 - Sept 27
+- Sept 18th (Mon): v1.9.0-alpha.1
+- Sept 20th (Weds): v1.8.release-candidate


Decision: keep it but drive harder to

have all tests passing (blocking, upgrades, skew)

grodrigues3 · 2017-07-21T21:43:58Z

release-1.8/release-1.8.md

+  * Only accept cherrypicks for release-blockers.
+  * Announce on mailing lists, Twitter, etc. to ask people to build and test the RC and submit feedback
+  * Perhaps managed k8s providers make rc.1 available on early access channels.
+  * More RCs as needed to verify release-blocking fixes.


grodrigues3 · 2017-07-21T21:44:05Z

release-1.8/release-1.8.md

+
+
+# Key features
+[Feature tracking spreadsheet


grodrigues3 · 2017-07-21T21:49:08Z

release-1.8/release-1.8.md

+## Operational Changes 
+1. For the 1.8 release, the [release team](https://github.com/kubernetes/features/blob/master/release-1.8/release_team.md)
+  will use the following procedure to identify release blocking issues
+  1. Any issues listed in the [v1.8 milestone](https://github.com/kubernetes/kubernetes/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20milestone%3Av1.8)


Let the 1.8 release team decide on this

grodrigues3 · 2017-07-21T22:18:20Z

Merging after the 07/21 scheduling finalization meeting

k8s-ci-robot added sig/release Categorizes an issue or PR as relevant to SIG Release. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 22, 2017

grodrigues3 requested review from pwittrock and calebamiles June 22, 2017 21:11

idvoretskyi reviewed Jun 22, 2017

View reviewed changes

luxas reviewed Jun 24, 2017

View reviewed changes

idvoretskyi reviewed Jun 28, 2017

View reviewed changes

pwittrock reviewed Jun 30, 2017

View reviewed changes

luxas assigned luxas and pwittrock Jun 30, 2017

pwittrock reviewed Jun 30, 2017

View reviewed changes

idvoretskyi reviewed Jun 30, 2017

View reviewed changes

spiffxp suggested changes Jul 6, 2017

View reviewed changes

jdumars reviewed Jul 20, 2017

View reviewed changes

jdumars mentioned this pull request Jul 20, 2017

Finalize 1.8 schedule kubernetes/sig-release#7

Closed

calebamiles approved these changes Jul 21, 2017

View reviewed changes

grodrigues3 commented Jul 21, 2017

View reviewed changes

1.8 schedule

8fbba89

grodrigues3 force-pushed the 1.8-release branch from ea4d8d1 to 8fbba89 Compare July 21, 2017 22:17

grodrigues3 merged commit 27fe14a into kubernetes:master Jul 21, 2017

		@@ -0,0 +1,12 @@
		\| Role \| Name (GitHub/Slack ID) \| Shadow Name(s) (GitHub/Slack ID), ...

First pass at 1.8 schedule #305

First pass at 1.8 schedule #305

Conversation

grodrigues3 commented Jun 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luxas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pwittrock commented Jun 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pwittrock Jun 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chenopis Jul 6, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pwittrock Jun 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdumars commented Jul 5, 2017

spiffxp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdumars Jul 20, 2017 • edited Loading

Choose a reason for hiding this comment

calebamiles left a comment

pwittrock Jun 30, 2017 •

edited

Loading

chenopis Jul 6, 2017 •

edited

Loading

pwittrock Jun 30, 2017 •

edited

Loading

jdumars Jul 20, 2017 •

edited

Loading

grodrigues3 Jul 21, 2017 •

edited

Loading