Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Establishment of Zowe Support Provider Conformance Program #182

Closed
armstro opened this issue Apr 27, 2020 · 35 comments
Closed

Establishment of Zowe Support Provider Conformance Program #182

armstro opened this issue Apr 27, 2020 · 35 comments
Assignees
@PeterFandelAtRocket @armstro @pdubz3
Labels
Epic

Comments

@armstro
Copy link
Contributor

armstro commented Apr 27, 2020

Proposal is to create a set of conformance criteria to be a Zowe Support Provider.

Examples

  • Agreement of what is included in a "core" Zowe distribution and acceptance of Zowe/OMP project policies since it will be shared support across multiple possible vendors
  • Agreement to recognize the Genuine Zowe process (hash tag/ watermark,...)
  • Agreement to distribute core Zowe as a separate upgradeable/serviceable component (SMP/E FMID, its own PSI package, etc.)
  • Use of Zowe build process and regression testing for core Zowe
  • Use of Zowe diag tools for core
  • Timely reporting of any security vulnerabilities to the Zowe community using the discrete reporting process
  • Willingness to assist in security exposures in code/repos where they are committers
@armstro armstro added the Epic label Apr 27, 2020
@jmertic
Copy link
Collaborator

jmertic commented May 13, 2020

Whenever y'all are ready to start here, let me know.

@jmertic jmertic mentioned this issue May 13, 2020
Closed
@armstro
Copy link
Contributor Author

armstro commented May 14, 2020

John - my crystal ball is foggy on this but my guess is this is 30 days out.....TSC mission and establishment and joint PI Planning are higher priorities at the moment.

@pdubz3 pdubz3 assigned pdubz3 and unassigned solsu01 Aug 4, 2020
@jmertic
Copy link
Collaborator

jmertic commented Sep 30, 2020

Want to bump this up - starting to see more public movement and advise that getting ahead of more growth would be something to consider sooner than later.

@armstro
Copy link
Contributor Author

armstro commented Sep 30, 2020

Once we get TSC moving we will come back to this item ......

@jmertic
Copy link
Collaborator

jmertic commented Sep 30, 2020

OK

@armstro
Copy link
Contributor Author

armstro commented Mar 23, 2021

OK - I wanted to dust off this item and see about getting it started..... @jmertic you mentioned Kubernetes has a program for Support - I found this https://www.cncf.io/certification/kcsp/ - do you have any other info? The good news is the criteria is short and sweet but the problem seems to be it is difficult to apply to Zowe....(like Three or more engineers who pass the Certified Kubernetes Administrator (CKA) exam. )..... but the list is a place to start. Do you have other examples for Support Conformance?

@jmertic
Copy link
Collaborator

jmertic commented Mar 23, 2021

That's probably the best I've seen thus far.

I'd not get hung up on the precise details, but more think about what you would hope that anyone saying they could support Zowe should be able to do. This might be having certain skillsets on staff. This might be community involvement. Might have something related other capacities. I'd encourage y'all to be creative on the way you think about this.

@armstro
Copy link
Contributor Author

armstro commented Apr 14, 2021

Update - Peter, Mike D and myself have been working on Support Conformance criteria in a couple documents (modeled after the current Zowe conformance programs for Web UI, API ML, etc.) - summary of status and key issues here - more work to do:

  • Support conformance would be subdivided like the other conformance programs meaning a vendor could choose to only offer Support on part of Zowe (CLI and Explorer for example) or all of Zowe (meaning multiple badges but when is too many badges too confusing)
  • Since we do not have exams like Kubernetes does - there is discussion on what "credentials" are needed by a Service Provider - at a minimum they need at least contributers (or committers) on the squad(s) where they claim Support (like CLI/Explorer)
  • There is a question on what role the squad plays in reporting status of contributor participation/competence - if a badged Support provider contributor does not show adequate skills or does not keep up with squad activities what should be done? We want to avoid a vendor just reporting issues and not contributing to working solutions
  • Can any OMP member be a Support Provider or is there an additional membership to be considered? (How to avoid Support being offered only by big firms?)
  • The Support "scope" generally relates to "core" Zowe capabilities (for example not incubators - although any vendor could offer incubator assistance for a fee but that is not "support") - do we need to revisit what core means and who/how determines core?
  • What is OMP role if a Support Provider has complaints from customers? Under what circumstances is a badge revoked?

Plans are for additional working sessions and to include others with experience with the existing conformance programs to gain lessons learned and advice.

@jmertic
Copy link
Collaborator

jmertic commented Apr 14, 2021

Glad this is taking shape! A few thoughts based on the questions...

  • Requiring squad participation is probably the best bet right now, until some sort of 'Zowe' coursework could go live ( nudge nudge ;-) ). One thought to think about is that you likely will have vendors interested in support conformance that might not have enough of a technical bench to contribute upstream - not to say that squad participation shouldn't be important criteria, but something to think about.
  • I would imaging running the support provider program fee requirements like we do for other Zowe Conformance ( free for members, cost for non-members ) would make the most sense.
  • Although we've been fortunate to not have challenges on conformance yet, the way I would see it work is to first attempt to mediate the concern, and if that is unsuccessful look to remove the provider from the program.

Please pull me in when appropriate.

@armstro
Copy link
Contributor Author

armstro commented May 25, 2021

Many thanks to Rose for keeping up moving on the Support Conformance Program. Latest draft is attached - comments:

  1. We state Support can be Comprehensive (meaning all of Core Zowe) or a subset (i.e, Partial). We also state "Best Practices" implies Support beyond Core - do we want to coin a term other than Best Practice - like Comprehensive+ ?

  2. Do we need to a "finger print" program equivalent for non-z/OS components ? If one already exists does it need to be made more visible
    to the community?

  3. Question on "no forked code" - does it just need to state abide by EPL2.0 with any code changes?

  4. Question on timeliness of reporting security vulnerabilities - I think 3-5 days? What would hold up a report being made? Do we need to say vulnerabilities would not be made public until a mitigation is ready for distribution?

  5. I think we need to work with the TSC on the definition of Core Zowe. I think TSC owns the declaration of core changes when a component, app, extension is ready to be considered core. Support vendors not ready to support a new core component would have the ability to work with the TSC.

Zowe Support Conformance V1.xlsx

@jmertic
Copy link
Collaborator

jmertic commented May 25, 2021

This does look good - thanks Rose!

Some comments...

Support Vendor agrees to support any binary-equivalent of all Zowe core components, regardless of where obtained

This feels like it could get messy, and considering some of the recent activities we've seen with ransomware the phrase "regardless of where obtained" is also something that should be discouraged.

We should also define "binary-equivalent"

Support Vendor Agrees: to abide by "no forked code distributions of Zowe" - for either full distributions of Zowe OR any partial code distributed as a core Zowe enhancement or fix

Something that should be defined, as Rose pointed out.

Support Vendor confirms: comprehensive penetration tests are conducted annually by said Support Vendor and said Support Vendor is able to provide to OMP, reports demonstrating as such, when asked to do so

What does this mean?

Support Vendor agrees: to adhere to all documented Zowe TSC Engineering Standards [include a link to the standards here] for any support-related code change (fixes, patches, PTFs etc.) submission

Support Vendor Confirms: Said Vendor has read, understands and agrees to abide by the Zowe TSC Support-Vendor-Fix Process [include a link to the process here]

Support Vendor agrees: to adhere to all documented Zowe TSC Documentation Standards [include a link to the standards here] for any support-related code change (fixes, patches, PTFs etc.) submission that includes documentation

I'd think this would basically be the contribution guidelines, right? Not sure if there is some difference that would be expected here.

Support Vendor Agrees: to contribute all fixes to the Zowe community in a timely manner, as soon as possible and such contributions may be delayed for no longer than 3 months (90 days)

The last part seems to be putting a contract on the squads, which should have the right to accept or reject the contribution ( or even delay it if it makes sense to apply at a later time ).

"Active contributors" should also be defined, but I'm also wondering if that could exclude those who don't have the right skills on the bench to contribute code.

@armstro
Copy link
Contributor Author

armstro commented May 25, 2021 via email

@armstro
Copy link
Contributor Author

armstro commented May 25, 2021

Let me try to summarize the changes made after the ZLC call this morning (and we need to work with the TSC on a few items).

In layman's terms

  • Used the term authenticated binary to be Supported (and pointer to Zowe doc that describes)
  • For clarity say support must be all active/maintenance releases in a version
  • Code changes due to fixes need to be contributed back (consistent with EPL 2.0)
  • Need to report vulnerabilities (Best practices to work with the Security team)
  • Use authenticated binary when (if) embedding Zowe
  • Follow TSC policies and practices
  • Honor the PTF numbering process
  • If vendor specific patches are created to inform the community and incorporate back to community (this is a little redundant with the EPL 2.0 comment above and maybe can be struck out)
  • List the names of contributors/committers

Areas where we need to work with TSC

  • Process for defining "core"
  • Confirm there is authentication process for z/OS, distributed, (and containers)
  • TSC policies and practices (code quality, automated test, doc, etc.) - maybe include practice of 90 day to contribute back and what to do/say about customer validation of a fix
  • How to define an active contributor?

Zowe Support Conformance V1a - 5-25-21.xlsx

@PeterFandelAtRocket
Copy link
Contributor

I added new text for footnote 1.2 and made items 9 and 11 requirements (x was missing)
Zowe.Support.Conformance.V1b.-.5-25-21.xlsx

@armstro
Copy link
Contributor Author

armstro commented Jun 1, 2021

Zowe.Support.Conformance.V1-Near Final .xlsx
OK - I did my best to incorporate all the comments from today's ZLC - please read top to bottom including footnotes. I would like to bring this to a vote ASAP. I think we also agreed TSC will own "what is core".

@jmertic
Copy link
Collaborator

jmertic commented Jun 1, 2021

Looking better! I think there is some wordsmithing that can be done to clean things up, but nothing major.

@PeterFandelAtRocket
Copy link
Contributor

PeterFandelAtRocket commented Jun 1, 2021 via email

@armstro
Copy link
Contributor Author

armstro commented Jun 2, 2021

@PeterFandelAtRocket happy to have you "hold the pen" on this - I had asked Rose to review but I don't see her on the issue assignee list so she may not be seeing these updates. (and I am not sure she's even in the zlc issues repo to be assigned). My comments to your comments:

  • Use of Bold was once used to show delta from one version of the spreadsheet to another - bold can be removed
  • Use up caps and footnote numbering is up to you (Rose might have done the footnotes to be consistent with the other conformance criteria spreadsheets - don't know)
  • The italics in footnote 3 was to highlight a comment made by Joe regarding 90 day notice of "core changes" (my term) with a fix. We can decide how (if) to include this - I think it makes sense for Support Vendor to notify the community - I don't see how to enforce any of this
  • The missing links in the spreadsheet is because I don't think the TSC has decided the various policies and documented them yet.

Unless @pdubz3 or Rose have any objections - I say make your changes - leave links as TBD. We make Jakub aware of the "near final" proposal. He's already taken to the TSC the issues we need help with (defining active contributor, confirming there are ways to declare code as "authentic", what is core, etc.) but we need to give the TSC some time to sort that out.

@jmertic we briefly discussed having some "public vetting" of the Support Conformance criteria - if you have particular parties we should seek out for feedback, we can take that next step if you want.

I think we (ZLC) needs to put 1-2 slides together to summarize what we are proposing for easier presentation.

@pdubz3
Copy link
Contributor

pdubz3 commented Jun 2, 2021

No objection to Peter making the updates he proposed.

@pdubz3
Copy link
Contributor

pdubz3 commented Jun 2, 2021

Minor wording question, we may have covered this before so I apologize in advance if we have ... if a criterion is indicated as "required" is it less ambiguous to avoid words like "should" in the description in favor of words like "must"? For example in the first section, saying "Customers being supported should not be required to go to a particular vendor distribution channel ..." makes that sound more like a recommendation or best practice. I would suggest "Customers being supported must not be required to go to a particular vendor distribution channel." Thoughts on this?

@PeterFandelAtRocket
Copy link
Contributor

Zowe.Support.Conformance.V1-Near.Final.rev2.xlsx

I applied all my edits as well as replaced "should" with "must" per Mike's previous comment.

I also made the cells containing URLs to be active hyperlinks.

I left the Joe W italicized comment for now.

@pdubz3
Copy link
Contributor

pdubz3 commented Jun 2, 2021

Thank you @PeterFandelAtRocket. There's a second instance of "should" in one of the footnotes but I'm less certain as to whether it's really a "must". The footnote is about emergency fixes and it reads "Support Vendor Agrees: If any fixes require changes to the Zowe codebase then under the terms of the EPL 2.0 license, the modified code must be made available to the Zowe community. This should be in the form of a git issue be created that describes the original defect together with either a pull request, or a link to where the support vendor has published their updated code. This must be done in a timely manner after the support vendor has made the updated code available to their customer and delayed for no longer than 3 months (90 days). "

@PeterFandelAtRocket
Copy link
Contributor

Replaced other 'should' with 'must'
Zowe.Support.Conformance.V1-Near.Final.rev3.xlsx

@RASakach
Copy link
Contributor

@armstro @PeterFandelAtRocket @pdubz3 : Apologies for the late review. This looks really good! I've attached a V4 with some minor comments for your consideration. They do not change the underlying messaging, this was my attempt to make the point a bit more clear.
Zowe.Support.Conformance.V1-Near.Final.rev4.xlsx

@armstro
Copy link
Contributor Author

armstro commented Jun 11, 2021

Thanks Rose - regarding "how the notification" will occur to the TSC regarding emergency fixes - @nkocsis was working with the TSC on how customer issues could be raised and be better visible - maybe there is something that can be done for emergency defects? Just a thought

Regarding filling in some of the links - the TSC is working on how to describe core (zowe/community#1213) , active contributor so we are getting closer to having something to present to John.

@nkocsis
Copy link
Member

nkocsis commented Jun 11, 2021

Once we have a process in place where we know that the squads are reviewing the incoming customer issues then I would expect that it would handle "Emergency" defects.

@armstro
Copy link
Contributor Author

armstro commented Jun 28, 2021

Proposed 1 slide for July 21 webinar - draft - 2-3 mins max

2021 July Zowe Quarterly Webinar Series - Support Conformance Slide .pptx

@pdubz3
Copy link
Contributor

pdubz3 commented Jun 29, 2021

Bruce, thanks for taking the time to create a slide for the webinar. As we discussed offline I created a second draft using mostly your content but focusing a little more on the value of the program to the Zowe user, and then the steps a vendor takes and the principles of the program. I left the original but added a second slide for the alternate draft.
2021.July.Zowe.Quarterly.Webinar.Series.-.Support.Conformance.Slide.pptx

@armstro
Copy link
Contributor Author

armstro commented Jun 29, 2021

Thanks Mike - your slide is great - let's go with it.

@armstro
Copy link
Contributor Author

armstro commented Jul 13, 2021

Zowe.Support.Conformance.V1-Near.Final.rev5.xlsx

Captured link to define "core" in the spreadsheet

@armstro
Copy link
Contributor Author

armstro commented Jul 16, 2021

Zowe.Support.Conformance.V1-Near.Near.Final.rev6.xlsx

If we are going to preview (and request comments for) the Support Conformance at the up coming Zowe Quarterly Webinar then I thought I would tweak some of the open gaps we had in prior spreadsheet. Please read closely - I incorporated Rose's comments. I think we need to take to TSC for comment but can be done while we ask for comments in the wider community. Maybe we have a new issue created to link to in the Webinar slides?

Few things to note:

  • Full time participation in Security team is best practice but I made working part time when a security issue has been identified by the vendor a requirement
  • Regarding PTF creation - I can't imagine when a vendor would do their own build? I say they need to use the community build process for PTFs but emergency fixes can be done whatever they feel necessary?
  • I say they need to work with TSC at the next TSC meeting on any code changes needed in core as the result of an emergency fix - emailing someone didn't seem enough (we could have an issue raised with some security tag but I thought meeting might be best way to start)

All of this can be tweaked further

@RASakach for the life of me I can get urls in the text to be actual links - my excel skills are lacking. If you know the steps please advise.

@armstro
Copy link
Contributor Author

armstro commented Jul 19, 2021

Zowe.Support.Conformance.V1-Final July 19 2021.xlsx

@RASakach
Copy link
Contributor

@armstro Please review to ensure all URLs have been changed - I believe I found all of them.
Zowe.Support.Conformance.V1-Final.July.19.2021+links.xlsx

@armstro
Copy link
Contributor Author

armstro commented Aug 20, 2021

Thanks x100 Rose - links look good and I checked each one - they work!

@PeterFandelAtRocket
Copy link
Contributor

Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PeterFandelAtRocket PeterFandelAtRocket

@armstro armstro

@pdubz3 pdubz3

Labels
Epic
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@jmertic @PeterFandelAtRocket @solsu01 @nkocsis @armstro @pdubz3 @RASakach