New case-studies directory with README

[rficcaglia]

Not sure if this is following @ultrasaurus 's process...ie should I create a "meta" issue to create the PR to create the proposal? Or do I just add the PR to create this proposal and maybe we reference it in a new issue?

[ultrasaurus]

suggested title: Cloud Native Security Case Studies
or just "Case Studies" since the rest is implicit

[ultrasaurus]

I don't understand what would be included in "process examples" ... For assessments, we've had a bunch of process hiccups having to do with coordinating the reviewers and format of the assessment document, which is probably not what you mean here.

[rficcaglia]

No, not referring to the logistics of the assessment itself...more about the effectiveness and efficiency of:

• initial engagement with a project,
• how the project completed their part
• how the assessors completed their part (was it ad hoc, did it follow a standard, was it Q&A based, code review based, pen testing, etc etc)
• how did the project process the assessment (did they agree, did they understand, can they act on the recommendations or are they too hypothetical, etc)
• how did the larger CNCF community process the assessment (did it inform their decision making, was it specific enough, was it helpful in planning deployments, etc etc)

[ultrasaurus]

Nit: vision is not to define a process, the vision is for the process to result in awesomeness

Suggestion:

• The process of collecting case studies results in knowledge-sharing that reduces security risks, providing good data about vendors and open source projects

I do think "vendors or FOSS projects just looking to market their wares indiscriminately" is a good point, but was a bit conflicted about putting it that way... maybe we could break that out as an issue using "suggestion" or "proposal" template to better define the process we've been following

[dshaw]

@rficcaglia Could you review and update?

[rficcaglia]

I have merged suggestions

[rficcaglia]

I merged in Sarah's suggestions ... is there anything more I need to do for the PR to be merged and closed?

…

On Tue, Jun 25, 2019 at 8:24 PM Dan Shaw ***@***.***> wrote: @rficcaglia <https://github.com/rficcaglia> Could you review and update? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#195>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGENIWNELUXYUAOXMGEOSTP4LVPPANCNFSM4HUBGBUQ> .

[rficcaglia]

@mhausenblas not being familiar with micro-site stuff...is there a quick start on how best to accomplish @ultrasaurus 's suggestion to incorporate the use case index into the microsite?

[ultrasaurus]

"in-toto" is all lower-case -- can you fix here and below?

see comment above -- maybe leave out this section, and we can add back in if/when the security assessment team creates any of these. I don't want to imply that they need to add a step in their process to prepare more documentation.

[rficcaglia]

No the idea isn't that the assessment team creates the case study; someone who did not do the assessment, nor part of the project team, should write these up and give unbiased feedback on what worked and what didn't.

The case study is separate and apart from the assessment itself. it is meta.

ie the assessments try to result in awesomeness, and the case studies are assessments of the assessments to ensure they are resulting in awesomeness!

[ultrasaurus]

@rficcaglia @mhausenblas pls also see maybe related #237

[lumjjb]

Hmm.. I am curious to when/how to decide if a case study should be done on something or not? I think a suggestion of when would be a good time to do it would be useful - maybe after 3 months of activity? Or if the project is not making progress after X weeks, or after each big project milestone?

Having a guideline (kind of like security reviews) on how long one should expect to take on a case study, and a case study outline document (asking the type of questions you ask above - which we can add later) sounds useful as well.

[rficcaglia]

I wouldn't expect more than a calendar week to gather feedback from everyone, write up some observations from that feedback, and syndicate the ideas with a few folks to get feedback...nothing too onerous. that said if we find value in doing more thorough case studies, we can add more structure and depth over iterations 3 months after the final report seems like a good time to review how everything went...and as you note, to make sure progress is being made. that said, I think the review of progress is best done by updating the assessment periodically (annually, biennially, depending on risk level)...that's a different issue I'm writing some docs on...coming soon!

…

On Tue, Jul 16, 2019 at 6:14 AM Brandon Lum ***@***.***> wrote: Hmm.. I am curious to when/how to decide if a case study should be done on something or not? I think a suggestion of when would be a good time to do it would be useful - maybe after 3 months of activity? Or if the project is not making progress after X weeks, or after each big project milestone? Having a guideline (kind of like security reviews) on how long one should expect to take on a case study, and a case study outline document (asking the type of questions you ask above - which we can add later) sounds useful as well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#195>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGENIVZV5JACQC57XHLNH3P7XCTHANCNFSM4HUBGBUQ> .

[rficcaglia]

I'm gonna park this for now --- my original intent was based on a flurry of activity, thinking that there would be several parallel assessments ongoing and that it would be important to single out some for detailed review.

I think realistically the pace will be much slower, and with everyone sort of overseeing the assessment as it proceeds, eg. in-toto and OPA.

If we need to revisit this I can always revive this PR.