Emailed comment from Microsoft

[h-m-f-t]

Department of Homeland Security
Cybersecurity and Infrastructure Security Agency (CISA)
245 Murray Lane
Washington, D.C. 20528-0380

Submitted via electronic mail to bod.feedback@cisa.dhs.gov: Microsoft’s Response to Binding Operational Directive 20-01 (draft), Develop and Publish a Vulnerability Disclosure Policy

To whom it may concern,

Microsoft appreciates the opportunity to review and comment on the draft Binding Operational Directive 20-01 (draft Directive), Develop and Publish a Vulnerability Disclosure Policy. For more than a decade, Microsoft has been developing and evolving its approach to vulnerability disclosure and handling, seeking to protect our customers and enhance ecosystem resilience by collaborating with security researchers and other vulnerability finders and reporters. Our experience has demonstrated the value that such collaboration can have for security as well as some of the challenges that may arise in the creation and implementation of effective vulnerability disclosure policies and handling procedures.

We’re encouraged that CISA is taking the critical step of requiring Federal civilian executive branch agencies to publish and operationalize vulnerability disclosure policies, and we welcome the opportunity to support this important initiative. Given the significant work that has been done to date to develop and share international standards and guidance for vulnerability disclosure and handling, we also appreciate CISA’s efforts to align the draft Directive with existing standards and good practices.

Overall, the draft Directive’s requirements, guidance, and template highlight critical elements of and provide useful context for developing and publishing vulnerability disclosure policies and implementing appropriate handling procedures. However, we encourage CISA to take into account, both in evolving the draft Directive and assessing how to further direct and support agencies moving forward, that developing policies and describing procedures is often more straightforward than managing implementation. In particular, we anticipate that establishing new procedures and using them consistently and at scale, especially given the complicated nature of Federal information technology infrastructure, may pose significant challenges. In addition, we offer the following specific feedback for CISA’s consideration as it refines the draft Directive’s requirements and guidance and explores the role of CISA and other resources:

• Partner effectively with vulnerability reporters (a.k.a. “security researchers” or “finders”) by clearly defining policies and expectations, prioritizing consistent communication, and recognizing reporters’ contributions to security when vulnerabilities are confirmed;
• Ensure sufficient capacity for processing and remediating confirmed vulnerabilities by supporting operations, including by exploring additional roles for CISA and/or other resources; and
• Prepare for complicated vulnerability disclosure and handling scenarios by facilitating readiness to both coordinate closely with third parties and mitigate risks to legacy infrastructure.

Partner effectively with vulnerability reporters

Our experience has demonstrated that consistent communication, clearly defined expectations, and transparency are foundational to effective partnerships with security researchers. In 2011, our Microsoft Security Response Center (MSRC) announced that we were adopting a new policy, called Coordinated Vulnerability Disclosure (CVD), in response to feedback from the security community and to demonstrate our focus on protecting technology users.[1] Since then, prioritizing communication throughout the vulnerability investigation and remediation process and providing appropriate transparency regarding shifts in expectations has helped to foster collaboration. Our support for these practices was also validated during our participation in a multi-stakeholder process on cybersecurity vulnerabilities convened by the National Telecommunications and Information Administration.[2] This process led to the publication of “Vulnerability Disclosure Attitudes and Actions: A Research Report.”[3] Demonstrating that communication is key, the report conveyed results of a survey in which most security researcher respondents shared that they expect regular updates on their reported vulnerability, including regarding its investigation and remediation; 95 percent expected to be notified when their reported vulnerability was resolved.[4]

The survey further demonstrated support for prioritizing communication over adhering to a resolution timeline. While many surveyed researchers expressed a desire for an anticipated resolution timeline:

...only 18% of the researchers that expressed an expectation of a resolution timeline thought that vendors should conform to a timeline without regard [for] the circumstances of a particular bug. Maintaining a definite resolution date, then, is less important than communicating the decision-making involved in determining resolution priority in a transparent manner, allowing [researchers] to calibrate their expectations.[5]

Consistent with the finding that communication and transparency related to the resolution process are more critical for collaboration than holding to resolution timelines, and given broader risk management issues associated with timelines, Microsoft’s CVD policy is focused on coordination and does not include timelines for resolution of a confirmed vulnerability. Specifically, our policy asks that the security researcher allows the vendor the opportunity to diagnose and offer fully tested updates, workarounds, or other corrective measures before any party discloses detailed vulnerability or exploit information to the public.[6]

While CISA’s draft Directive demonstrates the importance of communicating with vulnerability reporters and conveying expectations, there are further opportunities to bolster and prioritize that focus. For example, the draft Directive appropriately requires agencies to specify where reports should be sent (i.e., to provide a “front door” for reporters) and include within their policies a statement explaining when reporters can anticipate acknowledgement of their reports – both of which are foundational for establishing good communication and setting expectations. To further encourage collaboration, we recommend that CISA make clear to agencies that ongoing communication throughout the investigation and potential remediation process are critical to the successful implementation of policies.

Moreover, we encourage CISA to help agencies shift their focus away from providing resolution timelines[7] and instead recognize the importance of committing to regular cadences by which they will engage in meaningful follow up with reporters. Sticking to resolution timelines may be difficult and even undermine security due to variability in: a) risk prioritization of different vulnerabilities; and b) challenges associated with different investigations, remediation efforts, or compatibility testing circumstances. However, communicating updates, providing context for shifting expectations, and otherwise collaborating directly with security researchers is consistently achievable. What’s more, focusing on ongoing communication rather than resolution timelines will not only strengthen collaboration with reporters that better understand remediation efforts but also improve security and operations, ensuring that such efforts reflect risk priorities and that mitigations work in practice (e.g., fixes are sufficiently tested prior to deployment).

While we have learned from first-hand experience that imposing deadlines on resolving vulnerabilities is unhelpful for managing risk, timelines for other activities may be valuable. As such, in requiring agencies to set target timelines and track metrics, we encourage CISA to differentiate between timelines for acknowledging reports, assessing the validity of reports, resolving confirmed vulnerabilities, and notifying reporters of outcomes. Acknowledging reports and notifying reporters of outcomes (i.e.., after a resolution has been achieved) can and should happen according to a more consistent timeline. Delay in confirming receipt of reports, especially when a policy has committed an agency to responding within a specific timeframe, risks undermining an agency’s rapport and ability to coordinate with reporters. However, timelines for investigating and resolving vulnerabilities may reasonably vary, and without context regarding types of vulnerabilities or circumstances, metrics about median timeframes may be less meaningful.

Exploring how individual agencies and CISA might recognize researchers for their contributions to security could also foster collaboration between the Federal government and vulnerability finders and incentivize coordinated disclosure. Beyond bug bounties, Microsoft seeks to recognize and reward researchers for their contributions through a variety of mechanisms. For instance, as part of our Security Update Guide, as applicable and appropriate for each security vulnerability that’s addressed, MSRC acknowledges both internal and external researchers for their efforts.[8] To consistently measure and enable acknowledgment of especially impactful contributions, MSRC has developed a Researcher Recognition Program.[9] We also recognize our “most valuable” researchers across dimensions like high accuracy, high volume, and high impact in reporting vulnerabilities.[10] Likewise, the draft Directive could encourage individual agencies to acknowledge researchers that submitted confirmed vulnerabilities after they resolve them. In addition, CISA could look across agencies and acknowledge researchers that are making especially impactful contributions to the mission of protecting Federal civilian executive branch agencies.

Ensure sufficient capacity for processing and remediating confirmed vulnerabilities

Through Microsoft’s efforts to manage vulnerability handling processes across a wide array of products and services and our partnership with other organizations as they stand up and operationalize vulnerability disclosure policies, our experience consistently demonstrates that implementing policies is often challenging, in particular at the outset when report influx and resource demand are uncertain. Moreover, to the extent that implementation challenges disrupt efforts to operate according to defined policies (e.g., to meet timelines for response or remediation), challenges may be exacerbated by vulnerability reporters who are frustrated by inconsistent communication and unmet expectations.

Given this context, the draft Directive’s attention to helping agencies scale up slowly – by starting with at least one internet-accessible system or service in scope and adding others over time – is especially important. Likewise, while bug bounty programs can helpfully incentivize researcher focus on high-priority systems and services, from our engagement with others, we’ve learned that the experience of some organizations has been that bug bounty programs can also risk driving more vulnerability reports and exhausting bandwidth when teams are unprepared for an influx. As a result, exclusively focusing on vulnerability disclosure policies and handling processes at the outset may be a reasonable approach for agencies. CISA might also consider the following ideas to support agency capacity and operations:

• Highlight potential challenges with vulnerability mitigation or resolution, the importance of testing fixes to limit disruptions, and resources to support such efforts.
• Acknowledge that agencies might consider leveraging external services, such as those provided by HackerOne or Bugcrowd, to support their efforts to establish and implement a vulnerability disclosure policy and handling process; note that many organizations, including the Dept. of Defense and private sector companies of various sizes, have benefited from doing so.
• Offer a way to help address vulnerability reports for systems and services that are out of scope.
• Clarify that, at an agency’s request, CISA will assist with disclosure of reports to vendors when they’re inappropriately sent to agencies (e.g., because of a real or perceived regulatory role); note that while the current draft Directive acknowledges that CISA may do so, ensuring that reports reach vendors that can address vulnerabilities is critical to ecosystem security and an activity that, taken on by CISA, may enable agencies to focus their resources on other efforts.
• List all agency webforms and points of contact for vulnerability reports in a central location on a CISA webpage along with a CISA email alias to which vulnerability reporters should reach out if they do not receive a response to their agency outreach; note that, while accomplishing the same objective of CISA testing whether it receives a response after contacting an agency through its identified mechanism (e.g., webform and/or email alias), this approach has the additional benefits of helping to support a potentially overwhelmed agency and guarding against circumstances in which a frustrated reporter might proceed with public disclosure of a vulnerability.

Prepare for complicated vulnerability disclosure and handling scenarios

As CISA’s draft Directive acknowledges, some scenarios may complicate agency efforts to craft clear vulnerability disclosure policies or implement consistent vulnerability handling procedures. For instance, the draft Directive highlights that agencies may receive reports of potential vulnerabilities impacting not only their systems and services but also third-party products or services because: 1) systems or services included in their policy’s scope incorporate third-party products or services; or 2) reporters perceive that, due to their regulatory role, agencies should receive vulnerability information relating to the products or services of organizations within their sector. In addition, reporters may submit information to agencies if they do not want to report vulnerabilities directly to vendors or do not receive a response from vendors. They may also submit information to agencies when, in order to successfully exploit a system or service, a chain of vulnerabilities must be leveraged – and that chain implicates both an agency and a third party.

When reports of potential vulnerabilities in third-party products or services are submitted to agencies, two key issues must be addressed; both are identified in the draft Directive, but we encourage CISA to consider further emphasizing them in any future versions. First, ensuring that affected vendors have visibility into potential vulnerability information is critical to both mitigating agency risk and improving ecosystem security; whether through an enhanced CISA role (as requested by agencies), coordination with CERT/CC, or otherwise, the Directive should underline the importance of disclosure to vendors and establish appropriate processes accordingly. Second, to the extent that agencies wish to include within the scope of their policies any third-party products or services (for example, as dependencies of an agency’s own systems or services), the draft Directive’s reference to Dept. of Justice (DOJ) guidance[11] is critical. In implementing such guidance, we recommend coordinating tightly with third party providers, both to facilitate visibility into potential vulnerability information and to ensure that research activities are appropriately scoped to mitigate risks imposed on such third party offerings. (By way of example and as highlighted by the DOJ’s guidance, where the third party is a provider of shared platforms, such as cloud computing services, research activities must be scoped in a way that prevents any risk to co-tenants.)

Beyond instances in which third parties or both agencies and third parties are implicated by vulnerability disclosure, agencies may receive reports of potential vulnerabilities that affect many vendors and organizations (e.g., Heartbleed). In such circumstances, agencies should also be aware of the need to share vulnerability information more broadly. CISA could reference guidance from FIRST, a confederation of incident response teams that cooperate and share best practices,[12] for multi-party disclosure.[13]

In addition, agencies may receive reports of potential vulnerabilities affecting legacy infrastructure that they own and operate but for which fixes or mitigations are especially complex – given that they do not have access to the code impacted by a vulnerability, updates are not covered in contracts, or related circumstances. For example, in the case of Heartbleed, given the number of contractors implicated and the security maturity of affected infrastructure, for some Federal environments, there was a need to rebuild executables from the original code. CISA could acknowledge such potential challenges and point agencies to additional resources for handling these more complicated scenarios.

As stated at the outset, we applaud CISA for pursuing an important initiative – requiring Federal civilian executive branch agencies to publish and operationalize vulnerability disclosure policies – and for the approach taken in the draft Directive, which both focuses on many critical elements of vulnerability disclosure policies and handling processes and leverages existing standards and good practices to support implementation. We appreciate the opportunity to provide input and would welcome any further opportunities to engage with CISA at it moves forward with refining and issuing the draft Directive, including through policy and/or operational discussions with our MSRC experts.

Sincerely,
{signed}

Angela L. McKay
Senior Director
Customer Security & Trust Microsoft

[1]: https://msrc-blog.microsoft.com/2011/04/19/coordinated-vulnerability-disclosure-from-philosophy-to-practice/; https://blogs.technet.microsoft.com/ecostrat/2011/04/19/coordinated-vulnerability-disclosure-reloaded/.
[2]: https://www.ntia.doc.gov/other-publication/2016/multistakeholder-process-cybersecurity-vulnerabilities.
[3]: https://www.ntia.doc.gov/files/ntia/publications/2016_ntia_a_a_vulnerability_disclosure_insights_report.pdf.
[4]: Id.
[5]: Id. at 6.
[6]: https://www.microsoft.com/en-us/msrc/cvd. This approach is also consistent with the draft Directive’s finding that “it is generally ideal for any public disclosure to occur after a vulnerability has been fixed...”
[7]: For example, the current draft Directive notes that “While it is generally ideal for any public disclosure to occur after a vulnerability has been fixed, agencies have the primary responsibility of addressing vulnerabilities in a timely manner....Many in the security research community consider public disclosure of a vulnerability to be appropriate between 45 to 90 days after the first communication with the affected entity...Agencies may require that the researcher give the agency a defined window of time to address the vulnerability before public disclosure, but should not seek to limit publication after this window of time has passed...”
[8]: https://portal.msrc.microsoft.com/en-us/security-guidance/acknowledgments.
[9]: https://www.microsoft.com/en-us/msrc/researcher-recognition-program.
[10]: https://msrc-blog.microsoft.com/2019/08/07/announcing-2019-msrc-most-valuable-security-researchers/.
[11]: https://www.justice.gov/criminal-ccips/page/file/983996/download#page=4
[12]: https://www.first.org/about/mission
[13]: https://www.first.org/global/sigs/vulnerability-coordination/multiparty/