diff --git a/site/docs/guidance/assurance-ecosystem.md b/site/docs/guidance/assurance-ecosystem.md index c62a788f..d70b5278 100644 --- a/site/docs/guidance/assurance-ecosystem.md +++ b/site/docs/guidance/assurance-ecosystem.md @@ -8,22 +8,40 @@ tags: # Understanding the Assurance Ecosystem -In 2021, the UK Government's Centre for Data Ethics and Innovation released their [AI Assurance Roadmap](https://www.gov.uk/government/publications/the-roadmap-to-an-effective-ai-assurance-ecosystem). -This publication set an agenda and series of recommendations for how to build an effective AI Assurance ecosystem, including specifying key roles and responsibilities. -The following diagram shows a key set of actors, identified in this report: +In 2021, the UK Government's Centre for Data Ethics and Innovation released +their +[AI Assurance Roadmap](https://www.gov.uk/government/publications/the-roadmap-to-an-effective-ai-assurance-ecosystem). +This publication set an agenda and series of recommendations for how to build an +effective AI Assurance ecosystem, including specifying key roles and +responsibilities. The following diagram shows a key set of actors, identified in +this report: ![This diagram depicts the AI assurance ecosystem, illustrating interactions between AI supply chain participants, AI Assurance Service Providers, Independent Researchers, and Supporting Structures like regulators and standards bodies.](../assets/images/actors.png) -As the diagram depicts, certain actors have a direct influence into the (simplified) supply chain for AI systems. -For instance, organisations may have dedicated teams internally who are responsible for quality assurance of products or services (e.g. compliance with safety standards, adherence to data privacy and protection legislation). -However, there is a growing marketplace of independent assurance providers who offer consultancy or services to other companies or organisations.[^market] - -[^market]: For example, [Credo AI](https://www.credo.ai/) offer a paid-for service that comprises an interactive dashboard and set of tools to help companies comply with existing and emerging policies and regulation. Whereas, other organisations, such as the [Ada Lovelace Institute](https://www.adalovelaceinstitute.org/project/algorithmic-impact-assessment-healthcare/) have developed open-source tools for teams to implement within their own projects. - -This is a helpful starting point for gaining some purchase on the complex set of interacting roles and responsibilities that collectively make up what is admittedly a hard to delineate assurance ecosystem. -Rather than trying to build a map of this ecosystem, we can instead focus on some of the typical roles and responsibilities that the different actors have. - - diff --git a/site/docs/guidance/components.md b/site/docs/guidance/components.md index b5ba1572..b5846799 100644 --- a/site/docs/guidance/components.md +++ b/site/docs/guidance/components.md @@ -11,12 +11,19 @@ tags: In this section we will look at the core elements of an assurance case and how they relate to one another. -There are many ways to construct an assurance case and several standards exist to help users adopt shared practices. -For instance, the [Goal Structuring Notation](https://scsc.uk/r141C:1?t=1) has thorough and comprehensive documentation for building assurance cases that align with their community standard. - -Trustworthy and Ethical Assurance is inspired by GSN's form of argument-based assurance, but aims to simplify the process of developing, communicating, and evaluating an argument and the evidence that justifies it, in order to make the process more open and inclusive to a broader community of stakeholders and users. -That is, we prioritise *accessibility* and *simplicity*. -The trade-off is that assurance cases developed using our platform are *less expressive* than others, but (hopefully) easier to understand. +There are many ways to construct an assurance case and several standards exist +to help users adopt shared practices. For instance, the +[Goal Structuring Notation](https://scsc.uk/r141C:1?t=1) has thorough and +comprehensive documentation for building assurance cases that align with their +community standard. + +Trustworthy and Ethical Assurance is inspired by GSN's form of argument-based +assurance, but aims to simplify the process of developing, communicating, and +evaluating an argument and the evidence that justifies it, in order to make the +process more open and inclusive to a broader community of stakeholders and +users. That is, we prioritise _accessibility_ and _simplicity_. The trade-off is +that assurance cases developed using our platform are _less expressive_ than +others, but (hopefully) easier to understand. !!! warning "A Note on Terminology" @@ -27,7 +34,7 @@ All assurance cases contain the following core elements: ```mermaid flowchart TD G1[Goal Claim] - C1([Context]) + C1([Context]) S1[\Strategy\] P1[Property Claim]; E1[(Evidence)]; @@ -46,14 +53,18 @@ There are two types of claims: #### Goal Claims -A *goal claim* serves to direct the process of developing an assurance case towards some value or principle that is desirable or significant. -For instance, it may be important to communicate how a product is 'Sustainable', how an algorithmic decision-making system is 'Explainable', or how the deployment of some service is 'Fair'. -The type of goal chosen will determine the set of lower-level property claims and evidence that are *relevant* and *necessary* for the overall assurance case. -As such, a goal claim should be the first element to be established. -Although, like all elements, it can be iteratively revised and refined as the assurance process develops. +A _goal claim_ serves to direct the process of developing an assurance case +towards some value or principle that is desirable or significant. For instance, +it may be important to communicate how a product is 'Sustainable', how an +algorithmic decision-making system is 'Explainable', or how the deployment of +some service is 'Fair'. The type of goal chosen will determine the set of +lower-level property claims and evidence that are _relevant_ and _necessary_ for +the overall assurance case. As such, a goal claim should be the first element to +be established. Although, like all elements, it can be iteratively revised and +refined as the assurance process develops. -Because a goal claim for will be *high-level*, it will not have the necessary specificity to link directly to evidence. -Consider the following example. +Because a goal claim for will be _high-level_, it will not have the necessary +specificity to link directly to evidence. Consider the following example. ```mermaid graph TD @@ -61,8 +72,8 @@ graph TD The outputs of our system are *explainable*.`"]; ``` -Here, *explainable* is a broad goal that is insufficiently operationalised or specified. -Resolving this requires the use of additional elements. +Here, _explainable_ is a broad goal that is insufficiently operationalised or +specified. Resolving this requires the use of additional elements. !!! info "Multiple Goals and Modular Arguments" @@ -70,20 +81,29 @@ Resolving this requires the use of additional elements. #### Property Claim(s) -Goal claims need to be succinct and accessible. -However, this comes at the cost of *specificity*. -For instance, what does it mean to deploy a service in a fair manner, or to develop an explainable system? -Property claims help to answer such questions. - -In one respect, property claims can be treated as lower-level goals[^gsn]. -That is, when formulated they represent aspirations that may need to be established and justified through linking to evidence. - -[^gsn]: In the GSN standard, all claims are treated as goals and no distinction is made between goal claims and property claims. Our methodology maintains consistency with this standard, which is why property claims have the same type as goal claims, but adds an additional descriptive layer to better represent the ethical process of deliberation and reflection (see section on [Operationalising Principles](operationalising-principles.md)) - -An assurance case may have only one goal claim[^modularity], but multiple property claims. -Collectively, the property claims serve to establish the central argument for how the goal claim has been established by detailing properties of a project or the system that help justify why the top-level goal has been sufficiently established. -That is, they are the additional premises that support the conclusion. -Consider the following example. +Goal claims need to be succinct and accessible. However, this comes at the cost +of _specificity_. For instance, what does it mean to deploy a service in a fair +manner, or to develop an explainable system? Property claims help to answer such +questions. + +In one respect, property claims can be treated as lower-level goals[^gsn]. That +is, when formulated they represent aspirations that may need to be established +and justified through linking to evidence. + +[^gsn]: + In the GSN standard, all claims are treated as goals and no distinction is + made between goal claims and property claims. Our methodology maintains + consistency with this standard, which is why property claims have the same + type as goal claims, but adds an additional descriptive layer to better + represent the ethical process of deliberation and reflection (see section on + [Operationalising Principles](operationalising-principles.md)) + +An assurance case may have only one goal claim[^modularity], but multiple +property claims. Collectively, the property claims serve to establish the +central argument for how the goal claim has been established by detailing +properties of a project or the system that help justify why the top-level goal +has been sufficiently established. That is, they are the additional premises +that support the conclusion. Consider the following example. ```mermaid graph TD @@ -103,12 +123,16 @@ graph TD ### Evidence -Evidence is what grounds an assurance case. -Whereas goal claims orient a case and property claims help specify and establish an argument, evidence is what provides the basis for trusting the validity of the case as a whole. +Evidence is what grounds an assurance case. Whereas goal claims orient a case +and property claims help specify and establish an argument, evidence is what +provides the basis for trusting the validity of the case as a whole. -The types of evidence that need to be communicated will depend on the claims being put forward. -For instance, if a claim is made about user's attitudes towards some technology or system, then findings from a user workshop may be needed. -Alternatively, if the claim is about a model's performance exceeding some threshold, then evidence about the test will be needed (e.g. benchmarking scores and methodology). +The types of evidence that need to be communicated will depend on the claims +being put forward. For instance, if a claim is made about user's attitudes +towards some technology or system, then findings from a user workshop may be +needed. Alternatively, if the claim is about a model's performance exceeding +some threshold, then evidence about the test will be needed (e.g. benchmarking +scores and methodology). ```mermaid graph TD @@ -119,15 +143,15 @@ graph TD !!! info "Evidential Standards" Similar to a legal case, where evidence needs to be admissible, relevant, and reliable, there are also standards for which types of evidence are appropriate in a given context. - + In some cases, technical standards may exist that can help bolster the trustworthiness of an argument, by allowing a project team to show how their actions adhere to standards set by an external community. - + In other cases, consensus may only emerge through the communication and evaluation of the evidence itself. ## Context -There are various types of context statements that can be added to the core elements of an assurance case. -For instance, consider the following example: +There are various types of context statements that can be added to the core +elements of an assurance case. For instance, consider the following example: ```mermaid graph RL @@ -140,8 +164,9 @@ There are two types of links that are used in Trustworthy and Ethical Assurance. ### Support Links -The primary link used in Trustworthy and Ethical assurance cases is a *support link*. -These links represent a uni-directional relationship between two elements, such that the parent element is *supported by* the child element. +The primary link used in Trustworthy and Ethical assurance cases is a _support +link_. These links represent a uni-directional relationship between two +elements, such that the parent element is _supported by_ the child element. @@ -157,12 +182,15 @@ They are rendered as follows: - Goal Claim to Property Claim - Strategy to Property Claim - Property Claim to Property Claim - - Property Claim to Evidence + - Property Claim to Evidence ### Context Links -Context links provide additional information for relevant elements, which has a constraining effect on the scope of the claim being made. -For instance, goal claims made about a system may be constrained by a specific use context (e.g. an algorithm may operate fairly in the context of a highly constrained information environment where input data follow a particular structure). +Context links provide additional information for relevant elements, which has a +constraining effect on the scope of the claim being made. For instance, goal +claims made about a system may be constrained by a specific use context (e.g. an +algorithm may operate fairly in the context of a highly constrained information +environment where input data follow a particular structure). They are rendered as follows: @@ -185,11 +213,19 @@ Some examples of contextual information that could be added include: #### Evidential Claims -If the rationale for selecting some evidence to support a specific property claim (or set of claims) is not clear, an intermediate 'evidential claim' may be required. +If the rationale for selecting some evidence to support a specific property +claim (or set of claims) is not clear, an intermediate 'evidential claim' may be +required. -For instance, the relevance of a partial dependency plot as supporting evidence for how a machine learning model is interpretable may be clear to some stakeholders, but a) this depends on prior expertise and b) may not address further questions, such as why individual feature importance is sufficient for establishing interpretability. +For instance, the relevance of a partial dependency plot as supporting evidence +for how a machine learning model is interpretable may be clear to some +stakeholders, but a) this depends on prior expertise and b) may not address +further questions, such as why individual feature importance is sufficient for +establishing interpretability. -An evidential claim would help provide further clarity, by making explicit any assumptions made by the project team (e.g. interpretations of the system's behaviour will only be undertaken by trained experts). +An evidential claim would help provide further clarity, by making explicit any +assumptions made by the project team (e.g. interpretations of the system's +behaviour will only be undertaken by trained experts). ```mermaid graph TD diff --git a/site/docs/guidance/index.md b/site/docs/guidance/index.md index d7c595f4..f82422a8 100644 --- a/site/docs/guidance/index.md +++ b/site/docs/guidance/index.md @@ -10,52 +10,96 @@ tags: Assurance is about building trust. -Consider the following scenario. -You are in the market for a new car and go to a local dealership. -One of the sales advisors convinces you to buy a second hand car that later turns out to have an issue with the engine. -Frustrated, you take the car back and the sales advisor apologises. -They explain that all their second hand cars undergo a thorough assessment before they are placed on the market, and go on to process a return and get you a different car. -You are reassured and happy, but only for a short period of time. -Yet again, the car turns out to have a problem with the engine—the same problem as before! -The sales advisor tries to convince you that this is just a series of unlucky incidents, but without clear evidence to support their claims, you do not trust them and take your business elsewhere. +Consider the following scenario. You are in the market for a new car and go to a +local dealership. One of the sales advisors convinces you to buy a second hand +car that later turns out to have an issue with the engine. Frustrated, you take +the car back and the sales advisor apologises. They explain that all their +second hand cars undergo a thorough assessment before they are placed on the +market, and go on to process a return and get you a different car. You are +reassured and happy, but only for a short period of time. Yet again, the car +turns out to have a problem with the engine—the same problem as before! The +sales advisor tries to convince you that this is just a series of unlucky +incidents, but without clear evidence to support their claims, you do not trust +them and take your business elsewhere. -> Assurance involves communicating reasons and evidence that help people understand and evaluate the *trustworthiness* of a claim (or series of claims) about a system or technology. +> Assurance involves communicating reasons and evidence that help people +> understand and evaluate the _trustworthiness_ of a claim (or series of claims) +> about a system or technology. -In the above example, the sales advisor needed to provide assurance that their cars were *safe* or *reliable*, but the claims they made about the assessment process were undermined by the evidence. +In the above example, the sales advisor needed to provide assurance that their +cars were _safe_ or _reliable_, but the claims they made about the assessment +process were undermined by the evidence. -In another context, things may have gone differently. -For instance, you may have a higher level of trust with friends or other professionals (e.g. doctors, teachers) than with car sales persons. -And, as such, you may be more likely to accept claims in the absence of evidence or in spite of repeated instances of contrary evidence. -However, the relationship between trust and assurance is significant in nearly all contexts, and especially so in some domains (e.g. safety-critical engineering). +In another context, things may have gone differently. For instance, you may have +a higher level of trust with friends or other professionals (e.g. doctors, +teachers) than with car sales persons. And, as such, you may be more likely to +accept claims in the absence of evidence or in spite of repeated instances of +contrary evidence. However, the relationship between trust and assurance is +significant in nearly all contexts, and especially so in some domains (e.g. +safety-critical engineering). -Therefore, having clear methods, processes, and tools for communicating assurance and building trust is crucial. -And, this is increasingly important in the design, development, and deployment of data-driven technologies. +Therefore, having clear methods, processes, and tools for communicating +assurance and building trust is crucial. And, this is increasingly important in +the design, development, and deployment of data-driven technologies. ## Building Trust and Communicating Trustworthiness -There are many benefits and risks associated with the design, development, and deployment of data-driven technologies, such as machine learning (ML) or artificial intelligence (AI). -And, therefore, many organisations and companies find themselves in a situation of needing to communicate to customers, users, or stakeholders how they have maximised the benefits and minimised the risks associated with their product, service, or system. -For example, an organisation building an autonomous vehicle may need to explain how their system is safe, secure, fair, explainable, among other goals. -How they achieve this will depend on myriad contextual factors, including who they are communicating with (e.g. regulators, potential customers). +There are many benefits and risks associated with the design, development, and +deployment of data-driven technologies, such as machine learning (ML) or +artificial intelligence (AI). And, therefore, many organisations and companies +find themselves in a situation of needing to communicate to customers, users, or +stakeholders how they have maximised the benefits and minimised the risks +associated with their product, service, or system. For example, an organisation +building an autonomous vehicle may need to explain how their system is safe, +secure, fair, explainable, among other goals. How they achieve this will depend +on myriad contextual factors, including who they are communicating with (e.g. +regulators, potential customers). -Consider the goal of *safety* with respect to the following questions: +Consider the goal of _safety_ with respect to the following questions: -- *How was the performance of the system evaluated, and how will it be monitored?* There are many metrics that can be used to evaluate the performance of an autonomous vehicle, including metrics that assess the performance of components of the vehicle such as the object recognition system (e.g. its accuracy, robustness, interpretability) as well as metrics that consider broader societal or environmental impact (e.g. sustainability, usability and accessibility). -- *Who carried out processes such as failure mode and effects analysis or stakeholder engagement?* Diverse and inclusive teams can help reduce the likelihood of unintended consequences, especially those that may arise due to the presence of overlooked biases in the system (e.g. how were trade-offs in the design process handled and who was consulted). -- *Who will use the system?* Whether a system is safe depends, in part, on who the users are (e.g. trained professionals versus members of the public)—a key challenge in the area of *human factors* research. +- _How was the performance of the system evaluated, and how will it be + monitored?_ There are many metrics that can be used to evaluate the + performance of an autonomous vehicle, including metrics that assess the + performance of components of the vehicle such as the object recognition system + (e.g. its accuracy, robustness, interpretability) as well as metrics that + consider broader societal or environmental impact (e.g. sustainability, + usability and accessibility). +- _Who carried out processes such as failure mode and effects analysis or + stakeholder engagement?_ Diverse and inclusive teams can help reduce the + likelihood of unintended consequences, especially those that may arise due to + the presence of overlooked biases in the system (e.g. how were trade-offs in + the design process handled and who was consulted). +- _Who will use the system?_ Whether a system is safe depends, in part, on who + the users are (e.g. trained professionals versus members of the public)—a key + challenge in the area of _human factors_ research. -These are just three examples of how claims made about the *safety* of a system, in response to a small set of possible questions, are highly contextual. -If we were to consider different goals (e.g. fairness, explainability) or different areas of application (e.g. healthcare, defence and security), the types of claims that would be needed to provide assurance for the goal in question could be very different. +These are just three examples of how claims made about the _safety_ of a system, +in response to a small set of possible questions, are highly contextual. If we +were to consider different goals (e.g. fairness, explainability) or different +areas of application (e.g. healthcare, defence and security), the types of +claims that would be needed to provide assurance for the goal in question could +be very different. -And yet, in spite of the contextual variation, there are similarities that span the assurance of data-driven technologies, both within and between different domains. -There are, for instance, a recurring set of goals (or, principles) that people emphasise when asked about the ethical or societal issues related to data-driven technologies (e.g. fairness and bias, transparency and explainability). -And, furthermore, there is a well-established set of techniques and standards in place for building trust through transparent and accessible forms of communication. +And yet, in spite of the contextual variation, there are similarities that span +the assurance of data-driven technologies, both within and between different +domains. There are, for instance, a recurring set of goals (or, principles) that +people emphasise when asked about the ethical or societal issues related to +data-driven technologies (e.g. fairness and bias, transparency and +explainability). And, furthermore, there is a well-established set of techniques +and standards in place for building trust through transparent and accessible +forms of communication. -Trustworthy and ethical assurance is a framework that is anchored in these similarities and existing techniques, but also recognises the importance of understanding variation and difference. -At the centre of this framework is a *methodology* and *platform* for building *assurance cases*. -These cases communicate how a specific goal has been established within the context of the design, development, or deployment of a data-driven technology. -The methodology serves as a guide for developing the cases, while the platform helps to build and communicate them with the wider community or stakeholders. +Trustworthy and ethical assurance is a framework that is anchored in these +similarities and existing techniques, but also recognises the importance of +understanding variation and difference. At the centre of this framework is a +_methodology_ and _platform_ for building _assurance cases_. These cases +communicate how a specific goal has been established within the context of the +design, development, or deployment of a data-driven technology. The methodology +serves as a guide for developing the cases, while the platform helps to build +and communicate them with the wider community or stakeholders. -The following sections of this user guidance serve as an introduction to the trustworthy and ethical assurance framework. -The first set of sections are concerned with the methodology, including the context for why it was developed. -The second set of sections introduce the platform and serve as a practical guide for how to design, develop, and share assurance cases. +The following sections of this user guidance serve as an introduction to the +trustworthy and ethical assurance framework. The first set of sections are +concerned with the methodology, including the context for why it was developed. +The second set of sections introduce the platform and serve as a practical guide +for how to design, develop, and share assurance cases. diff --git a/site/docs/guidance/why-is-assurance-important.md b/site/docs/guidance/why-is-assurance-important.md index 6878fc62..69287cdc 100644 --- a/site/docs/guidance/why-is-assurance-important.md +++ b/site/docs/guidance/why-is-assurance-important.md @@ -1,21 +1,30 @@ -— -status: draft -tags: - - assurance - - trust -— +— status: draft tags: -# Why is Assurance Important? +- assurance +- trust — -Data-driven technologies, such as artificial intelligence, have a complex lifecycle. -In some cases, this complexity is further heightened by the scale at which a system is deployed (e.g. social media platforms with international reach). +# Why is Assurance Important? -The scale and complexity of certain data-driven technologies has already been clearly communicated by others, such as [this excellent infographic](https://anatomyof.ai) from the AI Now Institute showing the many societal impacts and touch points that occur in the development of Amazon’s smart speaker. -Therefore, it is not necessary to revisit this point here. -However, it is important to explain why this complexity and scale matters for the purpose of trustworthy and ethical assurance. -There are three (well-rehearsed) reasons that are salient within the context of the assurance ecosystem: +Data-driven technologies, such as artificial intelligence, have a complex +lifecycle. In some cases, this complexity is further heightened by the scale at +which a system is deployed (e.g. social media platforms with international +reach). -1. Complexity: as the complexity of a system increases it becomes harder to maintain transparency and explainability. -2. Scalability: the risk of harm increases proportional to the scale of a system, and mechanisms for holding people or organisations accountable become harder to implement. -3. Autonomous behaviour: where data-driven technologies are used to enable autonomous behaviour, opportunities for responsible human oversight are reduced. +The scale and complexity of certain data-driven technologies has already been +clearly communicated by others, such as +[this excellent infographic](https://anatomyof.ai) from the AI Now Institute +showing the many societal impacts and touch points that occur in the development +of Amazon’s smart speaker. Therefore, it is not necessary to revisit this point +here. However, it is important to explain why this complexity and scale matters +for the purpose of trustworthy and ethical assurance. There are three +(well-rehearsed) reasons that are salient within the context of the assurance +ecosystem: +1. Complexity: as the complexity of a system increases it becomes harder to + maintain transparency and explainability. +2. Scalability: the risk of harm increases proportional to the scale of a + system, and mechanisms for holding people or organisations accountable become + harder to implement. +3. Autonomous behaviour: where data-driven technologies are used to enable + autonomous behaviour, opportunities for responsible human oversight are + reduced. diff --git a/site/mkdocs.yml b/site/mkdocs.yml index a0244e34..453cbf93 100644 --- a/site/mkdocs.yml +++ b/site/mkdocs.yml @@ -134,7 +134,7 @@ nav: - Home: index.md - About: about.md - Guidance: - - Intro: guidance/index.md + - Intro: guidance/index.md - Platform Details: - About: platform-details/about.md - Installation Instructions: platform-details/installation.md