From 4f37ed3014c21fa7fc3fad1976d44bf969199c90 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tao=20Bojl=C3=A9n?= Date: Fri, 19 Jun 2026 19:41:03 +0100 Subject: [PATCH 1/5] initial data abstraction layer PRD draft --- .../001-data-abstraction-layer.md | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 product-requirement-docs/001-data-abstraction-layer.md diff --git a/product-requirement-docs/001-data-abstraction-layer.md b/product-requirement-docs/001-data-abstraction-layer.md new file mode 100644 index 0000000..b124700 --- /dev/null +++ b/product-requirement-docs/001-data-abstraction-layer.md @@ -0,0 +1,60 @@ +# PRD 001: Data Abstraction Layer + +## Problem statement + +T&S at scale has often been outsourced to BPOs. Increasingly, it is outsourced to AI. ROOST is preparing to build tooling to support AI-powered safety. + +More context can lead to better content moderation, whether it's done by a human or an AI agent. Today, a Coop reviewer can only see limited data (whatever is pushed in). Accessing anything beyond that -- like an account's history or other posts -- has to be looked up in other tools, or populated using the Partial Items endpoint (which cannot populate information about relationships). Getting richer context therefore requires custom integration, and right now that work needs to be redone for each T&S tool, because nothing abstracts over an org's objects, relationships, and signals. + +**Success would mean**: an org implements that integration *once* -- declaring its ontology/object graph and exposing a query/enrichment contract -- and every consumer reuses it, whether it's a UI for humans in Coop, agentic review, Osprey, or whatever comes next. + +## Background + +The current [roadmap](https://github.com/roostorg/community/blob/main/roadmap.md#data-abstraction-layer) describes two foundational components needed to prepare ROOST's projects for AI-powered features. One of these is a data abstraction layer (DAL) that lets both tools understand org-specific data models. This document focuses on the DAL and a Coop integration. There are a few more initial notes on this [here](https://github.com/orgs/roostorg/discussions/61#discussioncomment-16495095). + +Today, Coop lets orgs configure its data model via Item Types. There are three kinds of Item: +- Content (any piece of user-generated content) +- Thread (an ordered collection of Content) +- User (an account or profile on the org's platform) + +This is flexible, but has limitations. You cannot easily model that e.g. one user follows another, or that a comment is a reply to a specific other comment in the same Thread. + +Coop users push Items into Coop via our API, and items may then be shown in the Manual Review Tool (MRT). Items can be enriched at review time via the Partial Items feature -- the user can implement an API that returns further details about Items. This means that the latest version of an Item can be fetched on-demand. + +## Goals + +The roadmap frames the Data Abstraction Layer as being a prerequisite for agentic T&S tooling, but I do not think this is actually a strict necessity. You could certainly build AI content moderation into Coop as it exists today. However, a unified API to query for more information is a very good idea that benefits both human *and* AI-driven review. + +We want to solidify & generalize the good ideas in Coop's Item Types datamodel and Partial Items API. + +The concrete story we are initially targeting is: + +1. An org implements the graph API, deployed as a new DAL service. + - We can write an agent skill to make this process smoother. +2. In Coop's Manual Review Tool, if this DAL is configured, the Partial Items API is replaced with a query to the DAL. +3. This next part is up to us: do we want to focus on agentic review? Or enriching the data that is visible to human reviewers in Coop's UI? + +## Non-goals + +- We are not building a graph database. +- The DAL will not support arbitrary multi-hop graph traversal, or advanced graph queries like community detection. +- The DAL does not persist data long-term (beyond, perhaps, caching). Data continues to live in the org's source systems. +- The Safety Decision Taxonomy from ROOST's roadmap is not included in this PRD -- though the DAL may be a sensible place to centralize this, too. + +## What + +In brief: a new module, that Coop (and eventually perhaps also Osprey) both depend on. It exposes a read-only GraphQL API that matches an organization's ontology. The user then writes GraphQL resolvers that translate incoming GraphQL requests into reads against their source systems. It is completely agnostic to these source systems -- could be data warehouses, operational DBs, or further API calls to other services. The DAL centralizes reads of all an organization's data into a single well-defined GraphQL API. + +## Open questions + +1. Where will agentic review live -- in Coop's manual review tool, in Osprey's rules engine, or in some new service? + +## Risks + +1. Solidifying an ontology and writing resolvers is an additional engineering lift that orgs may be reluctant to take on. + +## Scope + +## Definitions + +- **Ontology**: Can also be called a schema. A description of the object types that exist, what properties they have, and the possible relationships between them. From 635d9617fcc1adde5e67b78b561171a755658252 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tao=20Bojl=C3=A9n?= Date: Mon, 22 Jun 2026 18:30:17 +0100 Subject: [PATCH 2/5] data abstraction layer PRD: expand scope, risks, and GraphQL framing Co-Authored-By: Claude Opus 4.8 (1M context) --- .../001-data-abstraction-layer.md | 76 ++++++++++++++++--- 1 file changed, 66 insertions(+), 10 deletions(-) diff --git a/product-requirement-docs/001-data-abstraction-layer.md b/product-requirement-docs/001-data-abstraction-layer.md index b124700..309b1a0 100644 --- a/product-requirement-docs/001-data-abstraction-layer.md +++ b/product-requirement-docs/001-data-abstraction-layer.md @@ -2,15 +2,15 @@ ## Problem statement -T&S at scale has often been outsourced to BPOs. Increasingly, it is outsourced to AI. ROOST is preparing to build tooling to support AI-powered safety. +In content moderation, more context can lead to better decisions, whether those decisions are taken by a human or an AI agent. Today, a Coop reviewer can only see limited data (whatever is pushed in). Accessing anything beyond that -- like an account's history or other posts -- has to be looked up in other tools, or populated using the Partial Items endpoint (which cannot populate information about relationships). If an org decides that some new field would be useful for moderation, then it requires engineering lift to get that data into each T&S tool. -More context can lead to better content moderation, whether it's done by a human or an AI agent. Today, a Coop reviewer can only see limited data (whatever is pushed in). Accessing anything beyond that -- like an account's history or other posts -- has to be looked up in other tools, or populated using the Partial Items endpoint (which cannot populate information about relationships). Getting richer context therefore requires custom integration, and right now that work needs to be redone for each T&S tool, because nothing abstracts over an org's objects, relationships, and signals. +In other words, getting richer context requires N custom integrations, because nothing abstracts over an org's objects, relationships, and signals. -**Success would mean**: an org implements that integration *once* -- declaring its ontology/object graph and exposing a query/enrichment contract -- and every consumer reuses it, whether it's a UI for humans in Coop, agentic review, Osprey, or whatever comes next. +**Success would mean**: an org implements that integration *once* -- declaring its ontology/object graph and exposing a query/enrichment API -- and every consumer reuses it, whether it's a UI for humans in Coop, agentic review, Osprey, or whatever comes next. ## Background -The current [roadmap](https://github.com/roostorg/community/blob/main/roadmap.md#data-abstraction-layer) describes two foundational components needed to prepare ROOST's projects for AI-powered features. One of these is a data abstraction layer (DAL) that lets both tools understand org-specific data models. This document focuses on the DAL and a Coop integration. There are a few more initial notes on this [here](https://github.com/orgs/roostorg/discussions/61#discussioncomment-16495095). +The current [roadmap](https://github.com/roostorg/community/blob/main/roadmap.md#data-abstraction-layer) describes two components needed to prepare ROOST's projects for AI-powered features. One of these is a data abstraction layer (DAL) that lets both Coop and Osprey understand org-specific data models. This PRD focuses on the DAL and a Coop integration. There are a few more initial notes on this [here](https://github.com/orgs/roostorg/discussions/61#discussioncomment-16495095). Today, Coop lets orgs configure its data model via Item Types. There are three kinds of Item: - Content (any piece of user-generated content) @@ -23,38 +23,94 @@ Coop users push Items into Coop via our API, and items may then be shown in the ## Goals -The roadmap frames the Data Abstraction Layer as being a prerequisite for agentic T&S tooling, but I do not think this is actually a strict necessity. You could certainly build AI content moderation into Coop as it exists today. However, a unified API to query for more information is a very good idea that benefits both human *and* AI-driven review. - -We want to solidify & generalize the good ideas in Coop's Item Types datamodel and Partial Items API. +A unified API to query for more information benefits both human, automated, and AI-driven review. We want to solidify & generalize the good ideas in Coop's Item Types datamodel and Partial Items API. The concrete story we are initially targeting is: 1. An org implements the graph API, deployed as a new DAL service. - We can write an agent skill to make this process smoother. 2. In Coop's Manual Review Tool, if this DAL is configured, the Partial Items API is replaced with a query to the DAL. -3. This next part is up to us: do we want to focus on agentic review? Or enriching the data that is visible to human reviewers in Coop's UI? + +This new DAL service will not be required to run Coop today, though that may change in the future. But for now, we will deprecate the Partial Items API and replace it with the DAL. Any org that uses Partial Items will have to migrate. ## Non-goals - We are not building a graph database. -- The DAL will not support arbitrary multi-hop graph traversal, or advanced graph queries like community detection. +- We are also not building a new GraphQL library -- we will re-use an existing one. +- The DAL will not support unbounded multi-hop graph traversal, or advanced graph queries like community detection. - The DAL does not persist data long-term (beyond, perhaps, caching). Data continues to live in the org's source systems. - The Safety Decision Taxonomy from ROOST's roadmap is not included in this PRD -- though the DAL may be a sensible place to centralize this, too. ## What +An org running Coop decides that they want to show more data in the reviewer UI. This is the flow I envision: + +1. A developer clones a Roost template repo, e.g. `roostorg/data-abstraction-layer-template`. +2. Inside this template, they first define their ontology (what object types exist in their org, what properties do they have, and how do they relate to each other). +3. Then, they write e.g. Python code to implement resolvers that serve this ontology as a GraphQL API. + - This might only involve reading from a single database, or it could mean calling out to various data warehouses, APIs, and more. The goal is to unify everything into one API. + - The template provides the scaffolding to do this with a great DevX, providing the library, test setup, etc. to make the process easy. + +For steps 2. and 3., an agent skill provides an easy walkthrough that does a lot of the heavy lifting. The developer's job then primarily becomes thinking through how to map the org's source system(s) together into a unified API. + +An example ontology for a simple social network might look like: + +> Object types +> - User +> - ID (UUID) +> - email (string) +> - username (string) +> - created_at (datetime) +> - Post +> - ID (UUID) +> - content (string) +> - created_at (datetime) +> +> Relationships +> - (User)-[:follows]->(User) +> - (User)-[:author_of]->(Post) +> - (Post)-[:reply_to]->(Post) + In brief: a new module, that Coop (and eventually perhaps also Osprey) both depend on. It exposes a read-only GraphQL API that matches an organization's ontology. The user then writes GraphQL resolvers that translate incoming GraphQL requests into reads against their source systems. It is completely agnostic to these source systems -- could be data warehouses, operational DBs, or further API calls to other services. The DAL centralizes reads of all an organization's data into a single well-defined GraphQL API. +### Why GraphQL? + +Instead of MCP, REST, bespoke protocol... (TBD). + +## Future work + +There are many things we could do. To keep things simple, this initial PRD is tightly scoped. Future work can/should include: + +- Integration with Osprey for ingest-time enrichment +- Ingest-time enrichment for Coop, i.e. fetching more data when an item is first sent in, before running rules +- Agentic moderators that can actually explore data via the DAL + ## Open questions -1. Where will agentic review live -- in Coop's manual review tool, in Osprey's rules engine, or in some new service? +1. How should this interact with Coop's existing Item Types concept? Can we replace that entirely to avoid duplication? +2. Where will agentic review live -- in Coop's manual review tool, in Osprey's rules engine, or in some new service? +3. How important is ingestion-time enrichment in Coop? Do we actually need it in v1? ## Risks 1. Solidifying an ontology and writing resolvers is an additional engineering lift that orgs may be reluctant to take on. +2. GraphQL makes it easy to create N+1s. + - We will have to solve this the standard way (using dataloaders, see e.g. [here](https://www.apollographql.com/tutorials/dataloaders-dgs/03-data-loaders-under-the-hood) and [here](https://strawberry.rocks/docs/guides/dataloaders)). +3. GraphQL queries can also get expensive, if they query a lot of expensive joins. + - We can mitigate this with query complexity/size limits, and great observability out of the box +4. Source systems may be slow to respond with data. + - This can be a real problem, especially in high-volume use cases! + - TBD +5. If the DAL is optional infrastructure, there is a risk that orgs will just not use it. ## Scope +- The API will be read-only. +- (TBD: authn/authz) +- (TBD: caching) +- (TBD: perf with slow source systems) +- (TBD: v1 vs later) + ## Definitions - **Ontology**: Can also be called a schema. A description of the object types that exist, what properties they have, and the possible relationships between them. From 530b026f05cc9d22f2b5da6c7bf86328f0aa6bac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tao=20Bojl=C3=A9n?= Date: Tue, 23 Jun 2026 13:32:49 +0100 Subject: [PATCH 3/5] flesh out PRD --- .../001-data-abstraction-layer.md | 93 ++++++++++++++----- 1 file changed, 72 insertions(+), 21 deletions(-) diff --git a/product-requirement-docs/001-data-abstraction-layer.md b/product-requirement-docs/001-data-abstraction-layer.md index 309b1a0..52f4285 100644 --- a/product-requirement-docs/001-data-abstraction-layer.md +++ b/product-requirement-docs/001-data-abstraction-layer.md @@ -53,7 +53,7 @@ An org running Coop decides that they want to show more data in the reviewer UI. For steps 2. and 3., an agent skill provides an easy walkthrough that does a lot of the heavy lifting. The developer's job then primarily becomes thinking through how to map the org's source system(s) together into a unified API. -An example ontology for a simple social network might look like: +An example ontology for a simple blogging platform might look like: > Object types > - User @@ -61,29 +61,63 @@ An example ontology for a simple social network might look like: > - email (string) > - username (string) > - created_at (datetime) -> - Post +> - BlogPost +> - ID (UUID) +> - content (string) +> - created_at (datetime) +> - Comment > - ID (UUID) > - content (string) > - created_at (datetime) > > Relationships > - (User)-[:follows]->(User) -> - (User)-[:author_of]->(Post) -> - (Post)-[:reply_to]->(Post) +> - (User)-[:author_of]->(BlogPost) +> - (User)-[:author_of]->(Comment) +> - (Comment)-[:reply_to]->(BlogPost) + +An example query to fetch a user, their number of followers, and their 10 most recent blog posts might look like: + +```gql +query { + user(id: "019ef42a-6531-7039-8a8b-096d1ee2db64") { + email + username + authorOf { + blogPosts(orderBy: [{ created_at: DESC }], first: 10) { + edges { + node { + content + created_at + } + } + } + } + # The schema auto-generates reverse relationships, too + reverseFollows { + user { + totalCount + } + } + } +} +``` In brief: a new module, that Coop (and eventually perhaps also Osprey) both depend on. It exposes a read-only GraphQL API that matches an organization's ontology. The user then writes GraphQL resolvers that translate incoming GraphQL requests into reads against their source systems. It is completely agnostic to these source systems -- could be data warehouses, operational DBs, or further API calls to other services. The DAL centralizes reads of all an organization's data into a single well-defined GraphQL API. ### Why GraphQL? -Instead of MCP, REST, bespoke protocol... (TBD). +GraphQL is a near-perfect fit for this use case for several reasons. -## Future work +For one, thinking in graphs is both common & intuitive for investigative use cases. GraphQL puts this first. It's also an existing standard which means that humans & LLMs are familiar with it, and there's a large ecosystem. -There are many things we could do. To keep things simple, this initial PRD is tightly scoped. Future work can/should include: +GraphQL also natively supports [introspection](https://graphql.org/learn/introspection/) which lets callers dynamically learn the schema (i.e. the ontology) of a given API. This is great since this will vary depending on the org that's implementing the DAL. We may not need to build a custom API to share the ontology between services. -- Integration with Osprey for ingest-time enrichment -- Ingest-time enrichment for Coop, i.e. fetching more data when an item is first sent in, before running rules -- Agentic moderators that can actually explore data via the DAL +Finally, writing a GraphQL API that calls out to one or more source systems is very accessible. Developers write [resolvers](https://graphql.org/learn/execution/) for objects or fields, and the GraphQL library automatically handles running these to serve an incoming GraphQL query. + +Using GraphQL for querying does not preclude MCP; they are different layers. It's still possible to build a GraphQL-over-MCP transport for LLM agents in the future if we wish, which may be useful for auth purposes. + +The main alternatives would be REST, but this loses the graph-native shape and introspection, or more advanced graph query languages like [Cypher](https://en.wikipedia.org/wiki/Cypher_(query_language)), which is a lot more advanced -- more comparable to SQL than to REST, for example. ## Open questions @@ -94,23 +128,40 @@ There are many things we could do. To keep things simple, this initial PRD is ti ## Risks 1. Solidifying an ontology and writing resolvers is an additional engineering lift that orgs may be reluctant to take on. -2. GraphQL makes it easy to create N+1s. - - We will have to solve this the standard way (using dataloaders, see e.g. [here](https://www.apollographql.com/tutorials/dataloaders-dgs/03-data-loaders-under-the-hood) and [here](https://strawberry.rocks/docs/guides/dataloaders)). -3. GraphQL queries can also get expensive, if they query a lot of expensive joins. - - We can mitigate this with query complexity/size limits, and great observability out of the box -4. Source systems may be slow to respond with data. + - This means that we need to make it worth it. +2. If the DAL is optional infrastructure, there is a risk that orgs will just not use it. + - Again, we have to make it worth it via product features. +3. GraphQL makes it easy to create N+1s. + - We will have to solve this the standard way using dataloaders, see e.g. [here](https://www.apollographql.com/tutorials/dataloaders-dgs/03-data-loaders-under-the-hood) and [here](https://strawberry.rocks/docs/guides/dataloaders). +4. GraphQL queries can get expensive, e.g. if they write underlying queries with a lot of expensive joins. + - We can mitigate this with query complexity/size limits, and great observability out of the box. +5. Source systems may be slow to respond with data. - This can be a real problem, especially in high-volume use cases! - - TBD -5. If the DAL is optional infrastructure, there is a risk that orgs will just not use it. + - We cannot control these source systems so the best thing we can do here is around observability, so it's easy to see *what* is slow. +6. The DAL may overload source systems if used heavily for T&S. + - This is an inherent risk. Orgs will have to manage this, and again we can ensure that it's easy to understand which queries the DAL is executing. +7. Organizations may be reluctant to build APIs that allow broad access to all or most data in their platform, for security reasons. + - We can mitigate this via some combination of audit logging, solid authentication + security fundamentals, and (in the future) field-level authorization. ## Scope - The API will be read-only. -- (TBD: authn/authz) -- (TBD: caching) -- (TBD: perf with slow source systems) -- (TBD: v1 vs later) +- We will re-use existing libraries rather than publishing something entirely new. This means that the main work will be building the wiring and a great template to make it easy for orgs to build their own DAL. +- Clients will authenticate via API bearer tokens. An API token gives access to the entire API. +- Observability: sensible logs, and importantly, OTel-compatible traces. + +## Future work + +There are many things we could do. To keep things simple, this initial PRD is tightly scoped. Future work can/should include: + +- Integration with Osprey for ingest-time enrichment +- Ingest-time enrichment for Coop, i.e. fetching more data when an item is first sent in, before running rules +- Agentic moderators that can actually explore data via the DAL +- Caching queries for performance (with good headers so clients know if the response was cached) +- Field-level authorization. + ## Definitions +- **GraphQL**: The Graph Query Language, an API format originally built by Meta, and now used widely. - **Ontology**: Can also be called a schema. A description of the object types that exist, what properties they have, and the possible relationships between them. From 9ae390ed242f5c79cb8050ff22ca794c0a839494 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tao=20Bojl=C3=A9n?= Date: Tue, 23 Jun 2026 15:19:44 +0100 Subject: [PATCH 4/5] flesh out requirements --- .../001-data-abstraction-layer.md | 50 +++++++++++++++---- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/product-requirement-docs/001-data-abstraction-layer.md b/product-requirement-docs/001-data-abstraction-layer.md index 52f4285..ea2fc69 100644 --- a/product-requirement-docs/001-data-abstraction-layer.md +++ b/product-requirement-docs/001-data-abstraction-layer.md @@ -41,7 +41,7 @@ This new DAL service will not be required to run Coop today, though that may cha - The DAL does not persist data long-term (beyond, perhaps, caching). Data continues to live in the org's source systems. - The Safety Decision Taxonomy from ROOST's roadmap is not included in this PRD -- though the DAL may be a sensible place to centralize this, too. -## What +## Approach An org running Coop decides that they want to show more data in the reviewer UI. This is the flow I envision: @@ -119,16 +119,53 @@ Using GraphQL for querying does not preclude MCP; they are different layers. It' The main alternatives would be REST, but this loses the graph-native shape and introspection, or more advanced graph query languages like [Cypher](https://en.wikipedia.org/wiki/Cypher_(query_language)), which is a lot more advanced -- more comparable to SQL than to REST, for example. +## Scope + +- The API will be read-only. +- We will re-use existing libraries rather than publishing something entirely new. The main work is the wiring and a great template, not a new GraphQL library. +- v1 targets Coop's MRT as the single consumer; Osprey integration is out of scope for this PRD. +- The (deployable) template repo + agent skill are in-scope deliverables. + +## Functional requirements + +1. A read-only GraphQL API derived from a declared ontology (i.e. object types, properties, and relationships). +2. The API handles relationships between object types, including self-referential; reverse relationships are auto-generated. +3. The API supports GraphQL introspection, so consumers can discover the ontology at runtime. +4. Relationship connections with [Relay-style pagination](https://relay.dev/graphql/connections.htm). +5. Query depth is bounded by a configurable maximum. +6. Developers write resolvers per object type/field. This is source-system agnostic. +7. A template repo with resolver scaffolding, a test harness, examples, and support for dataloaders to prevent N+1s. +8. When configured, Coop enriches from the DAL instead of Partial Items. +9. Coop's Partial Items API is deprecated with a migration path. +10. Bearer-token auth -- a token grants access to the entire API. +11. Requests are audit-logged with caller identity and query content. +12. Structured logs, OTel traces (per-resolver, with source-system spans), and RED metrics (count, latency, error rate). + +## Non-functional requirements + +1. The DAL adds no more than 100ms request latency (p95) on top of querying the source systems. +2. Coop stays functional when the DAL is absent or degraded. +3. Users can add TLS for all requests. +4. API tokens are revocable. +5. The DAL service is self-hosted by the org, deployable from the template with minimal custom code. +6. An agent skill guides ontology + resolvers. +7. The ontology is evolvable without coordinated downtime with Coop, as long as changes are backwards-compatible. However, since changing the ontology requires writing or modifying resolvers, it requires a code deploy. + +## Success criteria + +- >=1 users running the DAL in production +- The Partial Items API is deprecated, and the DAL replaces it fully + ## Open questions -1. How should this interact with Coop's existing Item Types concept? Can we replace that entirely to avoid duplication? +1. How should this interact with Coop's existing Item Types concept? Can we replace that entirely to avoid duplication? That would expand the scope but may be worth doing. 2. Where will agentic review live -- in Coop's manual review tool, in Osprey's rules engine, or in some new service? 3. How important is ingestion-time enrichment in Coop? Do we actually need it in v1? ## Risks 1. Solidifying an ontology and writing resolvers is an additional engineering lift that orgs may be reluctant to take on. - - This means that we need to make it worth it. + - This means that we need to make it worth it. Initially, replacing Partial Items will be that carrot, but soon after (post-this-PRD) we should add support for Osprey/Coop ingest-time enrichment and/or agentic moderation. 2. If the DAL is optional infrastructure, there is a risk that orgs will just not use it. - Again, we have to make it worth it via product features. 3. GraphQL makes it easy to create N+1s. @@ -143,13 +180,6 @@ The main alternatives would be REST, but this loses the graph-native shape and i 7. Organizations may be reluctant to build APIs that allow broad access to all or most data in their platform, for security reasons. - We can mitigate this via some combination of audit logging, solid authentication + security fundamentals, and (in the future) field-level authorization. -## Scope - -- The API will be read-only. -- We will re-use existing libraries rather than publishing something entirely new. This means that the main work will be building the wiring and a great template to make it easy for orgs to build their own DAL. -- Clients will authenticate via API bearer tokens. An API token gives access to the entire API. -- Observability: sensible logs, and importantly, OTel-compatible traces. - ## Future work There are many things we could do. To keep things simple, this initial PRD is tightly scoped. Future work can/should include: From dab14a4ef4446f65ad328101a396685d397ee442 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tao=20Bojl=C3=A9n?= Date: Tue, 7 Jul 2026 21:16:39 +0100 Subject: [PATCH 5/5] address comments, round 1 --- .../001-data-abstraction-layer.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/product-requirement-docs/001-data-abstraction-layer.md b/product-requirement-docs/001-data-abstraction-layer.md index ea2fc69..73ed1ce 100644 --- a/product-requirement-docs/001-data-abstraction-layer.md +++ b/product-requirement-docs/001-data-abstraction-layer.md @@ -17,7 +17,7 @@ Today, Coop lets orgs configure its data model via Item Types. There are three k - Thread (an ordered collection of Content) - User (an account or profile on the org's platform) -This is flexible, but has limitations. You cannot easily model that e.g. one user follows another, or that a comment is a reply to a specific other comment in the same Thread. +This is flexible, but has limitations. While it supports foreign-key relationships between Item Types, users cannot easily model many-to-many relationships such as users following each other. Coop users push Items into Coop via our API, and items may then be shown in the Manual Review Tool (MRT). Items can be enriched at review time via the Partial Items feature -- the user can implement an API that returns further details about Items. This means that the latest version of an Item can be fetched on-demand. @@ -46,12 +46,12 @@ This new DAL service will not be required to run Coop today, though that may cha An org running Coop decides that they want to show more data in the reviewer UI. This is the flow I envision: 1. A developer clones a Roost template repo, e.g. `roostorg/data-abstraction-layer-template`. -2. Inside this template, they first define their ontology (what object types exist in their org, what properties do they have, and how do they relate to each other). -3. Then, they write e.g. Python code to implement resolvers that serve this ontology as a GraphQL API. +2. Inside this template, they first define their ontology (what object types exist in their org, what properties do they have, and how do they relate to each other). The ontology is defined in some markup language like YAML or JSON. +3. Then, they write e.g. Python code to implement query resolvers that serve this ontology as a GraphQL API. - This might only involve reading from a single database, or it could mean calling out to various data warehouses, APIs, and more. The goal is to unify everything into one API. - The template provides the scaffolding to do this with a great DevX, providing the library, test setup, etc. to make the process easy. -For steps 2. and 3., an agent skill provides an easy walkthrough that does a lot of the heavy lifting. The developer's job then primarily becomes thinking through how to map the org's source system(s) together into a unified API. +For steps 2. and 3., an agent skill provides an easy walkthrough that does a lot of the heavy lifting. The developer's job then primarily becomes thinking through how to map the org's source system(s) together into a unified API. Note that this is a read-only API; the developer does *not* need to implement any mutations. An example ontology for a simple blogging platform might look like: @@ -159,13 +159,13 @@ The main alternatives would be REST, but this loses the graph-native shape and i ## Open questions 1. How should this interact with Coop's existing Item Types concept? Can we replace that entirely to avoid duplication? That would expand the scope but may be worth doing. -2. Where will agentic review live -- in Coop's manual review tool, in Osprey's rules engine, or in some new service? -3. How important is ingestion-time enrichment in Coop? Do we actually need it in v1? +2. How important is ingestion-time enrichment in Coop? Do we actually need it in v1? ## Risks 1. Solidifying an ontology and writing resolvers is an additional engineering lift that orgs may be reluctant to take on. - - This means that we need to make it worth it. Initially, replacing Partial Items will be that carrot, but soon after (post-this-PRD) we should add support for Osprey/Coop ingest-time enrichment and/or agentic moderation. + - This means that we need to make it worth it! + - We can formalize a longer-term vision: things like enrichment, Osprey integration, custom UIs for moderators in Coop, more flexible investigations in Coop (think Retool). 2. If the DAL is optional infrastructure, there is a risk that orgs will just not use it. - Again, we have to make it worth it via product features. 3. GraphQL makes it easy to create N+1s.