From ba6bd8674aa3be78a2c996150bf25c37d9dac7bb Mon Sep 17 00:00:00 2001 From: Raphael Date: Fri, 19 May 2023 09:17:28 -0700 Subject: [PATCH 01/23] Docs( Cloud ): removed redundant information (#576) (#578) Co-authored-by: Rajakavitha Kodhandapani --- content/dgraphcloud/admin/import-export.md | 58 ++-------- content/dgraphcloud/provision-backend.md | 5 +- content/graphql/quick-start/index.md | 119 ++++++++++----------- 3 files changed, 69 insertions(+), 113 deletions(-) diff --git a/content/dgraphcloud/admin/import-export.md b/content/dgraphcloud/admin/import-export.md index 3758b4e1..910b9bfd 100644 --- a/content/dgraphcloud/admin/import-export.md +++ b/content/dgraphcloud/admin/import-export.md @@ -5,58 +5,21 @@ weight = 4 parent = "cloud-admin" +++ -You can export your data from one Dgraph Cloud backend, and then import this data back into another Dgraph instance or Dgraph Cloud Backend. +## Exporting and Importing Data in Dgraph Cloud -## Exporting Data +You can export your data as an Administrator from one Dgraph Cloud backend, and then import this data back into another Dgraph instance or Dgraph Cloud backend. For more information about how to export data in Dgraph Cloud, see [Export data]({{< relref "howto/exportdata/export-data-cloud.md" >}}). You can also export data from Dgraph Cloud programatically using the Dgraph Cloud API. For more information, see [Cloud API documentation]({{< relref "dgraphcloud/cloud-api/backup.md" >}}). -You can export your data using JSON format. To do this, call the `export` mutation on `/admin/slash`. As an example, if your GraphQL endpoint is at `https://frozen-mango.us-west-2.aws.cloud.dgraph.io/graphql`, then the `/admin` endpoint for the schema is at `https://frozen-mango.us-west-2.aws.cloud.dgraph.io/admin/slash`. +To import data to Dgraph Cloud, see [live loader]({{< relref "howto/importdata/live-loader.md" >}}). -{{% notice "note" %}} -The `/admin/slash` endpoint requires [Authentication]({{}}). -{{% /notice %}} - -The following is an example of a GraphQL mutation to export data to JSON. - -```graphql -mutation { - export { - response { code message } - exportId - taskId - } -} -``` -Make sure to keep your `exportId` and `taskId` safe as you will need them for getting the signed URLs in order to download your export files. These URLs will be returned in the `signedUrls` output field and **they expire after 48 hours**. - -Export will usually return 3 files: -* `g01.gql_schema.gz`: The GraphQL schema file. This file can be re-imported via the [Schema APIs](/admin/schema) -* `g01.json.gz`: the data from your instance, which can be imported via live loader -* `g01.schema.gz`: This file is the internal Dgraph schema. If you have set up your backend with a GraphQL schema, then you should be able to ignore this file. - -The following is an example of GraphQL query to check the status of the export and get the signed URLs for downloading your export files. - -```graphql -query { - exportStatus ( - exportId:"" - taskId: "" - ){ - kind - lastUpdated - signedUrls - status - } -} -``` -## Exporting Data with Multi-Tenancy feature enabled +## Exporting Data with Multi-Tenancy feature enabled in Dgraph Cloud {{% notice "note" %}} -With Multi-Tenancy feature enabled, for any GraphQL request you will need to provide the `accessJWT` for the specific user in the `X-Dgraph-AccessToken` header. +With Multi-Tenancy feature enabled, for any GraphQL request you need to provide the `accessJWT` for the specific user in the `X-Dgraph-AccessToken` header. {{% /notice %}} You can trigger two types of exports: -* cluster-wide export: this is an export of the entire backend (including all namespaces). This request can be only triggered by the [*Guardian of Galaxy*]({{}}) users. -* namespace-specific export: this is an export of a specific namespace. This request can be triggered by the *Guardian of Galaxy* users and by the *Guardian of Namespace* users. +* Cluster-wide export: this is an export of the entire backend (including all namespaces). This request can be only triggered by the [*Guardian of Galaxy*](https://dgraph.io/docs/enterprise-features/multitenancy/#guardians-of-the-galaxy) users. +* Namespace-specific export: this is an export of a specific namespace. This request can be triggered by the *Guardian of Galaxy* users and by the *Guardian of Namespace* users. ### Cluster-wide Exports @@ -91,7 +54,7 @@ mutation { } } ``` -3. Once done, you can now send the following GraphQL mutation to get the `signedUrls` from where you can download your export files: +3. Once done, you can now send the following GraqhQL mutation to get the `signedUrls` from where you can download your export files: ```graphql query { exportStatus ( @@ -108,7 +71,7 @@ query { ### Namespace-specific Exports -Namespace-specific exports can be triggered by the *Guardian of Galaxy* users. In this case you can follow the same steps for the cluster-wide exports and replace the namespace value from `-1` to the namespace you want to export. It's important that you get the `accessJWT` token for the *Guardian of Galaxy* user and pass it in the `X-Dgraph-AccessToken` header. +Namespace-specific exports can be triggered by the *Guardian of Galaxy* users. In this case you can follow the same steps for the Cluster-wide exports and replace the namespace value from `-1` to the namespace you want to export. It's important that you get the `accessJWT` token for the *Guardian of Galaxy* user and pass it in the `X-Dgraph-AccessToken` header. E.g. if you want to export the namespace `0x123` your GraphQL request sent to the `/admin/slash` endpoint would look like: ```graphql @@ -133,6 +96,3 @@ mutation { } ``` -## Importing data with Live Loader - -It is possible to import data into a Dgraph Cloud backend using [live loader]({{}}). diff --git a/content/dgraphcloud/provision-backend.md b/content/dgraphcloud/provision-backend.md index a828ce45..a5b71414 100644 --- a/content/dgraphcloud/provision-backend.md +++ b/content/dgraphcloud/provision-backend.md @@ -1,4 +1,3 @@ - +++ date = "2023-02-14T22:25:17+11:00" title = "Provision a backend" @@ -8,7 +7,7 @@ weight = 1 +++ ### Before you begin -Signup to [Dgraph Cloud](https://cloud.dgraph.io) using **Sign in with Google**, **Sign in with GitHub** or any other email account that you prefer to use. +Log in to [Dgraph Cloud](https://cloud.dgraph.io) using **Sign in with Google**, **Sign in with GitHub** or any other email account that you prefer to use. ### Provision a backend @@ -37,4 +36,4 @@ That's it! You now have a running Dgraph backend. Time to build an app. The URL listed in "GraphQL Endpoint" is the URL at which Dgraph Cloud will serve data to your app. -You can copy it at any time to use in a GraphQL client application. +You can copy it at any time to use in a GraphQL client application. \ No newline at end of file diff --git a/content/graphql/quick-start/index.md b/content/graphql/quick-start/index.md index 3cf2a491..c05bd2d5 100644 --- a/content/graphql/quick-start/index.md +++ b/content/graphql/quick-start/index.md @@ -1,6 +1,6 @@ +++ title = "Quick Start" -description = "Go from an empty Dgraph database to a running GraphQL API in just one step : just define the schema of your graph and how you’d like to search it; Dgraph does the rest." +description = "Go from an empty Dgraph database to a running GraphQL API in just one step; just define the schema of your graph and how you’d like to search it; Dgraph does the rest." weight = 1 [menu.main] name = "Quick Start" @@ -8,85 +8,86 @@ weight = 1 parent = "graphql" +++ -You might be familiar with GraphQL types, fields and resolvers. Perhaps you've written an app that adds GraphQL over a REST endpoint or maybe over a relational database. If so, you know how GraphQL sits over those sources and issues many queries to translate the REST/relational data into something that looks like a graph. -There's none of that with Dgraph : you can generate a running GraphQL API with the associated graph backend just by deploying the GraphQL schema of your API; Dgraph does the rest. +When you write an application that implements GraphQL over a REST endpoint or maybe over a relational database, you know that GraphQL issues many queries to translate the REST/relational data into something that looks like a graph. You also have to be familiar with the GraphQL types, fields, and resolvers. +However, with Dgraph you can generate a running GraphQL API with the associated graph backend just by deploying the GraphQL schema of your API; Dgraph does the rest. +## Step 1: Run Dgraph -## Prerequisite : Run Dgraph +The easiest way to get Dgraph up and running is using the [Dgraph Cloud](https://cloud.dgraph.io). +You can log in to Dgraph cloud using **Sign in with Google**, **Sign in with GitHub** or any other email account that you prefer to use. -The easiest way to get Dgraph up and running is to [provision a Dgraph Cloud backend]({{< relref "provision-backend.md" >}}). +1. In the Dgraph cloud console, click **Launch new backend**. +1. Select a plan, cloud provider, and region that meets your requirements. +1. Type a name for your Dgraph cloud instance. +1. Click **Launch** +1. A new Dgraph cloud backend is created for you. + The URL listed in **GraphQL Endpoint** is the URL at which Dgraph cloud serves data to your app. You can copy it at any time to use in a GraphQL client application. +## Step 2: Deploy a GraphQL Schema -## Deploy a GraphQL Schema - -Access the [Dgraph Cloud console](https://cloud.dgraph.io). - -1. In the [Schema](https://cloud.dgraph.io/_/schema) tab of the Dgraph Cloud console, paste the following schema : +1. In the [Schema](https://cloud.dgraph.io/_/schema) tab of the Dgraph Cloud console, paste the following schema: ```graphql -type Product { - productID: ID! - name: String @search(by: [term]) - reviews: [Review] @hasInverse(field: about) -} + type Product { + productID: ID! + name: String @search(by: [term]) + reviews: [Review] @hasInverse(field: about) + } -type Customer { - username: String! @id @search(by: [hash, regexp]) - reviews: [Review] @hasInverse(field: by) -} + type Customer { + username: String! @id @search(by: [hash, regexp]) + reviews: [Review] @hasInverse(field: by) + } -type Review { - id: ID! - about: Product! - by: Customer! - comment: String @search(by: [fulltext]) - rating: Int @search -} -``` + type Review { + id: ID! + about: Product! + by: Customer! + comment: String @search(by: [fulltext]) + rating: Int @search + } + ``` - and Click `deploy` +2. Click `deploy` - That's it, now you've got a GraphQL API up and running and a graph database as a backend. +You now have a GraphQL API up and running and a graph database as a backend. -## Testing your GraphQL API +## Step 3: Test your GraphQL API -We will simply use the [API Explorer](https://cloud.dgraph.io/_/explorer) tab of your Dgraph cloud console, but you can access the `GraphQL endpoint` with any GraphQL clients such as [GraphQL Playground](https://github.com/prisma-labs/graphql-playground), [Insomnia](https://insomnia.rest/), [GraphiQL](https://github.com/graphql/graphiql), [Altair](https://github.com/imolorhe/altair) or Postman. +You can use the [API Explorer](https://cloud.dgraph.io/_/explorer) tab of your Dgraph cloud console, but you can access the `GraphQL endpoint` with any GraphQL clients such as [GraphQL Playground](https://github.com/prisma-labs/graphql-playground), [Insomnia](https://insomnia.rest/), [GraphiQL](https://github.com/graphql/graphiql), [Altair](https://github.com/imolorhe/altair) or Postman. If you want to use those clients, copy the `GraphQL endpoint` from the [Cloud dashboard](https://cloud.dgraph.io/_/dashboard). -You may want to use the introspection capability of the client to explore the schema, queries and mutations that were generated by Dgraph. +You may want to use the introspection capability of the client to explore the schema, queries, and mutations that were generated by Dgraph. ### A first GraphQL mutation To populate the database, -1. open the [API Explorer](https://cloud.dgraph.io/_/explorer) tab -2. paste the following code into the text area -3. click on the Execute Query button. - - - -```graphql -mutation { - addProduct(input: [ - { name: "GraphQL on Dgraph"}, - { name: "Dgraph: The GraphQL Database"} - ]) { - product { - productID - name - } - } - addCustomer(input: [{ username: "Michael"}]) { - customer { - username +1. Open the [API Explorer](https://cloud.dgraph.io/_/explorer) tab +2. Paste the following code into the text area: + ```graphql + mutation { + addProduct(input: [ + { name: "GraphQL on Dgraph"}, + { name: "Dgraph: The GraphQL Database"} + ]) { + product { + productID + name + } + } + addCustomer(input: [{ username: "Michael"}]) { + customer { + username + } } } -} -``` + ``` +3. Click **Execute Query** . -The GraphQL server will return a json response like: +The GraphQL server returns a json response similar to this: ```json { @@ -246,13 +247,9 @@ query { } } ``` -## Discussion - -Dgraph allows you to have a fully functional GraphQL API in minutes with a highly performant graph backend to serve complex nested queries. - -Moreover, you can update or change your schema freely and just re-deploy new versions. +## Conclusion -For GraphQL in Dgraph, you just concentrate on defining the schema of your graph and how you'd like to search that graph; Dgraph does the rest. +Dgraph allows you to have a fully functional GraphQL API in minutes with a highly performant graph backend to serve complex nested queries. Moreover, you can update or change your schema freely and just re-deploy new versions. For GraphQL in Dgraph, you just concentrate on defining the schema of your graph and how you'd like to search that graph; Dgraph does the rest. ## What's Next From 3944704b5d2e857fc3ffdcaf1333a73ee802a933 Mon Sep 17 00:00:00 2001 From: Raphael Date: Fri, 19 May 2023 10:00:08 -0700 Subject: [PATCH 02/23] add cloud intro (#581) --- content/dgraph-overview.md | 8 ++-- content/dgraphcloud/introduction.md | 64 ++++++++++++----------------- 2 files changed, 31 insertions(+), 41 deletions(-) diff --git a/content/dgraph-overview.md b/content/dgraph-overview.md index 47a92fe5..81949728 100644 --- a/content/dgraph-overview.md +++ b/content/dgraph-overview.md @@ -16,11 +16,13 @@ Dgraph is different from other graph databases in a number of ways, including: - **GraphQL Support**:   *GraphQL is built in to make data access simple and standards-compliant. Unlike most GraphQL solutions, no resolvers are needed - Dgraph resolves queries automatically through graph navigation.* -- **True Free Open Source Software (FOSS)**:   *Dgraph is free to use, and [available on github](https://github.com/dgraph-io/dgraph).* - - **Fully Transactional and ACID Compliant**:   *Dgraph satisfies demanding OLTP workloads that require frequent inserts and updates.* -- **Text Search**:   *Full-text searching is included.* +- **Language support & Text Search**:   *Full-text searching is included and strings can be expressed in multiple languages* + +- **Geo data and geo queries**:   *Dgraph supports points and shapes data and queries can use near, within, contains, or intersects geo functions* + +- **True Free Open Source Software (FOSS)**:   *Dgraph is free to use, and [available on github](https://github.com/dgraph-io/dgraph).* ## Dgraph and GraphQL In Dgraph, GraphQL is not an afterthought or an add-on; it is core to the product. GraphQL developers can get started in minutes, and need not concern themselves with the powerful graph database running in the background. diff --git a/content/dgraphcloud/introduction.md b/content/dgraphcloud/introduction.md index 89fdb848..dcf0002b 100644 --- a/content/dgraphcloud/introduction.md +++ b/content/dgraphcloud/introduction.md @@ -5,36 +5,40 @@ weight = 1 parent = "cloud" +++ -Dgraph Cloud, powered by Dgraph database, is a fully-managed GraphQL database -service that lets you focus on building apps, not managing infrastructure. Dgraph -Cloud is built from the ground up to support GraphQL, and uses a graph database -structure down to its lowest layers. So, it integrates seamlessly with your -existing ecosystem of GraphQL tools. +## Dgraph -Dgraph Cloud gives you the power of Dgraph database in a hosted environment, -providing the flexibility and performance of a horizontally-scalable and -distributed GraphQL database with a graph backend. With Dgraph Cloud, you -don’t need to configure and manage VMs, servers, firewalls, and HTTP endpoints -to power your modern apps and websites. +Designed from day one to be distributed for scale and speed, **Dgraph** is the native Graph database with native GraphQL support. It is open-source, scalable, distributed, highly-available, and lightning fast. -Dgraph Cloud is available in several pricing tiers: +Dgraph is different from other graph databases in a number of ways, including: -* **Free**: This tier is suitable for hobbyist use and evaluation, but has significant bandwidth limitations. -* **Shared Instances** (formerly *Slash GraphQL*): Backends in this tier run on a shared server cluster to provide a graph database service at a low price. -* **Dedicated Instances**: Backends in this tier run on dedicated server clusters to meet the heavy workloads and other needs of enterprise customers. This tier also provides high availability and the option to run Dgraph in your own virtual private cloud (VPC) or bring your own Kubernetes (BYOK) environment. +- **Distributed Scale**:   *Built from day 1 to be distributed, to handle larger data sets.* -To learn more about pricing for each tier, see the [Dgraph Pricing Page](https://dgraph.io/pricing). -To learn more about Dgraph database, see [Dgraph Database Overview]{{< relref "dgraph-overview.md">}}). +- **GraphQL Support**:   *GraphQL is built in to make data access simple and standards-compliant. Unlike most GraphQL solutions, no resolvers are needed - Dgraph resolves queries automatically through graph navigation.* + +- **Fully Transactional and ACID Compliant**:   *Dgraph satisfies demanding OLTP workloads that require frequent inserts and updates.* + +- **Language support & Text Search**:   *Full-text searching is included and strings can be expressed in multiple languages* + +- **Gelocation data and geo querie**:   *Dgraph can store Points and Shapes and queries can use near, within, contains, intersects geo functions* + +More details at [Dgraph Database Overview]({{< relref "dgraph-overview">}}) + +## Dgraph Cloud + +Dgraph Cloud gives you the power of Dgraph database including performance, high-availability, horizontally-scalability and the support of GraphQL for rapid application development in a hosted and fully-managed environment. Dgraph Cloud lets you focus on building apps, not managing infrastructure. + +### Dgraph Cloud Cluster Types + + - **Shared Instance**: Dgraph Cloud with [shared instances](https://cloud.dgraph.io/pricing?type=shared) is a fast and easy way to get started with GraphQL, and does not require any graph database knowledge to start and run. Shared instances run in a common database using Dgraph multi-tenancy. Your data is protected but you share resources and will have limited scale. + +- **Dedicated instances** run on their own dedicated hardware to ensure consistent performance. This option extends the capabilities of the lower-cost shared instances to support enterprise, production workloads, and includes a high availability option. -{{% notice "tip" %}} -You can check which Dgraph database release version is running on your Dgraph Cloud backend in [Dgraph Cloud Settings](https://cloud.dgraph.io/_/settings). -{{% /notice %}} ## Key features | Feature | Notes | | :------------- | :------------- | -| Production-ready | Dgraph Cloud is great for small app developers but is also built to meet the needs of your business as it grows with built-in authorization, encryption at rest, TLS, incremental backups, and more. | +| Production-ready | Dgraph Cloud is built to meet the needs of your business as it grows with built-in authorization, encryption at rest, TLS, incremental backups, and more. | | Scale and expand your app without rebuilding your backend | Dgraph Cloud stores and distributes data to optimize the execution of GraphQL traversals, joins, and retrievals. Dgraph natively parses and executes GraphQL, achieving great performance and the ability to scale horizontally up to terabytes of data. | | A high-performance, graph-first database | Dgraph Cloud runs Dgraph database, which is built to support the needs of modern apps with lightning-fast queries at any depth. | | Custom logic | Use JavaScript to add programmatic custom logic on the backend, adding power to your apps without sacrificing client-side performance. | @@ -43,29 +47,13 @@ You can check which Dgraph database release version is running on your Dgraph Cl | GraphQL-based authorization and management | GraphQL is used throughout Dgraph Cloud, so you don’t need to use another tool or learn another syntax to handle user authorization or database administration tasks such as schema changes and data exports. | | Work with the Open Source ecosystem | Because Dgraph is open-source, your app relies on a codebase that you can contribute to, not an opaque “black box”. | -## Architecture - -Dgraph Cloud runs Dgraph database in a fully-managed service, so you don't need -to manage servers or VM. Dgraph Cloud provides a truly distributed database that shards and -replicates data across cluster nodes, providing a scalable and production-ready -database solution for your app. -Behind the scenes, Dgraph Cloud instances run as as clusters of Dgraph Zero and -Dgraph Alpha server nodes. - -* Dgraph Zero nodes control the cluster. This includes storing information - about the cluster and moving data between Dgraph Alpha nodes to re-balance - based on transactional workloads. -* Dgraph Alpha nodes serve data to clients, and also provide administrator - endpoints. - -To learn more about the Dgraph clusters that power Dgraph Cloud, see -[Understanding Dgraph Cluster]({{< relref "cluster-setup.md#understanding-dgraph-cluster" >}}). ## Next steps + To learn more about how Dgraph Cloud makes it easier to develop apps, create a -trial account and [try Dgraph Cloud](https://cloud.dgraph.io) for yourself. +trial account at [Dgraph Cloud](https://cloud.dgraph.io) and try the [Introduction to GraphQL](https://dgraph.io/tour/graphqlintro/2/) tutorial to define a GraphQL schema, insert and query data in just a few minutes. ## Recommended Reading From 698b6080faac3a96470403289be80687e40e92bb Mon Sep 17 00:00:00 2001 From: Raphael Date: Tue, 23 May 2023 15:57:01 -0700 Subject: [PATCH 03/23] correct todo overview URLs (#587) (#588) --- .../developer/todo-app-tutorial/todo-overview.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/learn/developer/todo-app-tutorial/todo-overview.md b/content/learn/developer/todo-app-tutorial/todo-overview.md index 66af5b0a..7b53cd7c 100644 --- a/content/learn/developer/todo-app-tutorial/todo-overview.md +++ b/content/learn/developer/todo-app-tutorial/todo-overview.md @@ -14,11 +14,11 @@ This is a simple tutorial that will take you through making a basic to-do list a ### Steps -- [Schema Design](/graphql/todo-app-tutorial/todo-schema-design) -- [Basic UI](/graphql/todo-app-tutorial/todo-ui) -- [Add Auth Rules](/graphql/todo-app-tutorial/todo-auth-rules) -- [Use Auth0's JWT](/graphql/todo-app-tutorial/todo-auth0-jwt) -- [Use Firebase's JWT](/graphql/todo-app-tutorial/todo-firebase-jwt) -- [Deploy on Dgraph Cloud](/graphql/todo-app-tutorial/deploy) +- [Schema Design]({{< relref "todo-schema-design">}}) +- [Basic UI]({{< relref "todo-ui">}}) +- [Add Auth Rules]({{< relref "todo-auth-rules">}}) +- [Use Auth0's JWT]({{< relref "todo-auth0-jwt">}}) +- [Use Firebase's JWT]({{< relref "todo-firebase-jwt">}}) +- [Deploy on Dgraph Cloud]({{< relref "todo-deploy">}}) --- From 90896559eb690dcadc9e55aa9a5fe557d32cb39c Mon Sep 17 00:00:00 2001 From: Dilip <111321288+dshekhar95@users.noreply.github.com> Date: Tue, 6 Jun 2023 17:40:49 -0400 Subject: [PATCH 04/23] Update mutation.md (#601) Changed ID: to ID! --- content/graphql/custom/mutation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/graphql/custom/mutation.md b/content/graphql/custom/mutation.md index 8b731895..b0e13722 100644 --- a/content/graphql/custom/mutation.md +++ b/content/graphql/custom/mutation.md @@ -15,7 +15,7 @@ Let's say we have an application about authors and posts. Logged in authors can type Author { ... } type Post { - id: ID: + id: ID! title: String text: String datePublished: DateTime From 793bf7a1e473b6129e95cff5a6c41b11437c0ed8 Mon Sep 17 00:00:00 2001 From: Raphael Date: Fri, 16 Jun 2023 14:53:53 -0700 Subject: [PATCH 05/23] move lambda server in Installation section (#607) --- .../server.md => deploy/installation/lambda-server.md} | 8 +++++--- content/deploy/installation/production-checklist.md | 2 +- content/deploy/installation/single-host-setup.md | 2 +- 3 files changed, 7 insertions(+), 5 deletions(-) rename content/{graphql/lambda/server.md => deploy/installation/lambda-server.md} (96%) diff --git a/content/graphql/lambda/server.md b/content/deploy/installation/lambda-server.md similarity index 96% rename from content/graphql/lambda/server.md rename to content/deploy/installation/lambda-server.md index c4f19594..7aef22eb 100644 --- a/content/graphql/lambda/server.md +++ b/content/deploy/installation/lambda-server.md @@ -1,9 +1,11 @@ +++ title = "Lambda Server" -description = "Setup a Dgraph database with a lambda server. Dgraph Lambda is a serverless platform for running JavaScript on Dgraph and Dgraph Cloud." -weight = 6 +description = "Setup a Dgraph database with a lambda server. Dgraph Lambda is a serverless platform for running JavaScript on Dgraph and Dgraph Cloud" +aliases =['/graphql/lambda/server'] +weight = 3 [menu.main] - parent = "lambda-resolver" + + parent = "installation" +++ In this article you'll learn how to setup a Dgraph database with a lambda server. diff --git a/content/deploy/installation/production-checklist.md b/content/deploy/installation/production-checklist.md index 3f9f741f..16587940 100644 --- a/content/deploy/installation/production-checklist.md +++ b/content/deploy/installation/production-checklist.md @@ -2,7 +2,7 @@ date = "2017-03-20T22:25:17+11:00" title = "Production checklist" description = "Requirements to install Dgraph in a production environment" -weight = 2 +weight = 4 [menu.main] parent = "installation" +++ diff --git a/content/deploy/installation/single-host-setup.md b/content/deploy/installation/single-host-setup.md index 1e3e6120..5e2371a4 100644 --- a/content/deploy/installation/single-host-setup.md +++ b/content/deploy/installation/single-host-setup.md @@ -1,7 +1,7 @@ +++ date = "2017-03-20T22:25:17+11:00" title = "Learning Environment" -weight = 3 +weight = 2 [menu.main] parent = "installation" +++ From 7cba41c8c6e7726395e0a3527a8a8d6df71958cb Mon Sep 17 00:00:00 2001 From: Douglas Anderson Date: Wed, 28 Jun 2023 11:58:12 -0600 Subject: [PATCH 06/23] Docs(other): Fix mutations=strict typo in dgraph-administration.md (#612) Fix typo `mutation=strict` -> `mutations=strict` --- content/deploy/admin/dgraph-administration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/deploy/admin/dgraph-administration.md b/content/deploy/admin/dgraph-administration.md index da421a22..0e52199d 100644 --- a/content/deploy/admin/dgraph-administration.md +++ b/content/deploy/admin/dgraph-administration.md @@ -102,7 +102,7 @@ Before performing a mutation on a predicate that doesn't exist in the schema, you need to perform an alter operation with that predicate and its schema type. ```sh -dgraph alpha --limit "mutation=strict; mutations-nquad=1000000" +dgraph alpha --limit "mutations=strict; mutations-nquad=1000000" ``` ## Secure Alter Operations From 71e056d649b022872d022f0dd1bd7e91d1381da3 Mon Sep 17 00:00:00 2001 From: Raphael Date: Thu, 29 Jun 2023 09:26:28 -0700 Subject: [PATCH 07/23] add new skin (#616) --- content/_index.md | 193 ++++++++++++------ content/deploy/cli-command-reference.md | 4 +- content/graphql/schema/graph-links.md | 2 +- .../react/react-ui/react-app-boiler-plate.md | 1 + 4 files changed, 133 insertions(+), 67 deletions(-) diff --git a/content/_index.md b/content/_index.md index 64343acd..9c75db6a 100644 --- a/content/_index.md +++ b/content/_index.md @@ -9,81 +9,140 @@ aliases = ["/contribute"] weight = 1 +++ -
-
-

Get started with Dgraph

-

- Designed from the ground up to be run in production, Dgraph is the native GraphQL database with a graph backend. It is open-source, scalable, distributed, highly available and lightning fast. -

-

Tip: New to Dgraph? Take the Dgraph Tour to run live queries in your browser. Then, try Dgraph as a cloud service, or download Dgraph to deploy it yourself.

- -
-
-
- }}"> -

Overview

-

- Understand Dgraph core concepts and hosting options. -

- -
-
- GraphQL icon - }}"> -

GraphQL API

+
+
+
+

Docs

+

Get started with Dgraph

- Generate a GraphQL API and a graph backend from a simple GraphQL schema. + Designed from the ground up to be run in production, Dgraph is the native GraphQL database with a graph backend. It is open-source, scalable, distributed, highly available and lightning fast.

- -
-
-
- }}"> -

Dgraph Query Language (DQL)

-

- Learn Dgraph Query Language (DQL), Dgraph’s proprietary language to add, modify, delete and fetch data. -

- -
-
-
- }}"> -

Dgraph Cloud

-

- Run Dgraph in the Cloud. - Work with your data in a fully-managed cloud service. -

- -
+

Tip: New to Dgraph? Take the Dgraph Tour to run live queries in your browser. Then, try Dgraph as a cloud service, or download Dgraph to deploy it yourself.

+
+
+ }}"> + + + +

Overview

+

+ Understand Dgraph core concepts and hosting options. +

+ +
+
+ }}"> + + + + + + + + + + + + + + + + +

GraphQL API

+

+ Generate a GraphQL API and a graph backend from a simple GraphQL schema. +

+ +
+
+ }}"> + + + + + + +

Dgraph Query Language (DQL)

+

+ Learn Dgraph Query Language (DQL), Dgraph’s proprietary language to add, modify, delete and fetch data. +

+ +
+
+ }}"> + + + +

Dgraph Cloud

+

+ Run Dgraph in the Cloud. + Work with your data in a fully-managed cloud service. +

+ +
+
+ }}"> + + + +

Self-hosted cluster

+

+ Install and manage a Dgraph cluster in a variety of self-managed deployment scenarios. +

+ +
+
+ }}"> + + + + +

gRPC and HTTP Clients

+

+ Interact with a Dgraph cluster using libraries in various programming languages. +

+ +
-
-
- }}"> -

Self-hosted cluster

-

- Install and manage a Dgraph cluster in a variety of self-managed deployment scenarios. -

- -
-
-
- }}"> -

gRPC and HTTP Clients

-

- Interact with a Dgraph cluster using libraries in various programming languages. -

-
+ -
- +
+
+

Join the Community

+

Our mission is to help developers and organizations leverage the power of graph and GraphQL to build applications better, faster, and more easily.

+
+
+
+ + + +
+
Contribute to Dgraph
+

View tracker and issues

+
+ Github +
+
+ + + +
+
Contribute to Dgraph
+

View tracker and issues

+
+ Discuss +
+
+
+
diff --git a/content/deploy/cli-command-reference.md b/content/deploy/cli-command-reference.md index 3b20cd00..c44d1f8b 100644 --- a/content/deploy/cli-command-reference.md +++ b/content/deploy/cli-command-reference.md @@ -336,9 +336,9 @@ Flags: compression=snappy; [none, zstd:level, snappy] Specifies the compression algorithm and compression level (if applicable) for the postings directory."none" would disable compression, while "zstd:1" would set zstd compression at level 1. - goroutines=8; The number of goroutines to use in badger.Stream. + numgoroutines=8; The number of goroutines to use in badger.Stream. max-retries=-1; Commits to disk will give up after these number of retries to prevent locking the worker in a failed state. Use -1 to retry infinitely. - (default "compression=snappy; goroutines=8; max-retries=-1;") + (default "compression=snappy; numgoroutines=8; max-retries=-1;") --cache string Cache options percentage=0,65,35; Cache percentages summing up to 100 for various caches (FORMAT: PostingListCache,PstoreBlockCache,PstoreIndexCache) size-mb=1024; Total size of cache (in MB) to be used in Dgraph. diff --git a/content/graphql/schema/graph-links.md b/content/graphql/schema/graph-links.md index e6c8c823..28efb982 100644 --- a/content/graphql/schema/graph-links.md +++ b/content/graphql/schema/graph-links.md @@ -62,7 +62,7 @@ mutation { } ``` -will automatically add it to Diggy's list of `posts`. Deleting the post will remove it from Diggy's `posts`. Similarly, using an update mutation on an author to insert a new post will automatically add Diggy as the author the author +will automatically add it to Diggy's list of `posts`. Deleting the post will remove it from Diggy's `posts`. Similarly, using an update mutation on an author to insert a new post will automatically add Diggy as the author ```graphql mutation { diff --git a/content/learn/developer/react/react-ui/react-app-boiler-plate.md b/content/learn/developer/react/react-ui/react-app-boiler-plate.md index dc2409bb..b378e378 100644 --- a/content/learn/developer/react/react-ui/react-app-boiler-plate.md +++ b/content/learn/developer/react/react-ui/react-app-boiler-plate.md @@ -59,6 +59,7 @@ After you have the boilerplate code on your machine, you can start the app using the following `yarn` command: ```sh +yarn install yarn start ``` From 396ee096f65be0c51e70a77bb6edebdbc41a09d9 Mon Sep 17 00:00:00 2001 From: Jonathan Zollinger <62955101+Jonathan-Zollinger@users.noreply.github.com> Date: Mon, 4 Sep 2023 21:08:43 -0600 Subject: [PATCH 08/23] Docs(import data): fix small typo (#641) ## Description This is just a simple typo I found. --- content/howto/importdata/live-loader.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/howto/importdata/live-loader.md b/content/howto/importdata/live-loader.md index 0790e7ca..d1b17e27 100644 --- a/content/howto/importdata/live-loader.md +++ b/content/howto/importdata/live-loader.md @@ -24,7 +24,7 @@ Those files have been generated by an [export]({{< relref "about-export.md" >}}) 1. Obtain dgraph binary or the latest docker image by following the [installation]({{< relref "download.md" >}}) instructions. This is required to run Dgraph CLI command `dgraph live`. -1. Obtain the `GRPC endpoint` of you Dgraph Cloud backend and a valid `Client API key`. +1. Obtain the `GRPC endpoint` of your Dgraph Cloud backend and a valid `Client API key`. An administrator gets those information with the following steps: 1. Log into the Dgraph Cloud account, select the backend From 0b1d592ddc1edd48d7c73c80a2e66c5bdbcc2f6b Mon Sep 17 00:00:00 2001 From: Damon Feldman <700633+damonfeldman@users.noreply.github.com> Date: Tue, 5 Sep 2023 00:30:09 -0400 Subject: [PATCH 09/23] Update log-format.md (#642) It appears the logRequest flag changed to logDQLRequest on or about v23.0 or v23.1. Updating docs. --- content/deploy/admin/log-format.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/deploy/admin/log-format.md b/content/deploy/admin/log-format.md index 04daab7d..12975d5c 100644 --- a/content/deploy/admin/log-format.md +++ b/content/deploy/admin/log-format.md @@ -45,7 +45,7 @@ You can dynamically turn request logging on or off. To toggle request logging on ```graphql mutation { - config(input: {logRequest: true}) { + config(input: {logDQLRequest: true}) { response { code message @@ -53,6 +53,8 @@ mutation { } } ``` +Note this input flag was named logRequest until Dgraph version v23. + The response should look like the following: ```json @@ -114,4 +116,4 @@ mutation { With a Dgraph enterprise license, you can enable audit logging so that all requests are tracked and available for use in security audits. To learn more, see -[Audit Logging]({{< relref "enterprise-features/audit-logs.md" >}}). \ No newline at end of file +[Audit Logging]({{< relref "enterprise-features/audit-logs.md" >}}). From 330131f857077b53b6d1242ab48a96f89c9c0853 Mon Sep 17 00:00:00 2001 From: Damon Feldman <700633+damonfeldman@users.noreply.github.com> Date: Wed, 6 Sep 2023 13:47:57 -0400 Subject: [PATCH 10/23] Update cli-command-reference.md with max pending added max-pending-queries line to --limit superflag --- content/deploy/cli-command-reference.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/deploy/cli-command-reference.md b/content/deploy/cli-command-reference.md index c44d1f8b..80e96612 100644 --- a/content/deploy/cli-command-reference.md +++ b/content/deploy/cli-command-reference.md @@ -128,6 +128,7 @@ The `--badger` superflag allows you to set many advanced [Badger options](https: | `--query_edge_limit` | uint64 | `query-edge` | uint64 |`alpha`| Maximum number of edges that can be returned in a query | | `--normalize_node_limit` | int | `normalize-node` | int |`alpha`| Maximum number of nodes that can be returned in a query that uses the normalize directive | | `--mutations_nquad_limit` | int | `mutations-nquad` | int |`alpha`| Maximum number of nquads that can be inserted in a mutation request | +| `--max-pending-queries` | int | `max-pending-queries` | int |`alpha`| Maximum number of concurrently processing requests allowed before requests are rejected with 429 Too Many Requests | ### Raft superflag From 112aa05cf3dd818c1cd518d0b8196dace8c73864 Mon Sep 17 00:00:00 2001 From: Enrico Minack Date: Tue, 5 Sep 2023 06:46:47 +0200 Subject: [PATCH 11/23] Add note about complexity of offset and after in dql (#639) Some observations can be found at https://discuss.dgraph.io/t/pagination-with-offset-should-scale-like-after/8577. --- content/query-language/pagination.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/content/query-language/pagination.md b/content/query-language/pagination.md index 7dc43901..c05b97fe 100644 --- a/content/query-language/pagination.md +++ b/content/query-language/pagination.md @@ -74,6 +74,8 @@ Syntax Examples: With `offset: N` the first `N` results are not returned. Used in combination with first, `first: M, offset: N` skips over `N` results and returns the following `M`. +{{% notice "note" %}}Skipping over `N` results takes time proportional to `N` (complexity `O(N)`). In other words, the larger `N`, the longer it takes to compute the result set. Prefer [after]({{< relref "#after">}}) over `offset`.{{% /notice %}} + Query Example: Order Hark Tsui's films by English title, skip over the first 4 and return the following 6. {{< runnable >}} @@ -102,6 +104,7 @@ Syntax Examples: Another way to get results after skipping over some results is to use the default UID ordering and skip directly past a node specified by UID. For example, a first query could be of the form `predicate (after: 0x0, first: N)`, or just `predicate (first: N)`, with subsequent queries of the form `predicate(after: , first: N)`. +{{% notice "note" %}}Skipping over results with `after` takes constant time (complexity `O(1)`). In other words, no matter how many results are skipped, no extra time adds to computing the result set. This should be preferred over [offset]({{< relref "#offset">}}).{{% /notice %}} Query Example: The first five of Baz Luhrmann's films, sorted by UID order. From 550903dc189b9d491681e8ce0a7cd81542985955 Mon Sep 17 00:00:00 2001 From: Raphael Date: Thu, 21 Sep 2023 18:02:00 -0700 Subject: [PATCH 12/23] update graphql subscription doc (#645) add example using URQL and subscriptions-transport-ws --- .../directives/directive-withsubscription.md | 27 +++ content/graphql/subscriptions/index.md | 188 +++++++++++------- 2 files changed, 148 insertions(+), 67 deletions(-) create mode 100644 content/graphql/schema/directives/directive-withsubscription.md diff --git a/content/graphql/schema/directives/directive-withsubscription.md b/content/graphql/schema/directives/directive-withsubscription.md new file mode 100644 index 00000000..f8f959dd --- /dev/null +++ b/content/graphql/schema/directives/directive-withsubscription.md @@ -0,0 +1,27 @@ ++++ +title = "@withSubscription" +weight = 7 +[menu.main] + parent = "directives" + identifier = "directive-withsubscription" ++++ + + +The `@withSubscription` directive enables **subscription** operation on a GraphQL type. + +A subscription notifies your client with changes to back-end data using the WebSocket protocol. +Subscriptions are useful to get low-latency, real-time updates. + +To enable subscriptions on any type add the `@withSubscription` directive to the schema as part of the type definition, as in the following example: + +```graphql +type Todo @withSubscription { + id: ID! + title: String! + description: String! + completed: Boolean! +} +``` + +Refer to [GraphQL Subscriptions]({{< relref "graphql/subscriptions" >}}) to learn how to use subscriptions in you client application. + diff --git a/content/graphql/subscriptions/index.md b/content/graphql/subscriptions/index.md index d24106b0..57cdf771 100644 --- a/content/graphql/subscriptions/index.md +++ b/content/graphql/subscriptions/index.md @@ -15,7 +15,7 @@ The client can unsubscribe by sending a message to the server. The server can al ![Subscription](/images/graphql/subscription_flow.png "Subscription in GraphQL") -## How to enable subscriptions in GraphQL +## Enable subscriptions in GraphQL In GraphQL, it's straightforward to enable subscriptions on any type. You can add the `@withSubscription` directive to the schema as part of the type definition, as in the following example: @@ -28,52 +28,12 @@ type Todo @withSubscription { } ``` -### Example +## @withSubscription with @auth -After updating the schema with the `@withSubscription` directive, you can execute a subscription query and receive updates when the subscription query result is updated, as follows: +You can use [@auth]({{< relref "graphql/schema/directives/auth">}}) access control rules in conjunction with `@withSubscription`. -![Subscription](/images/graphql/subscription_example.gif "Subscription Example") -## Apollo client setup - -To learn about using subscriptions with Apollo client, see a blog post on [GraphQL Subscriptions with Apollo client](https://dgraph.io/blog/post/how-does-graphql-subscription/). - -## Subscriptions to custom DQL - -You can use the `@withSubscription` directive on GraphQL types to generate subscription queries for that type. -You can also apply this directive to custom DQL queries by specifying `@withSubscription` on individual DQL queries in `type Query`, -and those queries will be added to `type subscription`. - -{{% notice "note" %}} -Currently, Dgraph only supports subscriptions on custom DQL queries. So, you -can't subscribe to custom HTTP queries. -{{% /notice %}} - -For example, see the custom DQL query `queryUserTweetCounts` below: - -```graphql -type Query { - queryUserTweetCounts: [UserTweetCount] @withSubscription @custom(dql: """ - query { - queryUserTweetCounts(func: type(User)) { - screen_name: User.screen_name - tweetCount: count(User.tweets) - } - } - """) -} -``` - -Because the `queryUserTweetCounts` query has a `@withSubscription` directive, it -will be added to the `subscription` type, allowing users to subscribe to this query. - -## Authorization with subscriptions - -Authorization adds more power to GraphQL subscriptions. You can use all of the authorization features that are available when running queries. -Additionally, you can specify when the subscription automatically terminates (the "timeout" of the subscription) in the JWT. - -### Schema -Consider following Schema that has both the `@withSubscription` and `@auth` directives defined on type `Todo`. The authorization rule enforces that only to-do tasks owned by `$USER` (defined in the JWT) are visible. +Consider following Schema that has both the `@withSubscription` and `@auth` directives defined on type `Todo`. ```graphql type Todo @withSubscription @auth( @@ -89,48 +49,142 @@ type Todo @withSubscription @auth( text: String! @search(by: [term]) owner: String! @search(by: [hash]) } -# Dgraph.Authorization {"VerificationKey":"secret","Header":"Authorization","Namespace":"https://dgraph.io","Algo":"HS256"} +# Dgraph.Authorization {"Header":"X-Dgraph-AuthToken","Namespace":"https://dgraph.io/jwt/claims","jwkurl":"https://xyz.clerk.accounts.dev/.well-known/jwks.json","audience":["dgraph"],"ClosedByDefault":true} +``` +The generated GraphQL API expects a JWT token in the `X-Dgraph-AuthToken` header and uses the `USER` claim to apply a rule based access control (RBAC): the authorization rule enforces that only to-do tasks owned by `$USER` are returned. + + +## WebSocket client +Dgraph uses the websocket subprotocol `subscription-transport-ws`. + +Clients must be instantiated using the WebSocket URL of the GraphQL API which is your [Dgraph GraphQL endpoint]({{< relref "graphql/graphql-clients/endpoint/_index.md">}}) with ``https`` replaced by ``wss``. + +If your Dgraph endpoint is ``https://blue-surf-0033.us-east-1.aws.cloud.dgraph.io/graphql`` +the WebSocket URL is ``wss://blue-surf-0033.us-east-1.aws.cloud.dgraph.io/graphql`` + +If your GraphQL API is configured to expect a JWT token in a header, you must configure the WebSocket client to pass the token. Additionally, the subscription terminates when the JWT expires. + + +Here are some examples of frontend clients setup. + +### URQL client setup in a React application + +In this scenario, we are using [urql client](https://formidable.com/open-source/urql/) and `subscriptions-transport-ws` modules. + +In order to use a GraphQL subscription query in a component, you need to +- instantiate a subscriptionClient +- instantiate a URQL client with a 'subscriptionExchange' using the subscriptionClient + +```js +import { Client, Provider, cacheExchange, fetchExchange, subscriptionExchange } from 'urql'; +import { SubscriptionClient } from 'subscriptions-transport-ws'; + + const subscriptionClient = new SubscriptionClient( + process.env.REACT_APP_DGRAPH_WSS, + { reconnect: true, + connectionParams: {"X-Dgraph-AuthToken" : props.token} + } + ); + + const client = new Client({ + url: process.env.REACT_APP_DGRAPH_ENDPOINT, + fetchOptions: { headers: { "X-Dgraph-AuthToken": `Bearer ${props.token}` } }, + exchanges: [ + cacheExchange, + fetchExchange, + subscriptionExchange({ + forwardSubscription: request => subscriptionClient.request(request), + }) + ]}) + ``` + +In this example, + +- **process.env.REACT_APP_DGRAPH_ENDPOINT** is your [Dgraph GraphQL endpoint]({{< relref "graphql/graphql-clients/endpoint/_index.md">}}) +- **process.env.REACT_APP_DGRAPH_WSS** is the WebSocket URL +- **props.token** is the JWT token of the logged-in user. + +Note that we are passing the JWT token in the GraphQL client using 'fetchOptions' and in the WebSocket client using 'connectionParams'. + +Assuming we are using graphql-codegen, we can define a subcription query: +```js +import { graphql } from "../gql"; + +export const TodoFragment = graphql(` + fragment TodoItem on Todo { + id + text + } +`) + + +export const TodoSubscription = graphql(` + subscription myTodo { + queryTodo(first:100) { + ...TodoItem + } + } +`) ``` +and use it in a React component +```js +import { useQuery, useSubscription } from "urql"; +... +const [messages] = useSubscription({ query: MyMessagesDocument}); -### JWT +``` +That's it, the react component is able to use ``messages.data.queryTodo`` to display the updated list of Todos. -The subscription requires a JWT that declares the `$USER`, expiry, and other variables. -The JWT is passed from the GraphQL client as key-value pair, where the key is the `Header` given in the schema and the value is the JWT. -In the example below, the key is `Authorization` and the value is the JWT. -Most GraphQL clients have a separate header section to pass Header-JWT key-value pairs. In the Apollo client, these are passed -using `connectionParams`, as follows. +### Apollo client setup + +To learn about using subscriptions with Apollo client, see a blog post on [GraphQL Subscriptions with Apollo client](https://dgraph.io/blog/post/how-does-graphql-subscription/). + +To pass the user JWT token in the Apollo client,use `connectionParams`, as follows. ```javascript const wsLink = new WebSocketLink({ uri: `wss://${ENDPOINT}`, options: { reconnect: true, - connectionParams: { "Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTAxMjg2MjIsImh0dHBzOi8vZGdyYXBoLmlvIjp7IlJPTEUiOiJVU0VSIiwiVVNFUiI6IkFsaWNlIn0sImlzcyI6InRlc3QifQ.6AODlumsk9kbnwZHwy08l40PeqEmBHqK4E_ozNjQpuI", },}); + connectionParams: { "
": "", },}); ``` -{{% notice "note" %}} -Starting in release v21.03, Dgraph supports compression for subscriptions. -Dgraph uses `permessage-deflate` compression if the GraphQL client's -`Sec-Websocket-Extensions` request header includes `permessage-deflate`, as follows: -`Sec-WebSocket-Extensions: permessage-deflate`. -{{% /notice %}} +Use the header expected by the Dgraph.Authorization configuration of your GraphQL schema. -### Example +## Subscriptions to custom DQL -The following example shows the operation of subscriptions with authentication rules for the schema given above. +You can also apply `@withSubscription` directive to custom DQL queries by specifying `@withSubscription` on individual DQL queries in `type Query`, +and those queries will be added to `type subscription`. -First, you can generate the JWT as shown in the following image with expiry and `$USER` (the owner of a to-do task). -You can generate the JWT from [jwt.io](https://jwt.io/). The client should send the JWT to the server along with the request, as discussed above. +For example, see the custom DQL query `queryUserTweetCounts` below: -![Subscription-Generating-JWT](/images/graphql/Generating-JWT.png "Subscription with Auth Example") +```graphql +type Query { + queryUserTweetCounts: [UserTweetCount] @withSubscription @custom(dql: """ + query { + queryUserTweetCounts(func: type(User)) { + screen_name: User.screen_name + tweetCount: count(User.tweets) + } + } + """) +} +``` -Next, Dgraph runs the subscription and send updates. You can see that only the to-do tasks that were added with the owner name "Alice" are visible in the subscription. +`queryUserTweetCounts` is added to the `subscription` type, allowing users to subscribe to this query. -![Subscription+Auth-Action](/images/graphql/Auth-Action.gif "Subscription with Auth Example") +{{% notice "note" %}} +Currently, Dgraph only supports subscriptions on custom **DQL queries**. You +can't subscribe to custom **HTTP queries**. +{{% /notice %}} -Eventually, the JWT expires and the subscription terminates, as shown below. -![Subscription+Timeout](/images/graphql/Subscription-Timeout.gif "Subscription with Auth Example") +{{% notice "note" %}} +Starting in release v21.03, Dgraph supports compression for subscriptions. +Dgraph uses `permessage-deflate` compression if the GraphQL client's +`Sec-Websocket-Extensions` request header includes `permessage-deflate`, as follows: +`Sec-WebSocket-Extensions: permessage-deflate`. +{{% /notice %}} From 0fdb38e4f828c010647f07609306d689ff7c24c7 Mon Sep 17 00:00:00 2001 From: Raphael Date: Thu, 21 Sep 2023 19:17:28 -0700 Subject: [PATCH 13/23] add Clerk.com config example (#646) Add an example of Dgraph.Authorization configuration with Clerk.com auth provider. --- content/graphql/security/jwt.md | 57 ++++++++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 5 deletions(-) diff --git a/content/graphql/security/jwt.md b/content/graphql/security/jwt.md index 7a490d05..75670cd4 100644 --- a/content/graphql/security/jwt.md +++ b/content/graphql/security/jwt.md @@ -11,7 +11,7 @@ When deploying a GraphQL schema, the admin user can set a ``# Dgraph.Authorizat This line must start with the exact string ``# Dgraph.Authorization`` and be at the bottom of the schema file. -### Configure JWT token handling +## Configure JWT token handling To configure how Dgraph should handle JWT token for ``/graphql`` endpoint : 1. Add a line starting with ``# Dgraph.Authorization`` and with the following parameters at the very end of your GraphQL schema. @@ -51,8 +51,51 @@ See the [RBAC rules]({{< relref "RBAC-rules.md">}}) and [Graph traversal rules]( ### Require JWT token To not only accept but to require the JWT token regardless of @auth directives in your GraphQL schema, set option "ClosedByDefault" to true in the `# Dgraph.Authorization` line. -### Examples +## Working with Authentication providers +Dgraph.Authorization is fully configurable to work with various authentication providers. +Authentication providers have options to configure how to generate JWT tokens. +Here are some configuration examples. + +### Clerk.com + +In your clerk dashboard, Access `JWT Templates` and create a template for Dgraph. + +Your template must have an `aud` (audience), this is mandatory for Dgraph when the token is verified using JWKURL. + +Decide on a claim namespace and add the information you want to use in your RBAC rules. + +We are using 'https://dgraph.io/jwt/claims' namespace in this example and have decided to get the user current organization, role ( clerk has currently two roles 'admin' and 'basic_member') and email. + +This is our JWT Template in Clerk: +```json +{ + "aud": "dgraph", + "https://dgraph.io/jwt/claims": { + "org": "{{org.name}}", + "role": "{{org.role}}", + "userid": "{{user.primary_email_address}}" + } +} +``` + +In the same configuration panel +- set the **token lifetime** +- copy the **JWKS Endpoint** + +Configure your Dgraph GraphQL schema with the following authorization +``` +# Dgraph.Authorization {"header":"X-Dgraph-AuthToken","namespace":"https://dgraph.io/jwt/claims","jwkurl":"https://<>.clerk.accounts.dev/.well-known/jwks.json","audience":["dgraph"],"closedbydefault":true} +``` +Note that +- **namespace** matches the namespace used in the JWT Template +- **audience** is an array and contains the **aud** used in the JWT token +- **jwkurl** is the **JWKS Endpoint** from Clerk + +You can select the header to receive the JWT token from your client app, `X-Dgraph-AuthToken` is a header authorized by default by Dgraph GraphQL API to pass CORS requirements. + + +## Other Dgraph.Authorization Examples To use a single JWK URL: @@ -82,7 +125,7 @@ Using HMAC-SHA256 token in `X-My-App-Auth` header and authorization claims in `h ### JWT format The value of the JWT ``header`` is expected to be in one of the following forms: -* Just the JWT token. +* Bare token. For example: ``` eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJodHRwczovL215LmFwcC5pby9qd3QvY2xhaW1zIjp7fX0.Pjlxpf-3FhH61EtHBRo2g1amQPRi0pNwoLUooGbxIho @@ -96,7 +139,7 @@ The value of the JWT ``header`` is expected to be in one of the following forms: ### Error handling -If ClosedByDefault is set to true, and the JWT is not present or if the JWT token does not include the proper audience information, or is not properly encoded, Dgraph replies to requests on `/graphql` endpoint with an error message rejecting the operation similar to: +If ClosedByDefault is set to true, and the JWT is not present or if the JWT token does not include the proper audience information, or is not properly encoded, or is expired, Dgraph replies to requests on `/graphql` endpoint with an error message rejecting the operation similar to: ``` { "errors": [ @@ -111,4 +154,8 @@ If ClosedByDefault is set to true, and the JWT is not present or if the JWT toke "queryContact": [] },... ``` - +**Error messages** +- "couldn't rewrite query queryContact because a valid JWT is required but was not provided" +- "couldn't rewrite query queryMessage because unable to parse jwt token:token is expired by 5h49m46.236018623s" +- "couldn't rewrite query queryMessage because JWT `aud` value doesn't match with the audience" +- "couldn't rewrite query queryMessage because unable to parse jwt token:token signature is invalid" From 874284be68ca5e9b035a7312ab67e70a1577aa4b Mon Sep 17 00:00:00 2001 From: Raphael Date: Fri, 22 Sep 2023 08:26:15 -0700 Subject: [PATCH 14/23] Update RBAC-rules.md (#648) Clarify that a claim can be an array of strings in RBAC rules. --- content/graphql/security/RBAC-rules.md | 48 ++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 7 deletions(-) diff --git a/content/graphql/security/RBAC-rules.md b/content/graphql/security/RBAC-rules.md index 3741186d..643d769a 100644 --- a/content/graphql/security/RBAC-rules.md +++ b/content/graphql/security/RBAC-rules.md @@ -26,16 +26,51 @@ type User @auth( delete: ... ) ``` +RBAC rule supports ``eq`` or ``in`` functions to test the value of a [JWT claim]({{< relref "graphql/security/_index.md#jwt-claims" >}}) from the JWT token payload. -Where `` is a [JWT claim]({{< relref "graphql/security/_index.md#jwt-claims" >}}) from the JWT token payload. - -You can use ``eq`` or ``in`` function to test the value of any claim. +The claim value may be a string or array of strings. +For example the following schema has a @auth directive specifying that a delete operation on a User object can only be done if the connected user has a 'ROLE' claim in the JWT token with the value "admin" : +```graphql +type User @auth( + delete: { rule: "{$ROLE: { eq: \"admin\" } }"} + ) { + username: String! + @id todos: [Todo] +} +``` +The following JWT token payload will pass the test (provided that Dgraph.Authorization is configured correctly with the right namespace) +```json +{ + "aud": "dgraph", + "exp": 1695359621, + "https://dgraph.io/jwt/claims": { + "ROLE": "admin", + "USERID": "testuser@dgraph.io" + }, + "iat": 1695359591, + ... +} +``` +The rule is also working with an array of roles in the JWT token: +```json +{ + "aud": "dgraph", + "exp": 1695359621, + "https://dgraph.io/jwt/claims": { + "ROLE": ["admin","user"] + "USERID": "testuser@dgraph.io" + }, + "iat": 1695359591, + ... +} +``` +In the case of an array used with the "in" function, the rule is valid is at least one of the claim value is "in" the provided list. -For example the following schema has a @auth directive specifying that a delete operation on a User object can only be done in the connected user has a 'ROLE' claim in the JWT token with the value "ADMIN" : +For example, with the following rule, the previous token will be valid because one of the ROLE is in the authorized roles. ```graphql type User @auth( - delete: { rule: "{$ROLE: { eq: \"ADMIN\" } }"} + delete: { rule: "{$ROLE: { in: [\"admin\",\"superadmin\"] } }"} ) { username: String! @id todos: [Todo] @@ -70,12 +105,11 @@ For example, given the following JWT payload "https://xyz.io/jwt/claims": [ "ROLE": "ADMIN" ], - "USERROLE": "user1", "email": "random@example.com" } ``` -The authorization rules can use ``$ROLE`` (if `https://xyz.io/jwt/claims` is declared as the namespace to use ) and also ``$USERROLE`` or ``$email``. +If `https://xyz.io/jwt/claims` is declared as the namespace to use, the authorization rules can use ``$ROLE`` but also ``$email``. In cases where the same claim is present in the namespace and at the root level, the claim value in the namespace takes precedence. From 1192a7ada6b7237aefd6742f7465878335d10eaf Mon Sep 17 00:00:00 2001 From: Matt Johnson-Pint Date: Thu, 5 Oct 2023 13:37:34 -0700 Subject: [PATCH 15/23] Docs(GraphQL): Fix indentation on graphql sample (#650) Indentation was off, causing rendering issues on https://dgraph.io/docs/graphql/quick-start/ --- content/graphql/quick-start/index.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/content/graphql/quick-start/index.md b/content/graphql/quick-start/index.md index c05bd2d5..6fd3d2ab 100644 --- a/content/graphql/quick-start/index.md +++ b/content/graphql/quick-start/index.md @@ -67,19 +67,21 @@ You may want to use the introspection capability of the client to explore the sc To populate the database, 1. Open the [API Explorer](https://cloud.dgraph.io/_/explorer) tab 2. Paste the following code into the text area: - ```graphql - mutation { - addProduct(input: [ - { name: "GraphQL on Dgraph"}, - { name: "Dgraph: The GraphQL Database"} - ]) { - product { - productID - name - } + ```graphql + mutation { + addProduct( + input: [ + { name: "GraphQL on Dgraph" } + { name: "Dgraph: The GraphQL Database" } + ] + ) { + product { + productID + name } - addCustomer(input: [{ username: "Michael"}]) { - customer { + } + addCustomer(input: [{ username: "Michael" }]) { + customer { username } } From ad3f8aef1943e525ccc7798da58bfd21c64b4724 Mon Sep 17 00:00:00 2001 From: danieldcrooks <130470822+danieldcrooks@users.noreply.github.com> Date: Fri, 5 Jan 2024 13:19:14 -0700 Subject: [PATCH 16/23] Update replication-concept.md (#649) Typo --- content/design-concepts/replication-concept.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/design-concepts/replication-concept.md b/content/design-concepts/replication-concept.md index 03089403..6003e0da 100644 --- a/content/design-concepts/replication-concept.md +++ b/content/design-concepts/replication-concept.md @@ -9,4 +9,4 @@ weight = 85 Each Highly-Available (HA) group will be served by at least 3 instances (or two if one is temporarily unavailable). In the case of an alpha instance failure, other alpha instances in the same group still handle the load for data in that group. In case of a zero instance failure, the remaining two zeros in the zero group will continue to hand out timestamps and perform other zero functions. -In addition, Dgraph `Learner Nodes` are alpha instances that hold replicas of data, but this replication is to suupport read replicas, often in a different geography from the master cluster. This replication is implemented the same way as HA replication, but the learner nodes do not participate in quorum, and do not take over from failed nodes to provide high availability. +In addition, Dgraph `Learner Nodes` are alpha instances that hold replicas of data, but this replication is to support read replicas, often in a different geography from the master cluster. This replication is implemented the same way as HA replication, but the learner nodes do not participate in quorum, and do not take over from failed nodes to provide high availability. From a2b060e5306c91e7dcfef5d2ba2d42edfc341bad Mon Sep 17 00:00:00 2001 From: Ryan Fox-Tyler <60440289+ryanfoxtyler@users.noreply.github.com> Date: Fri, 5 Jan 2024 16:11:06 -0500 Subject: [PATCH 17/23] Catch up v23.0 branch (#655) Co-authored-by: Jonathan Zollinger <62955101+Jonathan-Zollinger@users.noreply.github.com> Co-authored-by: Raphael Co-authored-by: ShivajiKharse <115525374+shivaji-dgraph@users.noreply.github.com> --- content/dql/dql-schema.md | 20 ++++++++ content/enterprise-features/binary-backups.md | 45 ++++++++++++++++++ content/releases/index.md | 46 ++++++++++--------- 3 files changed, 89 insertions(+), 22 deletions(-) diff --git a/content/dql/dql-schema.md b/content/dql/dql-schema.md index fe3bfbb6..c55ca531 100644 --- a/content/dql/dql-schema.md +++ b/content/dql/dql-schema.md @@ -187,8 +187,28 @@ Query: } ``` +### Unique Directive +The unique constraint enables us to guarantee that all values of a predicate are distinct. To implement the @unique +directive for a predicate, you should define it in the schema and create an index on the predicate based on its type. +If a user does not add the proper index to the predicate, then Dgraph will return an error. + +Dgraph will automatically include the @upsert directive for the predicate. To enforce this uniqueness constraint, +a predicate must have an index, as explained below. Currently, we only support the @unique directive on newly created +predicates with the data types string and integer. + +| Data Type | Index | +| -------- | ------- | +| string | hash, exact | +| int | int | + +This is how you define the unique directive for a predicate. + +``` +email: string @unique @index(exact) . +``` + ### Upsert directive diff --git a/content/enterprise-features/binary-backups.md b/content/enterprise-features/binary-backups.md index cfef7966..44fc0999 100644 --- a/content/enterprise-features/binary-backups.md +++ b/content/enterprise-features/binary-backups.md @@ -648,6 +648,51 @@ mutation{ } } ``` + +## Namespace Aware Restore + +You can use namespace-aware restore to restore a single namespace from a backup that contains multiple namespaces. +The created restore will be available in the default namespace. For example, if you restore namespace 2 using the +restoreTenant API, then after the restore operation is completed, the cluster will have only the default namespace, +and it will contain the data from namespace 2. Namespace aware restore supports incremental restore. + +To restore from a backup to a live cluster, execute a mutation on the `/admin` endpoint with the following format: + +```graphql +mutation { + restoreTenant( + input: { + restoreInput: { + incrementalFrom: "incremental_backup_from" + location: "/path/to/backup/directory" + backupId: "id_of_backup_to_restore" + } + fromNamespace: namespaceToBeRestored + } + ) { + message + code + } +} +``` + +Documentation of restoreTenant inputs + +``` +input RestoreTenantInput { + """ + restoreInput contains fields that are required for the restore operation, + i.e., location, backupId, and backupNum + """ + restoreInput: RestoreInput + + """ + fromNamespace is the namespace of the tenant that needs to be restored into namespace 0 of the new cluster. + """ + fromNamespace: Int! +} +``` + ## Offline restore The restore utility is now a standalone tool. A new flag, `--encryption key-file=value`, is now part of the restore utility, so you can use it to decrypt the backup. The file specified using this flag must contain the same key that was used for encryption during backup. Alternatively, starting with `v20.07.0`, the `vault` superflag can be used to restore a backup. diff --git a/content/releases/index.md b/content/releases/index.md index 12f2c8b1..f8a10f11 100644 --- a/content/releases/index.md +++ b/content/releases/index.md @@ -6,7 +6,7 @@ title = "Dgraph Releases" weight = 14 +++ -The latest Dgraph release is the v23.00 series. +The latest Dgraph release is the v23.1 series. Dgraph releases starting v22.0.0 is following semver [See the post here](https://discuss.dgraph.io/t/dgraph-v22-0-0-rc1-20221003-release-candidate/). @@ -18,25 +18,27 @@ To learn about the latest releases and other important announcements, watch the ## Release series - Dgraph Release Series | Current Release | Supported? | First Release Date | End of life ------------------------|-----------------|------------|--------------------|-------------- - v23.0.x | [v23.0.0][] | Yes | May 2023 | N/A - v22.0.x | [v22.0.2][] | Yes | October 2022 | N/A - v21.12.x(discontinued)| v21.12.0 | No | December 2021 | December 2022 - v21.03.x | [v21.03.0][] | Yes | March 2021 | June 2023 - v20.11.x | [v20.11.0][] | No | December 2020 | December 2021 - v20.07.x | [v20.07.3][] | No | July 2020 | July 2021 - v20.03.x | [v20.03.7][] | No | March 2020 | March 2021 - v1.2.x | [v1.2.8][] | No | January 2020 | January 2021 - v1.1.x | [v1.1.1][] | No | January 2020 | January 2021 - v1.0.x | [v1.0.18][] | No | December 2017 | March 2020 + Release | First Release Date | End of Life +-----------------------|--------------------|-------------- + [v23.1][] | October 2023 | April 2025 + [v23.0][] | May 2023 | November 2024 + [v22.0][] | October 2022 | April 2024 + v21.12 (discontinued) | December 2021 | December 2022 + [v21.03][] | March 2021 | June 2023 + [v20.11][] | December 2020 | December 2021 + [v20.07][] | July 2020 | July 2021 + [v20.03][] | March 2020 | March 2021 + [v1.2][] | January 2020 | January 2021 + [v1.1][] | January 2020 | January 2021 + [v1.0][] | December 2017 | March 2020 -[v23.0.0]: https://discuss.dgraph.io/t/dgraph-release-v23-0-0-is-now-generally-available/18634 -[v22.0.2]: https://discuss.dgraph.io/t/dgraph-release-v22-0-2-is-now-generally-available/18117 -[v21.03.0]: https://discuss.dgraph.io/t/release-notes-v21-03-0-resilient-rocket/13587 -[v20.11.0]: https://discuss.dgraph.io/t/release-notes-v20-11-0-tenacious-tchalla/11942 -[v20.07.3]: https://discuss.dgraph.io/t/dgraph-v20-07-3-release/12107 -[v20.03.7]: https://discuss.dgraph.io/t/dgraph-v20-03-7-release/12077 -[v1.2.8]: https://discuss.dgraph.io/t/dgraph-v1-2-8-release/11183 -[v1.1.1]: https://discuss.dgraph.io/t/dgraph-v1-1-1-release/5664 -[v1.0.18]: https://discuss.dgraph.io/t/dgraph-v1-0-18-release/5663 +[v23.1]: https://discuss.dgraph.io/t/dgraph-23-1-0-is-generally-available-on-dgraph-cloud-dockerhub-and-github/18980 +[v23.0]: https://discuss.dgraph.io/t/dgraph-release-v23-0-0-is-now-generally-available/18634 +[v22.0]: https://discuss.dgraph.io/t/dgraph-release-v22-0-2-is-now-generally-available/18117 +[v21.03]: https://discuss.dgraph.io/t/release-notes-v21-03-0-resilient-rocket/13587 +[v20.11]: https://discuss.dgraph.io/t/release-notes-v20-11-0-tenacious-tchalla/11942 +[v20.07]: https://discuss.dgraph.io/t/dgraph-v20-07-3-release/12107 +[v20.03]: https://discuss.dgraph.io/t/dgraph-v20-03-7-release/12077 +[v1.2]: https://discuss.dgraph.io/t/dgraph-v1-2-8-release/11183 +[v1.1]: https://discuss.dgraph.io/t/dgraph-v1-1-1-release/5664 +[v1.0]: https://discuss.dgraph.io/t/dgraph-v1-0-18-release/5663 From f45064e1a55ccc63601c2f543986d87ab9827a29 Mon Sep 17 00:00:00 2001 From: Ryan Fox-Tyler <60440289+ryanfoxtyler@users.noreply.github.com> Date: Fri, 5 Jan 2024 16:28:22 -0500 Subject: [PATCH 18/23] Update LICENSE.md --- LICENSE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/LICENSE.md b/LICENSE.md index b06f1274..2e1d5976 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -1,6 +1,6 @@ ## Dgraph Licensing -Copyright 2016-2021 Dgraph Labs, Inc. +Copyright 2016-2024 Dgraph Labs, Inc. Source code in this repository is variously licensed under the Apache Public License 2.0 (APL) and the Dgraph Community License (DCL). A copy of each license From 6b9a8d0516db940e23c18697cf4233391f3b62ea Mon Sep 17 00:00:00 2001 From: Ryan Fox-Tyler <60440289+ryanfoxtyler@users.noreply.github.com> Date: Fri, 5 Jan 2024 16:35:40 -0500 Subject: [PATCH 19/23] Update accept.txt --- .github/styles/Vocab/Dgraph/accept.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/styles/Vocab/Dgraph/accept.txt b/.github/styles/Vocab/Dgraph/accept.txt index 6a23ba98..1ac0c675 100644 --- a/.github/styles/Vocab/Dgraph/accept.txt +++ b/.github/styles/Vocab/Dgraph/accept.txt @@ -130,6 +130,7 @@ rebalancing unary loopback snake_case +semver Leia Skywalker From 91fb890a1b5e7b759c3305d311d0824a53e1ff8a Mon Sep 17 00:00:00 2001 From: rderbier Date: Tue, 16 Apr 2024 11:01:41 -0700 Subject: [PATCH 20/23] Update README.md --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 83c2ff24..a3cdd76e 100644 --- a/README.md +++ b/README.md @@ -102,3 +102,6 @@ Pass custom Go-GRPC example to the runnable by passing a `customExampleGoGRPC` t ``` **Note:** Runnable doesn't support passing a multiline string as an argument to a shortcode. Therefore, you have to create the whole custom example in a single line string by replacing newlines with `\n`. + +## History +add Hypermode banner by updating the hugo-docs repository with topbat template. \ No newline at end of file From 0b5bf3faf436ccca79ec0ecc39c3fed2f8068560 Mon Sep 17 00:00:00 2001 From: rderbier Date: Mon, 22 Apr 2024 13:13:49 -0700 Subject: [PATCH 21/23] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index a3cdd76e..c5ab4eae 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,9 @@ Making our documentation easy to understand includes optimizing it for client-si Use hugo shortcode for relref. Example, to reference a term, use a relref to the glossary : +``` > [entity]({{< relref "dgraph-glossary.md#entity" >}}) +``` ### Staging doc updates locally From 953335fe531d7216d56077389d0e120cb7585fbd Mon Sep 17 00:00:00 2001 From: rderbier Date: Mon, 22 Apr 2024 13:28:39 -0700 Subject: [PATCH 22/23] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c5ab4eae..b4e09fd7 100644 --- a/README.md +++ b/README.md @@ -106,4 +106,5 @@ Pass custom Go-GRPC example to the runnable by passing a `customExampleGoGRPC` t **Note:** Runnable doesn't support passing a multiline string as an argument to a shortcode. Therefore, you have to create the whole custom example in a single line string by replacing newlines with `\n`. ## History -add Hypermode banner by updating the hugo-docs repository with topbat template. \ No newline at end of file +add Hypermode banner by updating the hugo-docs repository with topbat template. +v24.0: \ No newline at end of file From d10ab88d7ae0d8998cc938d6454798c0cb265ef3 Mon Sep 17 00:00:00 2001 From: Gajanan Date: Tue, 23 Apr 2024 14:01:29 -0700 Subject: [PATCH 23/23] Add DQL additions for supporting vector type. New datatype has been added from 24.0 for supporting vector tyoe and semantic search. --- content/dql/dql-schema.md | 9 +++++++++ content/dql/predicate-indexing.md | 6 ++++++ content/query-language/functions.md | 14 ++++++++++++++ 3 files changed, 29 insertions(+) diff --git a/content/dql/dql-schema.md b/content/dql/dql-schema.md index c55ca531..182f08d3 100644 --- a/content/dql/dql-schema.md +++ b/content/dql/dql-schema.md @@ -16,6 +16,9 @@ revenue: float . running_time: int . starring: [uid] . director: [uid] . +description: string . + +description_vector: float32vector @index(hnsw(metric:"cosine")) . type Person { name @@ -28,6 +31,8 @@ type Film { running_time starring director + description + description_vector } ``` @@ -112,6 +117,10 @@ For all triples with a predicate of scalar types the object is a literal. are RFC 3339 compatible which is different from ISO 8601(as defined in the RDF spec). You should convert your values to RFC 3339 format before sending them to Dgraph.{{% /notice %}} +### Vector Type + +The `float32vector` type denotes a vector of floating point numbers used to store embeddings of one or more predicates or node embeddings. A schema type can contain more than one vector. + ### UID Type The `uid` type denotes a relationship; internally each node is identified by it's UID which is a `uint64`. diff --git a/content/dql/predicate-indexing.md b/content/dql/predicate-indexing.md index 2f7669c4..c8973a80 100644 --- a/content/dql/predicate-indexing.md +++ b/content/dql/predicate-indexing.md @@ -34,6 +34,12 @@ transaction conflict rate. Use only the minimum number of and simplest indexes that your application needs. {{% /notice %}} +## Vector Indices + +For fast semantic search `hsnw` (**Hierarchical Navigable Small World**) index is available on `float32vector`. The index is created based on one of the following distance metrics: `cosine`, `euclidean`, and `dotproduct`. + +Note that vector index must be defined using `@index` directive: `description_vector: float32vector @index(hnsw(metric:"cosine")) .` + ## DateTime Indices The indices available for `dateTime` are as follows. diff --git a/content/query-language/functions.md b/content/query-language/functions.md index f05f0f94..39b3f005 100644 --- a/content/query-language/functions.md +++ b/content/query-language/functions.md @@ -177,6 +177,20 @@ Same query with a Levenshtein distance of 3. } {{< /runnable >}} +## Similarity (aka Sematic) Search + +Syntax Examples: `similar_to(description_vector, 3, "[0.9, 0.8, 0, 0]")` + +Alternatively the vector can be passed as a variable: `similar_to(description_vector, 3, $vec)` + +This function finds the objects that have description_vector similar to the input vector. The second parameter, `3` specifies that top 3 matches be returned. + +Schema Types: `float32vector` + +Index Required: `hnsw` + +Similarity search results are based on distance metrics used to create index (`cosine`, `euclidean`, and `dotproduct`). The shorter distance indicates more similarity. + ## Full-Text Search Syntax Examples: `alloftext(predicate, "space-separated text")` and `anyoftext(predicate, "space-separated text")`