From 61326535a3c6eff1c3096243f8ed64aff7d3c49b Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 12:38:23 -0700 Subject: [PATCH 01/13] RAG docs for Go --- docs-go/Makefile | 2 +- docs-go/rag.md | 378 +++++++++++++++++++++++++++ docs-go/rag.src | 209 +++++++++++++++ go/internal/doc-snippets/rag/main.go | 227 ++++++++++++++++ 4 files changed, 815 insertions(+), 1 deletion(-) create mode 100644 docs-go/rag.md create mode 100644 docs-go/rag.src create mode 100644 go/internal/doc-snippets/rag/main.go diff --git a/docs-go/Makefile b/docs-go/Makefile index 2497cc9daf..7ea1efc74d 100644 --- a/docs-go/Makefile +++ b/docs-go/Makefile @@ -1,6 +1,6 @@ WEAVE=$(HOME)/go/bin/weave -all: $(WEAVE) get-started-go.md flows.md models.md prompts.md dotprompt.md +all: $(WEAVE) get-started-go.md flows.md models.md prompts.md dotprompt.md rag.md $(WEAVE): ../go/internal/cmd/weave/*.go go -C ../go install ./internal/cmd/weave diff --git a/docs-go/rag.md b/docs-go/rag.md new file mode 100644 index 0000000000..1f960d6b62 --- /dev/null +++ b/docs-go/rag.md @@ -0,0 +1,378 @@ + + +# Retrieval-augmented generation (RAG) + +Firebase Genkit provides abstractions that help you build retrieval-augmented generation +(RAG) flows, as well as plugins that provide integrations with related tools. + +## What is RAG? + +Retrieval-augmented generation is a technique used to incorporate external +sources of information into an LLM’s responses. It's important to be able to do +so because, while LLMs are typically trained on a broad body of +material, practical use of LLMs often requires specific domain knowledge (for +example, you might want to use an LLM to answer customers' questions about your +company’s products). + +One solution is to fine-tune the model using more specific data. However, this +can be expensive both in terms of compute cost and in terms of the effort needed +to prepare adequate training data. + +In contrast, RAG works by incorporating external data sources into a prompt at +the time it's passed to the model. For example, you could imagine the prompt, +"What is Bart's relationship to Lisa?" might be expanded ("augmented") by +prepending some relevant information, resulting in the prompt, "Homer and +Marge's children are named Bart, Lisa, and Maggie. What is Bart's relationship +to Lisa?" + +This approach has several advantages: + +- It can be more cost effective because you don't have to retrain the model. +- You can continuously update your data source and the LLM can immediately make + use of the updated information. +- You now have the potential to cite references in your LLM's responses. + +On the other hand, using RAG naturally means longer prompts, and some LLM API +services charge for each input token you send. Ultimately, you must evaluate the +cost tradeoffs for your applications. + +RAG is a very broad area and there are many different techniques used to achieve +the best quality RAG. The core Genkit framework offers two main abstractions to +help you do RAG: + +- Indexers: add documents to an "index". +- Embedders: transforms documents into a vector representation +- Retrievers: retrieve documents from an "index", given a query. + +These definitions are broad on purpose because Genkit is un-opinionated about +what an "index" is or how exactly documents are retrieved from it. Genkit only +provides a `Document` format and everything else is defined by the retriever or +indexer implementation provider. + +### Indexers + +The index is responsible for keeping track of your documents in such a way that +you can quickly retrieve relevant documents given a specific query. This is most +often accomplished using a vector database, which indexes your documents using +multidimensional vectors called embeddings. A text embedding (opaquely) +represents the concepts expressed by a passage of text; these are generated +using special-purpose ML models. By indexing text using its embedding, a vector +database is able to cluster conceptually related text and retrieve documents +related to a novel string of text (the query). + +Before you can retrieve documents for the purpose of generation, you need to +ingest them into your document index. A typical ingestion flow does the +following: + +1. Split up large documents into smaller documents so that only relevant + portions are used to augment your prompts – "chunking". This is necessary + because many LLMs have a limited context window, making it impractical to + include entire documents with a prompt. + + Genkit doesn't provide built-in chunking libraries; however, there are open + source libraries available that are compatible with Genkit. + +1. Generate embeddings for each chunk. Depending on the database you're using, + you might explicitly do this with an embedding generation model, or you + might use the embedding generator provided by the database. + +1. Add the text chunk and its index to the database. + +You might run your ingestion flow infrequently or only once if you are working +with a stable source of data. On the other hand, if you are working with data +that frequently changes, you might continuously run the ingestion flow (for +example, in a Cloud Firestore trigger, whenever a document is updated). + +### Embedders + +An embedder is a function that takes content (text, images, audio, etc.) and creates a numeric vector that encodes the semantic meaning of the original content. As mentioned above, embedders are leveraged as part of the process of indexing, however, they can also be used independently to create embeddings without an index. + +### Retrievers + +A retriever is a concept that encapsulates logic related to any kind of document +retrieval. The most popular retrieval cases typically include retrieval from +vector stores, however, in Genkit a retriever can be any function that returns data. + +To create a retriever, you can use one of the provided implementations or +create your own. + +## Supported indexers, retrievers, and embedders + +Genkit provides indexer and retriever support through its plugin system. The +following plugins are officially supported: + +- [Cloud Firestore vector store](plugins/firebase.md) +- [Pinecone](plugins/pinecone.md) cloud vector database + +In addition, Genkit supports the following vector stores through predefined +code templates, which you can customize for your database configuration and +schema: + +- PostgreSQL with [`pgvector`](templates/pgvector.md) + +Embedding model support is provided through the following plugins: + +| Plugin | Models | +| ------------------------- | -------------------- | +| [Google Generative AI][1] | Gecko text embedding | +| [Google Vertex AI][2] | Gecko text embedding | + +[1]: plugins/google-genai.md +[2]: plugins/vertex-ai.md + +## Defining a RAG Flow + +The following examples show how you could ingest a collection of restaurant menu PDF documents +into a vector database and retrieve them for use in a flow that determines what food items are available. + +### Install dependencies + +In this example, we will use the `textsplitter` library from `langchaingo` and +the `ledongthuc/pdf` PDF parsing Library: + +```posix-terminal +go get github.com/tmc/langchaingo/textsplitter + +go get github.com/ledongthuc/pdf +``` + +### Define an Indexer + +The following example shows how to create an indexer to ingest a collection of PDF documents +and store them in a local vector database. + +It uses the local file-based vector similarity retriever +that Genkit provides out-of-the box for simple testing and prototyping (_do not +use in production_) + +#### Create the indexer + +```go +// Import Genkit's file-based vector retriever, (Don't use in production.) +import "github.com/firebase/genkit/go/plugins/localvec" + +// Vertex AI provides the text-embedding-004 embedder model. +import "github.com/firebase/genkit/go/plugins/vertexai" +``` + +```go +ctx := context.Background() + +err := vertexai.Init(ctx, "", "") +if err != nil { + log.Fatal(err) +} +err = localvec.Init() +if err != nil { + log.Fatal(err) +} + +menuPdfIndexer, _, err := localvec.DefineIndexerAndRetriever( + "menuQA", + localvec.Config{ + Embedder: vertexai.Embedder("text-embedding-004"), + }, +) +``` + +#### Create chunking config + +This example uses the `textsplitter` library which provides a simple text +splitter to break up documents into segments that can be vectorized. + +The following definition configures the chunking function to return document +segments of 200 characters, with an overlap between chunks of 20 characters. + +```go +splitter := textsplitter.NewRecursiveCharacter( + textsplitter.WithChunkSize(200), + textsplitter.WithChunkOverlap(20), +) +``` + +More chunking options for this library can be found in the +[`langchaingo` documentation](https://pkg.go.dev/github.com/tmc/langchaingo/textsplitter#Option). + +#### Define your indexer flow + +```go +genkit.DefineFlow( + "indexMenu", + func(ctx context.Context, path string) (any, error) { + // Extract plain text from the PDF. + pdfText, err := genkit.Run(ctx, "extract", func() (string, error) { + return readPdf(path) + }) + if err != nil { + return nil, err + } + + // Split the text into chunks. + docs, err := genkit.Run(ctx, "chunk", func() ([]*ai.Document, error) { + chunks, err := splitter.SplitText(pdfText) + if err != nil { + return nil, err + } + + var docs []*ai.Document + for i := range len(chunks) { + docs = append(docs, ai.DocumentFromText(chunks[i], nil)) + } + return docs, nil + }) + if err != nil { + return nil, err + } + + // Add chunks to the index. + err = menuPdfIndexer.Index(ctx, &ai.IndexerRequest{ + Documents: docs, Options: nil, + }) + return nil, err + }, +) +``` + +```go +// Helper function to extract plain text from a PDF. Excerpted from +// https://github.com/ledongthuc/pdf +func readPdf(path string) (string, error) { + f, r, err := pdf.Open(path) + defer f.Close() + if err != nil { + return "", err + } + + buf := bytes.Buffer{} + b, err := r.GetPlainText() + if err != nil { + return "", err + } + buf.ReadFrom(b) + return buf.String(), nil +} +``` + +#### Run the indexer flow + +```posix-terminal +genkit flow:run indexMenu "'menu.pdf'" +``` + +After running the `indexMenu` flow, the vector database will be seeded with +documents and ready to be used in Genkit flows with retrieval steps. + +### Define a flow with retrieval + +The following example shows how you might use a retriever in a RAG flow. Like +the indexer example, this example uses Genkit's file-based vector retriever, +which you should not use in production. + +```go + ctx := context.Background() + + err := vertexai.Init(ctx, "", "") + if err != nil { + log.Fatal(err) + } + err = localvec.Init() + if err != nil { + log.Fatal(err) + } + + gemini15Pro := vertexai.Model("gemini-1.5-pro") + + _, menuPdfRetriever, err := localvec.DefineIndexerAndRetriever( + "menuQA", + localvec.Config{ + Embedder: vertexai.Embedder("text-embedding-004"), + }, + ) + + genkit.DefineFlow( + "menuQA", + func(ctx context.Context, question string) (string, error) { + // Retrieve text relevant to the user's question. + docs, err := menuPdfRetriever.Retrieve(ctx, &ai.RetrieverRequest{ + Document: ai.DocumentFromText(question, nil), + }) + if err != nil { + return "", err + } + + // Construct a system message containing the menu excerpts you just + // retrieved. + menuInfo := ai.NewSystemTextMessage("Here's the menu context:") + for i := range len(docs.Documents) { + menuInfo.Content = append(menuInfo.Content, + docs.Documents[i].Content...) + } + + // Call Generate, including the menu information in your prompt. + resp, err := gemini15Pro.Generate(ctx, &ai.GenerateRequest{ + Messages: []*ai.Message{ + ai.NewSystemTextMessage(` +You are acting as a helpful AI assistant that can answer questions about the +food available on the menu at Genkit Grub Pub. +Use only the context provided to answer the question. If you don't know, do not +make up an answer. Do not add or change items on the menu.`), + menuInfo, + ai.NewUserTextMessage(question), + }, + }, nil) + if err != nil { + return "", err + } + + return resp.Text() + }) +``` + +## Write your own indexers and retrievers + +It's also possible to create your own retriever. This is useful if your +documents are managed in a document store that is not supported in Genkit (eg: +MySQL, Google Drive, etc.). The Genkit SDK provides flexible methods that let +you provide custom code for fetching documents. + +You can also define custom retrievers that build on top of existing retrievers +in Genkit and apply advanced RAG techniques (such as reranking or prompt +extension) on top. + +For example, suppose you have a custom re-ranking function you want to use. The +following example defines a custom retriever that applies your function to the +menu retriever defined earlier: + +```go +type CustomMenuRetrieverOptions struct { + K int + PreRerankK int +} +advancedMenuRetriever := ai.DefineRetriever( + "custom", + "advancedMenuRetriever", + func(ctx context.Context, req *ai.RetrieverRequest) (*ai.RetrieverResponse, error) { + // Handle options passed using our custom type. + k := 3 + preRerankK := 10 + if opts, ok := req.Options.(CustomMenuRetrieverOptions); ok { + preRerankK = opts.PreRerankK + } + + // Call the retriever as in the simple case. + response, err := menuPdfRetriever.Retrieve(ctx, &ai.RetrieverRequest{ + Document: req.Document, + Options: localvec.RetrieverOptions{K: preRerankK}, + }) + if err != nil { + return nil, err + } + + // Re-rank the returned documents using your custom function. + rerankedDocs := rerank(response.Documents) + response.Documents = rerankedDocs[:k] + + return response, nil + }, +) +``` diff --git a/docs-go/rag.src b/docs-go/rag.src new file mode 100644 index 0000000000..645af068c1 --- /dev/null +++ b/docs-go/rag.src @@ -0,0 +1,209 @@ +# Retrieval-augmented generation (RAG) + +Firebase Genkit provides abstractions that help you build retrieval-augmented generation +(RAG) flows, as well as plugins that provide integrations with related tools. + +## What is RAG? + +Retrieval-augmented generation is a technique used to incorporate external +sources of information into an LLM’s responses. It's important to be able to do +so because, while LLMs are typically trained on a broad body of +material, practical use of LLMs often requires specific domain knowledge (for +example, you might want to use an LLM to answer customers' questions about your +company’s products). + +One solution is to fine-tune the model using more specific data. However, this +can be expensive both in terms of compute cost and in terms of the effort needed +to prepare adequate training data. + +In contrast, RAG works by incorporating external data sources into a prompt at +the time it's passed to the model. For example, you could imagine the prompt, +"What is Bart's relationship to Lisa?" might be expanded ("augmented") by +prepending some relevant information, resulting in the prompt, "Homer and +Marge's children are named Bart, Lisa, and Maggie. What is Bart's relationship +to Lisa?" + +This approach has several advantages: + +- It can be more cost effective because you don't have to retrain the model. +- You can continuously update your data source and the LLM can immediately make + use of the updated information. +- You now have the potential to cite references in your LLM's responses. + +On the other hand, using RAG naturally means longer prompts, and some LLM API +services charge for each input token you send. Ultimately, you must evaluate the +cost tradeoffs for your applications. + +RAG is a very broad area and there are many different techniques used to achieve +the best quality RAG. The core Genkit framework offers two main abstractions to +help you do RAG: + +- Indexers: add documents to an "index". +- Embedders: transforms documents into a vector representation +- Retrievers: retrieve documents from an "index", given a query. + +These definitions are broad on purpose because Genkit is un-opinionated about +what an "index" is or how exactly documents are retrieved from it. Genkit only +provides a `Document` format and everything else is defined by the retriever or +indexer implementation provider. + +### Indexers + +The index is responsible for keeping track of your documents in such a way that +you can quickly retrieve relevant documents given a specific query. This is most +often accomplished using a vector database, which indexes your documents using +multidimensional vectors called embeddings. A text embedding (opaquely) +represents the concepts expressed by a passage of text; these are generated +using special-purpose ML models. By indexing text using its embedding, a vector +database is able to cluster conceptually related text and retrieve documents +related to a novel string of text (the query). + +Before you can retrieve documents for the purpose of generation, you need to +ingest them into your document index. A typical ingestion flow does the +following: + +1. Split up large documents into smaller documents so that only relevant + portions are used to augment your prompts – "chunking". This is necessary + because many LLMs have a limited context window, making it impractical to + include entire documents with a prompt. + + Genkit doesn't provide built-in chunking libraries; however, there are open + source libraries available that are compatible with Genkit. + +1. Generate embeddings for each chunk. Depending on the database you're using, + you might explicitly do this with an embedding generation model, or you + might use the embedding generator provided by the database. + +1. Add the text chunk and its index to the database. + +You might run your ingestion flow infrequently or only once if you are working +with a stable source of data. On the other hand, if you are working with data +that frequently changes, you might continuously run the ingestion flow (for +example, in a Cloud Firestore trigger, whenever a document is updated). + +### Embedders + +An embedder is a function that takes content (text, images, audio, etc.) and creates a numeric vector that encodes the semantic meaning of the original content. As mentioned above, embedders are leveraged as part of the process of indexing, however, they can also be used independently to create embeddings without an index. + +### Retrievers + +A retriever is a concept that encapsulates logic related to any kind of document +retrieval. The most popular retrieval cases typically include retrieval from +vector stores, however, in Genkit a retriever can be any function that returns data. + +To create a retriever, you can use one of the provided implementations or +create your own. + +## Supported indexers, retrievers, and embedders + +Genkit provides indexer and retriever support through its plugin system. The +following plugins are officially supported: + +- [Cloud Firestore vector store](plugins/firebase.md) +- [Pinecone](plugins/pinecone.md) cloud vector database + +In addition, Genkit supports the following vector stores through predefined +code templates, which you can customize for your database configuration and +schema: + +- PostgreSQL with [`pgvector`](templates/pgvector.md) + +Embedding model support is provided through the following plugins: + +| Plugin | Models | +| ------------------------- | -------------------- | +| [Google Generative AI][1] | Gecko text embedding | +| [Google Vertex AI][2] | Gecko text embedding | + +[1]: plugins/google-genai.md +[2]: plugins/vertex-ai.md + +## Defining a RAG Flow + +The following examples show how you could ingest a collection of restaurant menu PDF documents +into a vector database and retrieve them for use in a flow that determines what food items are available. + +### Install dependencies + +In this example, we will use the `textsplitter` library from `langchaingo` and +the `ledongthuc/pdf` PDF parsing Library: + +```posix-terminal +go get github.com/tmc/langchaingo/textsplitter + +go get github.com/ledongthuc/pdf +``` + +### Define an Indexer + +The following example shows how to create an indexer to ingest a collection of PDF documents +and store them in a local vector database. + +It uses the local file-based vector similarity retriever +that Genkit provides out-of-the box for simple testing and prototyping (_do not +use in production_) + +#### Create the indexer + +```go +// Import Genkit's file-based vector retriever, (Don't use in production.) +import "github.com/firebase/genkit/go/plugins/localvec" + +// Vertex AI provides the text-embedding-004 embedder model. +import "github.com/firebase/genkit/go/plugins/vertexai" +``` + +%include ../go/internal/doc-snippets/rag/main.go vec + +#### Create chunking config + +This example uses the `textsplitter` library which provides a simple text +splitter to break up documents into segments that can be vectorized. + +The following definition configures the chunking function to return document +segments of 200 characters, with an overlap between chunks of 20 characters. + +%include ../go/internal/doc-snippets/rag/main.go splitcfg + +More chunking options for this library can be found in the +[`langchaingo` documentation](https://pkg.go.dev/github.com/tmc/langchaingo/textsplitter#Option). + +#### Define your indexer flow + +%include ../go/internal/doc-snippets/rag/main.go indexflow + +%include ../go/internal/doc-snippets/rag/main.go readpdf + +#### Run the indexer flow + +```posix-terminal +genkit flow:run indexMenu "'menu.pdf'" +``` + +After running the `indexMenu` flow, the vector database will be seeded with +documents and ready to be used in Genkit flows with retrieval steps. + +### Define a flow with retrieval + +The following example shows how you might use a retriever in a RAG flow. Like +the indexer example, this example uses Genkit's file-based vector retriever, +which you should not use in production. + +%include ../go/internal/doc-snippets/rag/main.go retrieve + +## Write your own indexers and retrievers + +It's also possible to create your own retriever. This is useful if your +documents are managed in a document store that is not supported in Genkit (eg: +MySQL, Google Drive, etc.). The Genkit SDK provides flexible methods that let +you provide custom code for fetching documents. + +You can also define custom retrievers that build on top of existing retrievers +in Genkit and apply advanced RAG techniques (such as reranking or prompt +extension) on top. + +For example, suppose you have a custom re-ranking function you want to use. The +following example defines a custom retriever that applies your function to the +menu retriever defined earlier: + +%include ../go/internal/doc-snippets/rag/main.go customret diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go new file mode 100644 index 0000000000..2d2c4af01a --- /dev/null +++ b/go/internal/doc-snippets/rag/main.go @@ -0,0 +1,227 @@ +// Copyright 2024 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package rag + +import ( + "bytes" + "context" + "log" + + "github.com/firebase/genkit/go/ai" + "github.com/firebase/genkit/go/genkit" + "github.com/firebase/genkit/go/plugins/localvec" + "github.com/firebase/genkit/go/plugins/vertexai" + + "github.com/ledongthuc/pdf" + "github.com/tmc/langchaingo/textsplitter" +) + +func main() { + //!+vec + ctx := context.Background() + + err := vertexai.Init(ctx, "", "") + if err != nil { + log.Fatal(err) + } + err = localvec.Init() + if err != nil { + log.Fatal(err) + } + + menuPdfIndexer, _, err := localvec.DefineIndexerAndRetriever( + "menuQA", + localvec.Config{ + Embedder: vertexai.Embedder("text-embedding-004"), + }, + ) + //!-vec + //!+splitcfg + splitter := textsplitter.NewRecursiveCharacter( + textsplitter.WithChunkSize(200), + textsplitter.WithChunkOverlap(20), + ) + //!-splitcfg + //!+indexflow + genkit.DefineFlow( + "indexMenu", + func(ctx context.Context, path string) (any, error) { + // Extract plain text from the PDF. + pdfText, err := genkit.Run(ctx, "extract", func() (string, error) { + return readPdf(path) + }) + if err != nil { + return nil, err + } + + // Split the text into chunks. + docs, err := genkit.Run(ctx, "chunk", func() ([]*ai.Document, error) { + chunks, err := splitter.SplitText(pdfText) + if err != nil { + return nil, err + } + + var docs []*ai.Document + for i := range len(chunks) { + docs = append(docs, ai.DocumentFromText(chunks[i], nil)) + } + return docs, nil + }) + if err != nil { + return nil, err + } + + // Add chunks to the index. + err = menuPdfIndexer.Index(ctx, &ai.IndexerRequest{ + Documents: docs, Options: nil, + }) + return nil, err + }, + ) + //!-indexflow + + genkit.Init(ctx, nil) +} + +//!+readpdf +// Helper function to extract plain text from a PDF. Excerpted from +// https://github.com/ledongthuc/pdf +func readPdf(path string) (string, error) { + f, r, err := pdf.Open(path) + defer f.Close() + if err != nil { + return "", err + } + + buf := bytes.Buffer{} + b, err := r.GetPlainText() + if err != nil { + return "", err + } + buf.ReadFrom(b) + return buf.String(), nil +} +//!-readpdf + +func menuQA() { + //!+retrieve + ctx := context.Background() + + err := vertexai.Init(ctx, "", "") + if err != nil { + log.Fatal(err) + } + err = localvec.Init() + if err != nil { + log.Fatal(err) + } + + gemini15Pro := vertexai.Model("gemini-1.5-pro") + + _, menuPdfRetriever, err := localvec.DefineIndexerAndRetriever( + "menuQA", + localvec.Config{ + Embedder: vertexai.Embedder("text-embedding-004"), + }, + ) + + genkit.DefineFlow( + "menuQA", + func(ctx context.Context, question string) (string, error) { + // Retrieve text relevant to the user's question. + docs, err := menuPdfRetriever.Retrieve(ctx, &ai.RetrieverRequest{ + Document: ai.DocumentFromText(question, nil), + }) + if err != nil { + return "", err + } + + // Construct a system message containing the menu excerpts you just + // retrieved. + menuInfo := ai.NewSystemTextMessage("Here's the menu context:") + for i := range len(docs.Documents) { + menuInfo.Content = append(menuInfo.Content, + docs.Documents[i].Content...) + } + + // Call Generate, including the menu information in your prompt. + resp, err := gemini15Pro.Generate(ctx, &ai.GenerateRequest{ + Messages: []*ai.Message{ + ai.NewSystemTextMessage(` +You are acting as a helpful AI assistant that can answer questions about the +food available on the menu at Genkit Grub Pub. +Use only the context provided to answer the question. If you don't know, do not +make up an answer. Do not add or change items on the menu.`), + menuInfo, + ai.NewUserTextMessage(question), + }, + }, nil) + if err != nil { + return "", err + } + + return resp.Text() + }) + //!-retrieve +} + +func customret() { + _, menuPdfRetriever, _ := localvec.DefineIndexerAndRetriever( + "menuQA", + localvec.Config{ + Embedder: vertexai.Embedder("text-embedding-004"), + }, + ) + + //!+customret + type CustomMenuRetrieverOptions struct { + K int + PreRerankK int + } + advancedMenuRetriever := ai.DefineRetriever( + "custom", + "advancedMenuRetriever", + func(ctx context.Context, req *ai.RetrieverRequest) (*ai.RetrieverResponse, error) { + // Handle options passed using our custom type. + k := 3 + preRerankK := 10 + if opts, ok := req.Options.(CustomMenuRetrieverOptions); ok { + preRerankK = opts.PreRerankK + } + + // Call the retriever as in the simple case. + response, err := menuPdfRetriever.Retrieve(ctx, &ai.RetrieverRequest{ + Document: req.Document, + Options: localvec.RetrieverOptions{K: preRerankK}, + }) + if err != nil { + return nil, err + } + + // Re-rank the returned documents using your custom function. + rerankedDocs := rerank(response.Documents) + response.Documents = rerankedDocs[:k] + + return response, nil + }, + ) + //!-customret + + _ = advancedMenuRetriever +} + +func rerank(document []*ai.Document) []*ai.Document { + panic("unimplemented") +} From ffd9e29f5d6b6c71188742f0e8eb6d7462b2f35a Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 12:42:27 -0700 Subject: [PATCH 02/13] dependencies in go.mod?? --- go/go.mod | 16 ++++++++++++++-- go/go.sum | 23 +++++++++++++++++++++++ 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/go/go.mod b/go/go.mod index 60f1a33dcb..c8ea70b340 100644 --- a/go/go.mod +++ b/go/go.mod @@ -9,7 +9,7 @@ require ( github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.46.0 github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.22.0 github.com/aymerick/raymond v2.0.2+incompatible - github.com/google/generative-ai-go v0.13.1-0.20240530125111-8decc9df4add + github.com/google/generative-ai-go v0.14.0 github.com/google/go-cmp v0.6.0 github.com/google/uuid v1.6.0 github.com/invopop/jsonschema v0.12.0 @@ -27,9 +27,19 @@ require ( gopkg.in/yaml.v3 v3.0.1 ) +require ( + github.com/dlclark/regexp2 v1.10.0 // indirect + github.com/pkoukk/tiktoken-go v0.1.6 // indirect + gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181 // indirect + gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82 // indirect + gitlab.com/golang-commonmark/markdown v0.0.0-20211110145824-bf3e522c626a // indirect + gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84 // indirect + gitlab.com/golang-commonmark/puny v0.0.0-20191124015043-9f83538fa04f // indirect +) + require ( cloud.google.com/go v0.114.0 // indirect - cloud.google.com/go/ai v0.5.0 // indirect + cloud.google.com/go/ai v0.6.0 // indirect cloud.google.com/go/auth v0.5.1 // indirect cloud.google.com/go/auth/oauth2adapt v0.2.2 // indirect cloud.google.com/go/compute/metadata v0.3.0 // indirect @@ -49,7 +59,9 @@ require ( github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect github.com/googleapis/gax-go/v2 v2.12.4 // indirect github.com/kr/text v0.2.0 // indirect + github.com/ledongthuc/pdf v0.0.0-20240201131950-da5b75280b06 github.com/mailru/easyjson v0.7.7 // indirect + github.com/tmc/langchaingo v0.1.12 github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f // indirect github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect go.opencensus.io v0.24.0 // indirect diff --git a/go/go.sum b/go/go.sum index ea68f9f6e6..8e9d841af3 100644 --- a/go/go.sum +++ b/go/go.sum @@ -3,6 +3,7 @@ cloud.google.com/go v0.114.0 h1:OIPFAdfrFDFO2ve2U7r/H5SwSbBzEdrBdE7xkgwc+kY= cloud.google.com/go v0.114.0/go.mod h1:ZV9La5YYxctro1HTPug5lXH/GefROyW8PPD4T8n9J8E= cloud.google.com/go/ai v0.5.0 h1:x8s4rDn5t9OVZvBCgtr5bZTH5X0O7JdE6zYo+O+MpRw= cloud.google.com/go/ai v0.5.0/go.mod h1:96VBphk70e0zdXZrbtgPuKYRZsQ3UktSUXhuojwiKA8= +cloud.google.com/go/ai v0.6.0/go.mod h1:6/mrRq6aJdK7MZH76ZvcMpESiAiha5aRvurmroiOrgI= cloud.google.com/go/aiplatform v1.68.0 h1:EPPqgHDJpBZKRvv+OsB3cr0jYz3EL2pZ+802rBPcG8U= cloud.google.com/go/aiplatform v1.68.0/go.mod h1:105MFA3svHjC3Oazl7yjXAmIR89LKhRAeNdnDKJczME= cloud.google.com/go/auth v0.5.1 h1:0QNO7VThG54LUzKiQxv8C6x1YX7lUrzlAa1nVLF8CIw= @@ -45,6 +46,8 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dlclark/regexp2 v1.10.0 h1:+/GIL799phkJqYW+3YbOd8LCcbHzT0Pbo8zl70MHsq0= +github.com/dlclark/regexp2 v1.10.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= @@ -74,6 +77,7 @@ github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/google/generative-ai-go v0.13.1-0.20240530125111-8decc9df4add h1:ppPgQNwv1OidlzYyoeN6AEfcPJ5f2cO0hK2UzKDnoXc= github.com/google/generative-ai-go v0.13.1-0.20240530125111-8decc9df4add/go.mod h1:Pmy+JWGfZt1kjjKPpufz2uunTIOy+dhWA3aOIC7ub3Q= +github.com/google/generative-ai-go v0.14.0/go.mod h1:hOzbW3cB5hRV2x05McOwJS4GsqSluYwejjk5tSfb6YY= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= @@ -100,13 +104,18 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/ledongthuc/pdf v0.0.0-20240201131950-da5b75280b06 h1:kacRlPN7EN++tVpGUorNGPn/4DnB7/DfTY82AOn6ccU= +github.com/ledongthuc/pdf v0.0.0-20240201131950-da5b75280b06/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/pkoukk/tiktoken-go v0.1.6 h1:JF0TlJzhTbrI30wCvFuiw6FzP2+/bR+FIxUdgEAcUsw= +github.com/pkoukk/tiktoken-go v0.1.6/go.mod h1:9NiV+i9mJKGj1rYOT+njbv+ZwA/zJxYdewGl6qVatpg= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= @@ -116,6 +125,8 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/tmc/langchaingo v0.1.12 h1:yXwSu54f3b1IKw0jJ5/DWu+qFVH1NBblwC0xddBzGJE= +github.com/tmc/langchaingo v0.1.12/go.mod h1:cd62xD6h+ouk8k/QQFhOsjRYBSA1JJ5UVKXSIgm7Ni4= github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc= github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw= github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f h1:J9EGpcZtP0E/raorCMxlFGSTBrsSlaDGf3jU/qvAE2c= @@ -124,6 +135,17 @@ github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHo github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ= github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74= github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y= +gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181 h1:K+bMSIx9A7mLES1rtG+qKduLIXq40DAzYHtb0XuCukA= +gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181/go.mod h1:dzYhVIwWCtzPAa4QP98wfB9+mzt33MSmM8wsKiMi2ow= +gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82 h1:oYrL81N608MLZhma3ruL8qTM4xcpYECGut8KSxRY59g= +gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82/go.mod h1:Gn+LZmCrhPECMD3SOKlE+BOHwhOYD9j7WT9NUtkCrC8= +gitlab.com/golang-commonmark/markdown v0.0.0-20211110145824-bf3e522c626a h1:O85GKETcmnCNAfv4Aym9tepU8OE0NmcZNqPlXcsBKBs= +gitlab.com/golang-commonmark/markdown v0.0.0-20211110145824-bf3e522c626a/go.mod h1:LaSIs30YPGs1H5jwGgPhLzc8vkNc/k0rDX/fEZqiU/M= +gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84 h1:qqjvoVXdWIcZCLPMlzgA7P9FZWdPGPvP/l3ef8GzV6o= +gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84/go.mod h1:IJZ+fdMvbW2qW6htJx7sLJ04FEs4Ldl/MDsJtMKywfw= +gitlab.com/golang-commonmark/puny v0.0.0-20191124015043-9f83538fa04f h1:Wku8eEdeJqIOFHtrfkYUByc4bCaTeA6fL0UJgfEiFMI= +gitlab.com/golang-commonmark/puny v0.0.0-20191124015043-9f83538fa04f/go.mod h1:Tiuhl+njh/JIg0uS/sOJVYi0x2HEa5rc1OAaVsb5tAs= +gitlab.com/opennota/wd v0.0.0-20180912061657-c5d65f63c638/go.mod h1:EGRJaqe2eO9XGmFtQCvV3Lm9NLico3UhFwUpCG/+mVU= go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.51.0 h1:A3SayB3rNyt+1S6qpI9mHPkeHTZbD7XILEqWnYZb2l0= @@ -173,6 +195,7 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y= golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk= golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= From 857316baa30995d8f2bdb35772d06ad22f69c0ba Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 12:44:14 -0700 Subject: [PATCH 03/13] prettier --- docs-go/rag.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-go/rag.md b/docs-go/rag.md index 1f960d6b62..2926ff0453 100644 --- a/docs-go/rag.md +++ b/docs-go/rag.md @@ -234,7 +234,7 @@ genkit.DefineFlow( ``` ```go -// Helper function to extract plain text from a PDF. Excerpted from +// Helper function to extract plain text from a PDF. Excerpted from // https://github.com/ledongthuc/pdf func readPdf(path string) (string, error) { f, r, err := pdf.Open(path) From 00438836a32dc80bbd6e6cb08c521639df521357 Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 13:32:24 -0700 Subject: [PATCH 04/13] Don't add 3p dependencies --- go/go.mod | 18 ++-------- go/go.sum | 25 +------------- go/internal/doc-snippets/rag/main.go | 6 ++-- go/internal/doc-snippets/rag/pdf/pdf.go | 30 +++++++++++++++++ .../rag/textsplitter/textsplitter.go | 33 +++++++++++++++++++ 5 files changed, 71 insertions(+), 41 deletions(-) create mode 100644 go/internal/doc-snippets/rag/pdf/pdf.go create mode 100644 go/internal/doc-snippets/rag/textsplitter/textsplitter.go diff --git a/go/go.mod b/go/go.mod index c8ea70b340..52b7fe0f68 100644 --- a/go/go.mod +++ b/go/go.mod @@ -9,7 +9,7 @@ require ( github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.46.0 github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.22.0 github.com/aymerick/raymond v2.0.2+incompatible - github.com/google/generative-ai-go v0.14.0 + github.com/google/generative-ai-go v0.13.1-0.20240530125111-8decc9df4add github.com/google/go-cmp v0.6.0 github.com/google/uuid v1.6.0 github.com/invopop/jsonschema v0.12.0 @@ -27,19 +27,9 @@ require ( gopkg.in/yaml.v3 v3.0.1 ) -require ( - github.com/dlclark/regexp2 v1.10.0 // indirect - github.com/pkoukk/tiktoken-go v0.1.6 // indirect - gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181 // indirect - gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82 // indirect - gitlab.com/golang-commonmark/markdown v0.0.0-20211110145824-bf3e522c626a // indirect - gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84 // indirect - gitlab.com/golang-commonmark/puny v0.0.0-20191124015043-9f83538fa04f // indirect -) - require ( cloud.google.com/go v0.114.0 // indirect - cloud.google.com/go/ai v0.6.0 // indirect + cloud.google.com/go/ai v0.5.0 // indirect cloud.google.com/go/auth v0.5.1 // indirect cloud.google.com/go/auth/oauth2adapt v0.2.2 // indirect cloud.google.com/go/compute/metadata v0.3.0 // indirect @@ -59,9 +49,7 @@ require ( github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect github.com/googleapis/gax-go/v2 v2.12.4 // indirect github.com/kr/text v0.2.0 // indirect - github.com/ledongthuc/pdf v0.0.0-20240201131950-da5b75280b06 github.com/mailru/easyjson v0.7.7 // indirect - github.com/tmc/langchaingo v0.1.12 github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f // indirect github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect go.opencensus.io v0.24.0 // indirect @@ -79,4 +67,4 @@ require ( google.golang.org/genproto/googleapis/rpc v0.0.0-20240604185151-ef581f913117 // indirect google.golang.org/grpc v1.64.0 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect -) +) \ No newline at end of file diff --git a/go/go.sum b/go/go.sum index 8e9d841af3..b8053bb28e 100644 --- a/go/go.sum +++ b/go/go.sum @@ -3,7 +3,6 @@ cloud.google.com/go v0.114.0 h1:OIPFAdfrFDFO2ve2U7r/H5SwSbBzEdrBdE7xkgwc+kY= cloud.google.com/go v0.114.0/go.mod h1:ZV9La5YYxctro1HTPug5lXH/GefROyW8PPD4T8n9J8E= cloud.google.com/go/ai v0.5.0 h1:x8s4rDn5t9OVZvBCgtr5bZTH5X0O7JdE6zYo+O+MpRw= cloud.google.com/go/ai v0.5.0/go.mod h1:96VBphk70e0zdXZrbtgPuKYRZsQ3UktSUXhuojwiKA8= -cloud.google.com/go/ai v0.6.0/go.mod h1:6/mrRq6aJdK7MZH76ZvcMpESiAiha5aRvurmroiOrgI= cloud.google.com/go/aiplatform v1.68.0 h1:EPPqgHDJpBZKRvv+OsB3cr0jYz3EL2pZ+802rBPcG8U= cloud.google.com/go/aiplatform v1.68.0/go.mod h1:105MFA3svHjC3Oazl7yjXAmIR89LKhRAeNdnDKJczME= cloud.google.com/go/auth v0.5.1 h1:0QNO7VThG54LUzKiQxv8C6x1YX7lUrzlAa1nVLF8CIw= @@ -46,8 +45,6 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/dlclark/regexp2 v1.10.0 h1:+/GIL799phkJqYW+3YbOd8LCcbHzT0Pbo8zl70MHsq0= -github.com/dlclark/regexp2 v1.10.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= @@ -77,7 +74,6 @@ github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/google/generative-ai-go v0.13.1-0.20240530125111-8decc9df4add h1:ppPgQNwv1OidlzYyoeN6AEfcPJ5f2cO0hK2UzKDnoXc= github.com/google/generative-ai-go v0.13.1-0.20240530125111-8decc9df4add/go.mod h1:Pmy+JWGfZt1kjjKPpufz2uunTIOy+dhWA3aOIC7ub3Q= -github.com/google/generative-ai-go v0.14.0/go.mod h1:hOzbW3cB5hRV2x05McOwJS4GsqSluYwejjk5tSfb6YY= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= @@ -104,18 +100,13 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/ledongthuc/pdf v0.0.0-20240201131950-da5b75280b06 h1:kacRlPN7EN++tVpGUorNGPn/4DnB7/DfTY82AOn6ccU= -github.com/ledongthuc/pdf v0.0.0-20240201131950-da5b75280b06/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= -github.com/pkoukk/tiktoken-go v0.1.6 h1:JF0TlJzhTbrI30wCvFuiw6FzP2+/bR+FIxUdgEAcUsw= -github.com/pkoukk/tiktoken-go v0.1.6/go.mod h1:9NiV+i9mJKGj1rYOT+njbv+ZwA/zJxYdewGl6qVatpg= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= -github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= @@ -125,8 +116,6 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= -github.com/tmc/langchaingo v0.1.12 h1:yXwSu54f3b1IKw0jJ5/DWu+qFVH1NBblwC0xddBzGJE= -github.com/tmc/langchaingo v0.1.12/go.mod h1:cd62xD6h+ouk8k/QQFhOsjRYBSA1JJ5UVKXSIgm7Ni4= github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc= github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw= github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f h1:J9EGpcZtP0E/raorCMxlFGSTBrsSlaDGf3jU/qvAE2c= @@ -135,17 +124,6 @@ github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHo github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ= github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74= github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y= -gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181 h1:K+bMSIx9A7mLES1rtG+qKduLIXq40DAzYHtb0XuCukA= -gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181/go.mod h1:dzYhVIwWCtzPAa4QP98wfB9+mzt33MSmM8wsKiMi2ow= -gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82 h1:oYrL81N608MLZhma3ruL8qTM4xcpYECGut8KSxRY59g= -gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82/go.mod h1:Gn+LZmCrhPECMD3SOKlE+BOHwhOYD9j7WT9NUtkCrC8= -gitlab.com/golang-commonmark/markdown v0.0.0-20211110145824-bf3e522c626a h1:O85GKETcmnCNAfv4Aym9tepU8OE0NmcZNqPlXcsBKBs= -gitlab.com/golang-commonmark/markdown v0.0.0-20211110145824-bf3e522c626a/go.mod h1:LaSIs30YPGs1H5jwGgPhLzc8vkNc/k0rDX/fEZqiU/M= -gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84 h1:qqjvoVXdWIcZCLPMlzgA7P9FZWdPGPvP/l3ef8GzV6o= -gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84/go.mod h1:IJZ+fdMvbW2qW6htJx7sLJ04FEs4Ldl/MDsJtMKywfw= -gitlab.com/golang-commonmark/puny v0.0.0-20191124015043-9f83538fa04f h1:Wku8eEdeJqIOFHtrfkYUByc4bCaTeA6fL0UJgfEiFMI= -gitlab.com/golang-commonmark/puny v0.0.0-20191124015043-9f83538fa04f/go.mod h1:Tiuhl+njh/JIg0uS/sOJVYi0x2HEa5rc1OAaVsb5tAs= -gitlab.com/opennota/wd v0.0.0-20180912061657-c5d65f63c638/go.mod h1:EGRJaqe2eO9XGmFtQCvV3Lm9NLico3UhFwUpCG/+mVU= go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.51.0 h1:A3SayB3rNyt+1S6qpI9mHPkeHTZbD7XILEqWnYZb2l0= @@ -195,7 +173,6 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y= golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk= golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= @@ -247,4 +224,4 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= \ No newline at end of file diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index 2d2c4af01a..c2455090c7 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -24,8 +24,10 @@ import ( "github.com/firebase/genkit/go/plugins/localvec" "github.com/firebase/genkit/go/plugins/vertexai" - "github.com/ledongthuc/pdf" - "github.com/tmc/langchaingo/textsplitter" + // "github.com/ledongthuc/pdf" + // "github.com/tmc/langchaingo/textsplitter" + "github.com/firebase/genkit/go/internal/doc-snippets/rag/pdf" + "github.com/firebase/genkit/go/internal/doc-snippets/rag/textsplitter" ) func main() { diff --git a/go/internal/doc-snippets/rag/pdf/pdf.go b/go/internal/doc-snippets/rag/pdf/pdf.go new file mode 100644 index 0000000000..3cb2c48d8f --- /dev/null +++ b/go/internal/doc-snippets/rag/pdf/pdf.go @@ -0,0 +1,30 @@ +// Copyright 2024 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package pdf + +import ( + "io" + "os" +) + +type Reader struct {} + +func (Reader) GetPlainText() (io.Reader, error) { + panic("Stub.") +} + +func Open(file string) (*os.File, *Reader, error) { + panic("Stub.") +} \ No newline at end of file diff --git a/go/internal/doc-snippets/rag/textsplitter/textsplitter.go b/go/internal/doc-snippets/rag/textsplitter/textsplitter.go new file mode 100644 index 0000000000..c827cd2e88 --- /dev/null +++ b/go/internal/doc-snippets/rag/textsplitter/textsplitter.go @@ -0,0 +1,33 @@ +// Copyright 2024 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package textsplitter + +type Splitter struct{} + +func (Splitter) SplitText(t string) ([]string, error) { + panic("Stub.") +} + +func NewRecursiveCharacter(args ...any) Splitter { + panic("Stub.") +} + +func WithChunkSize(n int) any { + panic("Stub.") +} + +func WithChunkOverlap(n int) any { + panic("Stub.") +} From 6d3f50c3bf68812ba89eb56856706586d037674a Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 13:33:52 -0700 Subject: [PATCH 05/13] crlf --- go/go.mod | 2 +- go/go.sum | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/go/go.mod b/go/go.mod index 52b7fe0f68..60f1a33dcb 100644 --- a/go/go.mod +++ b/go/go.mod @@ -67,4 +67,4 @@ require ( google.golang.org/genproto/googleapis/rpc v0.0.0-20240604185151-ef581f913117 // indirect google.golang.org/grpc v1.64.0 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect -) \ No newline at end of file +) diff --git a/go/go.sum b/go/go.sum index b8053bb28e..ea68f9f6e6 100644 --- a/go/go.sum +++ b/go/go.sum @@ -224,4 +224,4 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= \ No newline at end of file +honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= From 972b74072656888c43c037df078f90b0817684bc Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 13:53:50 -0700 Subject: [PATCH 06/13] feedback --- docs-go/rag.md | 52 ++++++++++---------- go/internal/doc-snippets/rag/main.go | 71 +++++++++++++++------------- 2 files changed, 63 insertions(+), 60 deletions(-) diff --git a/docs-go/rag.md b/docs-go/rag.md index 2926ff0453..88c329d4b1 100644 --- a/docs-go/rag.md +++ b/docs-go/rag.md @@ -167,7 +167,7 @@ if err != nil { log.Fatal(err) } -menuPdfIndexer, _, err := localvec.DefineIndexerAndRetriever( +menuPDFIndexer, _, err := localvec.DefineIndexerAndRetriever( "menuQA", localvec.Config{ Embedder: vertexai.Embedder("text-embedding-004"), @@ -199,15 +199,17 @@ More chunking options for this library can be found in the genkit.DefineFlow( "indexMenu", func(ctx context.Context, path string) (any, error) { - // Extract plain text from the PDF. + // Extract plain text from the PDF. Wrap the logic in Tun so it + // appears as a step in your traces. pdfText, err := genkit.Run(ctx, "extract", func() (string, error) { - return readPdf(path) + return readPDF(path) }) if err != nil { return nil, err } - // Split the text into chunks. + // Split the text into chunks. Wrap the logic in Tun so it + // appears as a step in your traces. docs, err := genkit.Run(ctx, "chunk", func() ([]*ai.Document, error) { chunks, err := splitter.SplitText(pdfText) if err != nil { @@ -215,8 +217,8 @@ genkit.DefineFlow( } var docs []*ai.Document - for i := range len(chunks) { - docs = append(docs, ai.DocumentFromText(chunks[i], nil)) + for _, chunk := range chunks { + docs = append(docs, ai.DocumentFromText(chunk, nil)) } return docs, nil }) @@ -225,9 +227,7 @@ genkit.DefineFlow( } // Add chunks to the index. - err = menuPdfIndexer.Index(ctx, &ai.IndexerRequest{ - Documents: docs, Options: nil, - }) + err = menuPDFIndexer.Index(ctx, &ai.IndexerRequest{Documents: docs}) return nil, err }, ) @@ -236,20 +236,21 @@ genkit.DefineFlow( ```go // Helper function to extract plain text from a PDF. Excerpted from // https://github.com/ledongthuc/pdf -func readPdf(path string) (string, error) { +func readPDF(path string) (string, error) { f, r, err := pdf.Open(path) - defer f.Close() + if err != nil { + return "", err + } + defer f.Close() + + reader, err := r.GetPlainText() if err != nil { return "", err } - buf := bytes.Buffer{} - b, err := r.GetPlainText() - if err != nil { - return "", err - } - buf.ReadFrom(b) - return buf.String(), nil + var buf []byte + _, err = reader.Read(buf) + return string(buf), err } ``` @@ -280,7 +281,7 @@ which you should not use in production. log.Fatal(err) } - gemini15Pro := vertexai.Model("gemini-1.5-pro") + model := vertexai.Model("gemini-1.5-pro") _, menuPdfRetriever, err := localvec.DefineIndexerAndRetriever( "menuQA", @@ -303,13 +304,12 @@ which you should not use in production. // Construct a system message containing the menu excerpts you just // retrieved. menuInfo := ai.NewSystemTextMessage("Here's the menu context:") - for i := range len(docs.Documents) { - menuInfo.Content = append(menuInfo.Content, - docs.Documents[i].Content...) + for _, doc := range docs.Documents { + menuInfo.Content = append(menuInfo.Content, doc.Content...) } // Call Generate, including the menu information in your prompt. - resp, err := gemini15Pro.Generate(ctx, &ai.GenerateRequest{ + resp, err := model.Generate(ctx, &ai.GenerateRequest{ Messages: []*ai.Message{ ai.NewSystemTextMessage(` You are acting as a helpful AI assistant that can answer questions about the @@ -345,7 +345,7 @@ menu retriever defined earlier: ```go type CustomMenuRetrieverOptions struct { - K int + K int PreRerankK int } advancedMenuRetriever := ai.DefineRetriever( @@ -360,9 +360,9 @@ advancedMenuRetriever := ai.DefineRetriever( } // Call the retriever as in the simple case. - response, err := menuPdfRetriever.Retrieve(ctx, &ai.RetrieverRequest{ + response, err := menuPDFRetriever.Retrieve(ctx, &ai.RetrieverRequest{ Document: req.Document, - Options: localvec.RetrieverOptions{K: preRerankK}, + Options: localvec.RetrieverOptions{K: preRerankK}, }) if err != nil { return nil, err diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index c2455090c7..b68515b102 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -15,7 +15,6 @@ package rag import ( - "bytes" "context" "log" @@ -43,7 +42,7 @@ func main() { log.Fatal(err) } - menuPdfIndexer, _, err := localvec.DefineIndexerAndRetriever( + menuPDFIndexer, _, err := localvec.DefineIndexerAndRetriever( "menuQA", localvec.Config{ Embedder: vertexai.Embedder("text-embedding-004"), @@ -60,15 +59,17 @@ func main() { genkit.DefineFlow( "indexMenu", func(ctx context.Context, path string) (any, error) { - // Extract plain text from the PDF. + // Extract plain text from the PDF. Wrap the logic in Tun so it + // appears as a step in your traces. pdfText, err := genkit.Run(ctx, "extract", func() (string, error) { - return readPdf(path) + return readPDF(path) }) if err != nil { return nil, err } - // Split the text into chunks. + // Split the text into chunks. Wrap the logic in Tun so it + // appears as a step in your traces. docs, err := genkit.Run(ctx, "chunk", func() ([]*ai.Document, error) { chunks, err := splitter.SplitText(pdfText) if err != nil { @@ -76,8 +77,8 @@ func main() { } var docs []*ai.Document - for i := range len(chunks) { - docs = append(docs, ai.DocumentFromText(chunks[i], nil)) + for _, chunk := range chunks { + docs = append(docs, ai.DocumentFromText(chunk, nil)) } return docs, nil }) @@ -86,35 +87,38 @@ func main() { } // Add chunks to the index. - err = menuPdfIndexer.Index(ctx, &ai.IndexerRequest{ - Documents: docs, Options: nil, - }) + err = menuPDFIndexer.Index(ctx, &ai.IndexerRequest{Documents: docs}) return nil, err }, ) //!-indexflow - - genkit.Init(ctx, nil) + + err = genkit.Init(ctx, nil) + if err != nil { + log.Fatal(err) + } } -//!+readpdf -// Helper function to extract plain text from a PDF. Excerpted from +// !+readpdf +// Helper function to extract plain text from a PDF. Excerpted from // https://github.com/ledongthuc/pdf -func readPdf(path string) (string, error) { +func readPDF(path string) (string, error) { f, r, err := pdf.Open(path) - defer f.Close() if err != nil { return "", err } - - buf := bytes.Buffer{} - b, err := r.GetPlainText() - if err != nil { - return "", err - } - buf.ReadFrom(b) - return buf.String(), nil + defer f.Close() + + reader, err := r.GetPlainText() + if err != nil { + return "", err + } + + var buf []byte + _, err = reader.Read(buf) + return string(buf), err } + //!-readpdf func menuQA() { @@ -130,7 +134,7 @@ func menuQA() { log.Fatal(err) } - gemini15Pro := vertexai.Model("gemini-1.5-pro") + model := vertexai.Model("gemini-1.5-pro") _, menuPdfRetriever, err := localvec.DefineIndexerAndRetriever( "menuQA", @@ -153,13 +157,12 @@ func menuQA() { // Construct a system message containing the menu excerpts you just // retrieved. menuInfo := ai.NewSystemTextMessage("Here's the menu context:") - for i := range len(docs.Documents) { - menuInfo.Content = append(menuInfo.Content, - docs.Documents[i].Content...) + for _, doc := range docs.Documents { + menuInfo.Content = append(menuInfo.Content, doc.Content...) } // Call Generate, including the menu information in your prompt. - resp, err := gemini15Pro.Generate(ctx, &ai.GenerateRequest{ + resp, err := model.Generate(ctx, &ai.GenerateRequest{ Messages: []*ai.Message{ ai.NewSystemTextMessage(` You are acting as a helpful AI assistant that can answer questions about the @@ -176,11 +179,11 @@ make up an answer. Do not add or change items on the menu.`), return resp.Text() }) - //!-retrieve + //!-retrieve } func customret() { - _, menuPdfRetriever, _ := localvec.DefineIndexerAndRetriever( + _, menuPDFRetriever, _ := localvec.DefineIndexerAndRetriever( "menuQA", localvec.Config{ Embedder: vertexai.Embedder("text-embedding-004"), @@ -189,7 +192,7 @@ func customret() { //!+customret type CustomMenuRetrieverOptions struct { - K int + K int PreRerankK int } advancedMenuRetriever := ai.DefineRetriever( @@ -204,9 +207,9 @@ func customret() { } // Call the retriever as in the simple case. - response, err := menuPdfRetriever.Retrieve(ctx, &ai.RetrieverRequest{ + response, err := menuPDFRetriever.Retrieve(ctx, &ai.RetrieverRequest{ Document: req.Document, - Options: localvec.RetrieverOptions{K: preRerankK}, + Options: localvec.RetrieverOptions{K: preRerankK}, }) if err != nil { return nil, err From a83cb8eedddc6b039e0241f01c8996140b2a6b2e Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 14:00:02 -0700 Subject: [PATCH 07/13] update vertex.Init() --- go/internal/doc-snippets/rag/main.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index b68515b102..fb88536d61 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -33,7 +33,7 @@ func main() { //!+vec ctx := context.Background() - err := vertexai.Init(ctx, "", "") + err := vertexai.Init(ctx, &vertexai.Config{}) if err != nil { log.Fatal(err) } @@ -125,7 +125,7 @@ func menuQA() { //!+retrieve ctx := context.Background() - err := vertexai.Init(ctx, "", "") + err := vertexai.Init(ctx, &vertexai.Config{}) if err != nil { log.Fatal(err) } From bce6a18b03bfb4695d6c123e76a247488864a19d Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 14:06:58 -0700 Subject: [PATCH 08/13] check f != nil --- go/internal/doc-snippets/rag/main.go | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index fb88536d61..7e0277b07e 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -107,7 +107,9 @@ func readPDF(path string) (string, error) { if err != nil { return "", err } - defer f.Close() + if f != nil { + defer f.Close() + } reader, err := r.GetPlainText() if err != nil { @@ -203,6 +205,7 @@ func customret() { k := 3 preRerankK := 10 if opts, ok := req.Options.(CustomMenuRetrieverOptions); ok { + k = opts.K preRerankK = opts.PreRerankK } From e356f58fddeecaab1310af73c266bea1414b69e8 Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 14:07:50 -0700 Subject: [PATCH 09/13] order --- go/internal/doc-snippets/rag/main.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index 7e0277b07e..f107f507cd 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -104,12 +104,12 @@ func main() { // https://github.com/ledongthuc/pdf func readPDF(path string) (string, error) { f, r, err := pdf.Open(path) - if err != nil { - return "", err - } if f != nil { defer f.Close() } + if err != nil { + return "", err + } reader, err := r.GetPlainText() if err != nil { From 86bc341c8d156b1b0357704e45c14688448908a7 Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 14:38:03 -0700 Subject: [PATCH 10/13] make --- docs-go/rag.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs-go/rag.md b/docs-go/rag.md index 88c329d4b1..c74c902d8f 100644 --- a/docs-go/rag.md +++ b/docs-go/rag.md @@ -158,7 +158,7 @@ import "github.com/firebase/genkit/go/plugins/vertexai" ```go ctx := context.Background() -err := vertexai.Init(ctx, "", "") +err := vertexai.Init(ctx, &vertexai.Config{}) if err != nil { log.Fatal(err) } @@ -238,10 +238,12 @@ genkit.DefineFlow( // https://github.com/ledongthuc/pdf func readPDF(path string) (string, error) { f, r, err := pdf.Open(path) + if f != nil { + defer f.Close() + } if err != nil { return "", err } - defer f.Close() reader, err := r.GetPlainText() if err != nil { @@ -272,7 +274,7 @@ which you should not use in production. ```go ctx := context.Background() - err := vertexai.Init(ctx, "", "") + err := vertexai.Init(ctx, &vertexai.Config{}) if err != nil { log.Fatal(err) } @@ -356,6 +358,7 @@ advancedMenuRetriever := ai.DefineRetriever( k := 3 preRerankK := 10 if opts, ok := req.Options.(CustomMenuRetrieverOptions); ok { + k = opts.K preRerankK = opts.PreRerankK } From 44ebcac73c51d6509ea4f7c89e597976aa2fdde0 Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Wed, 10 Jul 2024 14:40:25 -0700 Subject: [PATCH 11/13] typo --- go/internal/doc-snippets/rag/main.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index f107f507cd..3276f9335d 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -59,7 +59,7 @@ func main() { genkit.DefineFlow( "indexMenu", func(ctx context.Context, path string) (any, error) { - // Extract plain text from the PDF. Wrap the logic in Tun so it + // Extract plain text from the PDF. Wrap the logic in Run so it // appears as a step in your traces. pdfText, err := genkit.Run(ctx, "extract", func() (string, error) { return readPDF(path) @@ -68,7 +68,7 @@ func main() { return nil, err } - // Split the text into chunks. Wrap the logic in Tun so it + // Split the text into chunks. Wrap the logic in Run so it // appears as a step in your traces. docs, err := genkit.Run(ctx, "chunk", func() ([]*ai.Document, error) { chunks, err := splitter.SplitText(pdfText) From 8b49d516ca7e18179e690a30ef20131ea9054e31 Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Thu, 11 Jul 2024 08:31:48 -0700 Subject: [PATCH 12/13] feedback --- docs-go/rag.md | 35 ++++++++++++++++++---------- go/internal/doc-snippets/rag/main.go | 32 +++++++++++++++++-------- 2 files changed, 45 insertions(+), 22 deletions(-) diff --git a/docs-go/rag.md b/docs-go/rag.md index c74c902d8f..de56631eb3 100644 --- a/docs-go/rag.md +++ b/docs-go/rag.md @@ -173,6 +173,9 @@ menuPDFIndexer, _, err := localvec.DefineIndexerAndRetriever( Embedder: vertexai.Embedder("text-embedding-004"), }, ) +if err != nil { + log.Fatal(err) +} ``` #### Create chunking config @@ -199,7 +202,7 @@ More chunking options for this library can be found in the genkit.DefineFlow( "indexMenu", func(ctx context.Context, path string) (any, error) { - // Extract plain text from the PDF. Wrap the logic in Tun so it + // Extract plain text from the PDF. Wrap the logic in Run so it // appears as a step in your traces. pdfText, err := genkit.Run(ctx, "extract", func() (string, error) { return readPDF(path) @@ -208,7 +211,7 @@ genkit.DefineFlow( return nil, err } - // Split the text into chunks. Wrap the logic in Tun so it + // Split the text into chunks. Wrap the logic in Run so it // appears as a step in your traces. docs, err := genkit.Run(ctx, "chunk", func() ([]*ai.Document, error) { chunks, err := splitter.SplitText(pdfText) @@ -250,9 +253,11 @@ func readPDF(path string) (string, error) { return "", err } - var buf []byte - _, err = reader.Read(buf) - return string(buf), err + bytes, err := io.ReadAll(reader) + if err != nil { + return "", err + } + return string(bytes), nil } ``` @@ -291,6 +296,9 @@ which you should not use in production. Embedder: vertexai.Embedder("text-embedding-004"), }, ) + if err != nil { + log.Fatal(err) + } genkit.DefineFlow( "menuQA", @@ -355,17 +363,20 @@ advancedMenuRetriever := ai.DefineRetriever( "advancedMenuRetriever", func(ctx context.Context, req *ai.RetrieverRequest) (*ai.RetrieverResponse, error) { // Handle options passed using our custom type. - k := 3 - preRerankK := 10 - if opts, ok := req.Options.(CustomMenuRetrieverOptions); ok { - k = opts.K - preRerankK = opts.PreRerankK + opts, _ := req.Options.(CustomMenuRetrieverOptions) + // Set fields to default values when either the field was undefined + // or when req.Options is not a CustomMenuRetrieverOptions. + if opts.K == 0 { + opts.K = 3 + } + if opts.PreRerankK == 0 { + opts.PreRerankK = 10 } // Call the retriever as in the simple case. response, err := menuPDFRetriever.Retrieve(ctx, &ai.RetrieverRequest{ Document: req.Document, - Options: localvec.RetrieverOptions{K: preRerankK}, + Options: localvec.RetrieverOptions{K: opts.PreRerankK}, }) if err != nil { return nil, err @@ -373,7 +384,7 @@ advancedMenuRetriever := ai.DefineRetriever( // Re-rank the returned documents using your custom function. rerankedDocs := rerank(response.Documents) - response.Documents = rerankedDocs[:k] + response.Documents = rerankedDocs[:opts.K] return response, nil }, diff --git a/go/internal/doc-snippets/rag/main.go b/go/internal/doc-snippets/rag/main.go index 3276f9335d..7b4a155f77 100644 --- a/go/internal/doc-snippets/rag/main.go +++ b/go/internal/doc-snippets/rag/main.go @@ -16,6 +16,7 @@ package rag import ( "context" + "io" "log" "github.com/firebase/genkit/go/ai" @@ -48,6 +49,9 @@ func main() { Embedder: vertexai.Embedder("text-embedding-004"), }, ) + if err != nil { + log.Fatal(err) + } //!-vec //!+splitcfg splitter := textsplitter.NewRecursiveCharacter( @@ -116,9 +120,11 @@ func readPDF(path string) (string, error) { return "", err } - var buf []byte - _, err = reader.Read(buf) - return string(buf), err + bytes, err := io.ReadAll(reader) + if err != nil { + return "", err + } + return string(bytes), nil } //!-readpdf @@ -144,6 +150,9 @@ func menuQA() { Embedder: vertexai.Embedder("text-embedding-004"), }, ) + if err != nil { + log.Fatal(err) + } genkit.DefineFlow( "menuQA", @@ -202,17 +211,20 @@ func customret() { "advancedMenuRetriever", func(ctx context.Context, req *ai.RetrieverRequest) (*ai.RetrieverResponse, error) { // Handle options passed using our custom type. - k := 3 - preRerankK := 10 - if opts, ok := req.Options.(CustomMenuRetrieverOptions); ok { - k = opts.K - preRerankK = opts.PreRerankK + opts, _ := req.Options.(CustomMenuRetrieverOptions) + // Set fields to default values when either the field was undefined + // or when req.Options is not a CustomMenuRetrieverOptions. + if opts.K == 0 { + opts.K = 3 + } + if opts.PreRerankK == 0 { + opts.PreRerankK = 10 } // Call the retriever as in the simple case. response, err := menuPDFRetriever.Retrieve(ctx, &ai.RetrieverRequest{ Document: req.Document, - Options: localvec.RetrieverOptions{K: preRerankK}, + Options: localvec.RetrieverOptions{K: opts.PreRerankK}, }) if err != nil { return nil, err @@ -220,7 +232,7 @@ func customret() { // Re-rank the returned documents using your custom function. rerankedDocs := rerank(response.Documents) - response.Documents = rerankedDocs[:k] + response.Documents = rerankedDocs[:opts.K] return response, nil }, From ba34b3cc1708c5acee16903ade597f974c58d91c Mon Sep 17 00:00:00 2001 From: Kevin Cheung Date: Thu, 11 Jul 2024 16:02:58 -0700 Subject: [PATCH 13/13] no firestore --- docs-go/rag.md | 1 - docs-go/rag.src | 1 - 2 files changed, 2 deletions(-) diff --git a/docs-go/rag.md b/docs-go/rag.md index de56631eb3..beb0aa65c7 100644 --- a/docs-go/rag.md +++ b/docs-go/rag.md @@ -101,7 +101,6 @@ create your own. Genkit provides indexer and retriever support through its plugin system. The following plugins are officially supported: -- [Cloud Firestore vector store](plugins/firebase.md) - [Pinecone](plugins/pinecone.md) cloud vector database In addition, Genkit supports the following vector stores through predefined diff --git a/docs-go/rag.src b/docs-go/rag.src index 645af068c1..3f5a963aa7 100644 --- a/docs-go/rag.src +++ b/docs-go/rag.src @@ -99,7 +99,6 @@ create your own. Genkit provides indexer and retriever support through its plugin system. The following plugins are officially supported: -- [Cloud Firestore vector store](plugins/firebase.md) - [Pinecone](plugins/pinecone.md) cloud vector database In addition, Genkit supports the following vector stores through predefined