diff --git a/.trunk/.gitignore b/.trunk/.gitignore index 15966d08..2c890198 100644 --- a/.trunk/.gitignore +++ b/.trunk/.gitignore @@ -7,3 +7,4 @@ plugins user_trunk.yaml user.yaml tmp +.DS_Store diff --git a/.vale.ini b/.vale.ini index 221d3c76..971fcda0 100644 --- a/.vale.ini +++ b/.vale.ini @@ -13,4 +13,4 @@ mdx = md BasedOnStyles = Vale, Google Google.Exclamation = OFF Google.Parens = OFF -Google.We = OFF \ No newline at end of file +Google.We = OFF diff --git a/getting-started-with-hyper-commerce.mdx b/getting-started-with-hyper-commerce.mdx new file mode 100644 index 00000000..2d6c4340 --- /dev/null +++ b/getting-started-with-hyper-commerce.mdx @@ -0,0 +1,274 @@ +--- +title: "Quickstart: Hyper Commerce" +description: "Learn how to instantly get started with the Hypermode Commerce template" +--- + +This guide walks you through getting started instantly with Hypermode using the [Instant Vector Search template](https://github.com/hypermodeAI/hyper-commerce). You’ll also learn how to customize your functions to tailor the app to your needs. + +## What's Hypermode? + +[Hypermode](/introduction) is framework designed to simplify the development and deployment of AI features and assistants. With its [Functions SDK](/sdk/functions-sdk) and runtime, developers can quickly deploy intelligent features into production without the need for extensive data wrangling. Hypermode supports rapid iteration, allowing you to start with a large language model and later fine-tune smaller, open source models. Its production-ready GraphQL API auto-generates and enables seamless integration. Hypermode is ideal for developers looking to quickly prototype and evolve AI capabilities. + +## Quickstart + +You can get started with Hypermode’s [Instant Vector Search](https://github.com/hypermodeAI/hyper-commerce) template using either sample data or your own, without needing to write any code or set up a GitHub repository. This lets you explore the template and Hypermode’s features before committing any time and effort. + +### Step 1: Creating a new project + +1. Visit the [Hypermode website](https://hypermode.com/sign-in) and sign up with your GitHub, Google, or email address. Once signed in, you'll see the Hypermode projects dashboard. +2. After accessing your dashboard, click the “Create New Project” button + + + The Hypermode projects dashboard. + + +3. Select “Deploy Instant Vector Search” from the options. + + + Selecting the 'Deploy instant vector search' template in the dropdown when creating a new project. + + +4. Choose between using sample data or your own data: + + + Selecting to use sample data when creating the new hypermode project. + + + > If you choose to use your own data, you'll need to upload it using a CSV file and a Python script provided in Step 2. + +5. Once created, Hypermode provisions a runtime with instant connectivity to shared AI models that bring the functions within your project to life. The source code for the project is open source, available in [this repository](https://github.com/hypermodeAI/hyper-commerce). + +Behind the scenes, Hypermode automatically exposes the functions from the backend directory as GraphQL APIs, allowing you to interact with them like any other GraphQL endpoint. + +### Step 2: Adding your data + +If you selected sample data, your project is fully setup. You can move on to Step 3 and immediately start exploring the project and its features. If you chose to use your own data, follow these steps to seed your data into the collections: + +1. Ensure you have [Python](https://www.python.org/downloads/) installed on your machine. You can verify this by running `python --version` or `python3 --version` in your terminal. +2. Prepare your CSV file with the following headers: `Unique Id,Product Name,Category,Selling Price,About Product,Image,Num Stars,In Stock`. +3. Copy the script from [this file](https://www.notion.so/Getting-Started-with-Hypermode-1fba345ba60c4ecbade5663a2251eedc?pvs=21). +4. Update the script with your endpoint (located in your projects dashboard), API key (located in the settings tab of your Hypermode console), and the path to your CSV file. +5. Install the necessary dependencies: + + ```bash + pip install requests gql requests_toolbelt pandas numpy + ``` + +6. Run the script to populate your data: + + ```bash + python3 ecommerce_populate.py + ``` + +### Step 3: Exploring the Console + +In the Hypermode console, you’ll see several key components of your project: + + + The hypermode project dashboard. + + +- **[Functions](/sdk/functions-sdk):** These are serverless functions written in AssemblyScript (a TypeScript-like language) that are automatically exposed as GraphQL APIs. Once deployed, you can query these functions within your app. +- **[Collections](/sdk/collections):** Hypermode offers built-in key-value storage, known as collections, with support for vector embeddings. This allows you to store and retrieve data efficiently, without requiring a database for basic use cases. +- **[Models](/sdk/models):** This section represents the AI models defined for your project. These models handle tasks like embedding text into vectors for search. Hypermode provides open source shared and dedicated model hosting for rapid experimentation. You can also connect to your preferred large language model, including OpenAI, Anthropic, and Gemini. +- **[Hosts](/define-hosts#define-hosts):** You define all external connections from your functions as hosts, with the runtime denying all other egress for secure-by-default processing. +- **Endpoint:** The GraphQL endpoint for your project, which you’ll use to interact with your APIs and query your data. + +### Step 4: Querying your data + +Now that you set up, deployed, and seeded your project with data, you can test your functions using the query interface in the Hypermode console. + +1. Navigate to the Query tab in your Hypermode console to test your data. +2. Paste the following query to retrieve sample product data: + + ```graphql + query { + searchProducts(maxItems: 4, query: "sparkly shoes") { + searchObjs { + product { + name + description + id + stars + isStocked + } + } + } + } + ``` + +3. You should see the data for 4 items returned from the `searchProducts` endpoint that match your query, `sparkly shoes`. Feel free to experiment with the query—adjust the `maxItems` value to return more items, or change the search query to see how the returned data matches your input. Additionally, notice that the function ranks items based on their star rating and whether they're in stock. + + + The query interface. + + +### Step 5: Testing in the Frontend + +Now that you've set up your project and queried your data in the console, you can test the capability in a frontend UI. + +1. Clone the repository that contains a pre-built UI for testing. You can find the repo [here](https://github.com/hypermodeAI/hyper-commerce/tree/main/frontend). +2. Retrieve your API key from the Settings section of your Hypermode console. +3. Create a `.env.local` file in the root of your project and add your API key and endpoint to it, like this: + + ```bash + HYPERMODE_API_TOKEN=YOUR_API_TOKEN + HYPERMODE_API_ENDPOINT=YOUR_API_ENDPOINT + ``` + +4. **Run the project locally** by executing the following command: + + ```bash + npm run dev + ``` + +5. With the project running locally, you can now test the search capability in the provided UI. Try searching for products to see how your Hypermode project's API integrates and returns data. + +> Note: The intent of this quickstart is for proof of concepts. For more advanced usage, such as customizing search re-ranking logic, you'll need to clone the template to your own repository to make and push changes. Refer to the next section for further instructions. + +--- + +## Customizing the app + +In this section, you’ll learn how to tailor the template to fit your specific needs. We’ll show you how to edit your backend functions and deploy those changes to Hypermode. + +### Step 1: Clone the template repository + +1. Go to the template repo [hyper-commerce](https://github.com/hypermodeAI/hyper-commerce). +2. Clone the repo by clicking `Use this template` in the upper-right corner and selecting `Create new repo.` This clones the code into your own GitHub repository +3. Visit the [Hypermode website](https://hypermode.com/sign-in) and sign up with your GitHub, Google, or email address. Once signed in, you'll see your Hypermode projects dashboard. +4. In the Hypermode console, click `New Project`. +5. Select `Import a GitHub Repo`. + + + Selecting 'import a GitHub repo' from the dropdown. + + +6. Choose the repo you just created. + + + Selecting the repo we want to import. + + +7. Click “Deploy” to finish creating your project. +8. Once deployed, your functions and collections are visible in the Hypermode console. + +### Step 2: Seeding your data + +1. Make your first commit to the repo to trigger a deployment. +2. Ensure you have [Python](https://www.python.org/downloads/) installed on your machine. You can verify this by running `python --version` or `python3 --version` in your terminal. +3. The template includes a script at `/backend/extras/ecommerce_populate.py` to seed your data, as well as sample data located in `/backend/extras/hyper_toys.csv`. +4. If you want to use your own data, replace the content of the sample CSV (`hyper_toys.csv`) with your own data. Make sure the headers in your CSV match the following headers: `Uniq Id,Product Name,Category,Selling Price,About Product,Image,Num Stars,In Stock` +5. Install the required dependencies by running the following command in your project directory: + + ```bash + pip install -r requirements.txt + ``` + +6. Edit the `ecommerce_populate.py` file to include your endpoint and API key, which you can find in your Hypermode dashboard. +7. Run the script to seed the data into your project: + + ```bash + python3 ecommerce_populate.py + ``` + +8. The script batch inserts the data and displays the time taken for each operation. Inserting the full dataset (10,000 rows) may take around 18 minutes. If you want to test with a smaller dataset, feel free to reduce the size of the CSV. + +### Step 3: Customizing your functions + +You can modify the template to suit your needs by customizing the functions in the `/backend/functions/assembly` directory. + +#### Example: Customizing product rankings + +If you'd like to rank products based solely on their star rating, without considering whether they're in stock, follow these steps: + +1. Go to the `search.ts` file and locate the `reRankAndFilterSearchResultObjects` function. +2. Modify the function to only rank based on the star rating, like this: + +```tsx +function reRankAndFilterSearchResultObjects( + objs: collections.CollectionSearchResultObject[], + thresholdStars: f32, +): collections.CollectionSearchResultObject[] { + for (let i = 0; i < objs.length; i++) { + const starRes = collections.getText( + consts.productStarCollection, + objs[i].key, + ); + const stars = parseFloat(starRes); + + objs[i].score *= stars * 0.1; + } + + objs.sort((a, b) => (a.score < b.score ? -1 : a.score > b.score ? 1 : 0)); + + const filteredResults: collections.CollectionSearchResultObject[] = []; + for (let i = 0; i < objs.length; i++) { + const starRes = collections.getText( + consts.productStarCollection, + objs[i].key, + ); + const stars = parseFloat(starRes); + if (stars >= thresholdStars) { + filteredResults.push(objs[i]); + } + } + + return filteredResults; +} +``` + +#### Deploying the change + +1. Once you’ve updated the function, commit the changes to your repo. +2. Any commit to the `main` branch automatically triggers a Hypermode deployment. +3. After deployment, when you query the `searchProducts` endpoint again, it ranks products solely based on their star rating. + +### Step 4: Testing your functions + +Once you’ve made changes to your backend functions and deployed them to Hypermode, it's time to test the updates. + +#### Test in the console IDE + +In the Hypermode console, navigate to the Query tab to test your modified functions directly. Run queries similar to the ones you used earlier to see how the changes impact the results. + +#### Run the frontend locally + +The repo you cloned includes a frontend. Move into the frontend directory and add the following values to your `.env.local` file: + +```jsx +HYPERMODE_API_TOKEN = YOUR_API_TOKEN; +HYPERMODE_API_ENDPOINT = YOUR_API_ENDPOINT; +``` + +> Note: Both of these values are available in the Hypermode console + +Next, just run the command `npm run dev` in your terminal to run the app locally. Now you can test the changes you made to your backend functions. diff --git a/images/getting-started-guide/create-project.png b/images/getting-started-guide/create-project.png new file mode 100644 index 00000000..1bfe8f85 Binary files /dev/null and b/images/getting-started-guide/create-project.png differ diff --git a/images/getting-started-guide/deploy-template-dropdown.png b/images/getting-started-guide/deploy-template-dropdown.png new file mode 100644 index 00000000..f21b7d88 Binary files /dev/null and b/images/getting-started-guide/deploy-template-dropdown.png differ diff --git a/images/getting-started-guide/import-github-repo.png b/images/getting-started-guide/import-github-repo.png new file mode 100644 index 00000000..2cffcf55 Binary files /dev/null and b/images/getting-started-guide/import-github-repo.png differ diff --git a/images/getting-started-guide/project-dash.png b/images/getting-started-guide/project-dash.png new file mode 100644 index 00000000..f36712d3 Binary files /dev/null and b/images/getting-started-guide/project-dash.png differ diff --git a/images/getting-started-guide/query-interface.png b/images/getting-started-guide/query-interface.png new file mode 100644 index 00000000..f62a7a40 Binary files /dev/null and b/images/getting-started-guide/query-interface.png differ diff --git a/images/getting-started-guide/sample-data-select.png b/images/getting-started-guide/sample-data-select.png new file mode 100644 index 00000000..fd96225f Binary files /dev/null and b/images/getting-started-guide/sample-data-select.png differ diff --git a/images/getting-started-guide/select-repo.png b/images/getting-started-guide/select-repo.png new file mode 100644 index 00000000..42edb24d Binary files /dev/null and b/images/getting-started-guide/select-repo.png differ diff --git a/mint.json b/mint.json index aaf46e03..552ebc43 100644 --- a/mint.json +++ b/mint.json @@ -54,7 +54,11 @@ "navigation": [ { "group": "Get Started", - "pages": ["introduction", "quickstart", "worlds-fair-workshop"] + "pages": [ + "introduction", + "quickstart", + "getting-started-with-hyper-commerce" + ] }, { "group": "Configure Project", diff --git a/quickstart.mdx b/quickstart.mdx index f79ed73f..b53cdcb9 100644 --- a/quickstart.mdx +++ b/quickstart.mdx @@ -1,5 +1,5 @@ --- -title: Quickstart +title: "Quickstart: Base Template" description: "Start building AI features in under five minutes" mode: "wide" --- diff --git a/styles/config/vocabularies/general/accept.txt b/styles/config/vocabularies/general/accept.txt index 68a7354d..70b7f50e 100644 --- a/styles/config/vocabularies/general/accept.txt +++ b/styles/config/vocabularies/general/accept.txt @@ -29,4 +29,5 @@ triaging upsert UUID nnClassify +serverless getVector diff --git a/worlds-fair-workshop.mdx b/worlds-fair-workshop.mdx deleted file mode 100644 index b9d08868..00000000 --- a/worlds-fair-workshop.mdx +++ /dev/null @@ -1,745 +0,0 @@ ---- -title: AI Engineer World's Fair Workshop -description: "git push, get AI API" ---- - -In this workshop you'll build a demo that augments an app with AI for classification, -natural language search, and summarization. Then you'll learn how to iteratively improve -before and after shipping to prod. We'll use Hypermode to move fast and reduce infra overhead. -You'll walk away with a greater intuition for when & how to integrate AI without refactoring. - -If you need help at any point, please don't hesitate to ask. - -## Prerequisites - -To get the most out of this workshop, you'll need: - -- A GitHub account where you can create a new repo -- [Node.js 20](https://nodejs.org/) or higher installed and activated. -- An IDE or text editor such as [VS Code](https://code.visualstudio.com/) - -You can install Node.js with any supported method, including: - -- Downloading the official [prebuilt installer](https://nodejs.org/en/download) -- Using a [package manager](https://nodejs.org/en/download/package-manager) - - - [Join the - conversation](https://ai-engineer-workspace.slack.com/archives/C079FEPGY69) on - the AI Engineer Slack. ([Slack - Invite](https://join.slack.com/t/ai-engineer-workspace/shared_invite/zt-2lcyging7-nb9owJrFthJb0dbfdt3cpw)) - - -## Agenda - -We'll spend our time together on the following: - -{/* */} - -1. Playing a familiar game, using AI to scale the format -2. Understanding how the game is constructed, using both large and small models -3. Applying the same building blocks to an app triaging GitHub issues -4. Generalizing the concepts for adding AI to your apps - -{/* */} - -## Hypercategories - -We're going to first play a game called Hypercategories, a scaled-variant of the game -[Scattergories](https://en.wikipedia.org/wiki/Scattergories). In Scattergories, players -receive a list of categories and a letter. They have to come up with a unique word that -fits each category and starts with the given letter. - -In Hypercategories, for each submission that matches the category and letter, you'll -receive a point. If other submissions match yours, you'll split the point. - -### Let's play - -When the game starts, scan the QR code or go to https://tinyurl.com/4mdhx29j - -### AI-powered filtering and scoring - -## GitHub issue triage - -We're going to apply what we learned from Hypercategories to a real-world problem: -triaging GitHub issues. We'll build a tool that helps you quickly understand trends, -classify, and identify similar issues in their repos. - -### Set up your project - -Hypermode makes it easy to spin up new projects for experimentation. - -- Go to [justship.ai](https://justship.ai) and select `GitHub Issue Triage` -- Click `Deploy` to start the onboarding flow -- Authenticate your account with GitHub and edit the repo name as needed -- Click `Create` to deploy your project - -While your project is spinning up, personalize your workspace. - -### Set up local environment - -- Clone the newly created GitHub repository locally, using any method you prefer. - - For example, you can use the GitHub CLI: - - ```sh - gh repo clone - ``` - - Or the Git CLI: - - ```sh - git clone - ``` - - Or you can use a graphical client like [GitHub Desktop](https://desktop.github.com/). - -- Open the local repository in your IDE or text editor. - -- Using a terminal window (preferably from within your IDE), open the `functions` directory, - install dependencies, and build the project: - - ```sh - cd functions - npm install - npm run build - ``` - - - -```none -Building issue-triage@1.0.0 in debug mode... - - __ __ __ - / // /_ _____ ___ ______ _ ___ ___/ /__ - / _ / // / _ \/ -_) __/ ' \/ _ \/ _ / -_) - /_//_/\_, / .__/\__/_/ /_/_/_/\___/\_,_/\__/ - /___/_/ - -Plugin Metadata: - Plugin Name: issue-triage@1.0.0 - Library: @hypermode/functions-as@0.9.1 - Build ID: cptr9kv9lp6m35581s7g - Build Timestamp: 2024-06-26T06:27:31.320Z - Git Repo: https://github.com/hypermodeAI/ship-issue-triage - Git Commit: 556a604357827e0e135cd530432ec5dd7d21489a - -Hypermode Functions: - classifyIssue(id: string, title: string, description: string): string - trendSummary(owner: string, repo: string): string -``` - - - -#### What just happened? - -You compiled a Hypermode project written in [AssemblyScript](https://www.assemblyscript.org/), -a TypeScript-like language that compiles to WebAssembly for fast, secure, and efficient execution. - -A successful compilation indicates that the resulting project is ready to deploy on Hypermode. -The Hypermode Functions listed in the output automatically become part of the project's GraphQL API. - -In this case, you can see two functions: - -- `classifyIssue` that takes an issue ID, title, and description and returns a classification label. -- `trendSummary` that takes a GitHub owner and repo and returns a summary of trends for that repo. - -Since you created the project from a template, this version of the code is already deployed. -When you make any changes, simply commit and push back to the GitHub repository. -The changes deploy automatically using the GitHub Actions workflow set up in the project. - -### Review and modify the code - -For this next step of the workshop, we'll review the code of the Hypermode Functions -in the project and make some small modifications. - -We'll start this together, as a group. Please follow along in your own project. - - - If using VS Code, you may be seeing files and code underlined in red at this stage, - even though the project built successfully. This is because the TypeScript language services - aren't yet aware of the AssemblyScript dependency. - -You can fix this by forcing a reset of the TypeScript language services, in any of these ways: - -- You can close and reopen the project in VS Code. -- You can open the VS Code command pallette (`Cmd+Shift+P` on Mac, `Ctrl+Shift+P` on Windows) - and run the command `TypeScript: Restart TS server`. -- You can modify the `assembly/tsconfig.json` file by adding a comment or whitespace and saving it. -- On Linux or Mac, you can open a terminal and run `touch assembly/tsconfig.json`. - - - -### Trend summarization - -Have a look at the function `trendSummary` in [functions/assembly/trends.ts](https://github.com/hypermodeAI/ship-issue-triage/blob/main/functions/assembly/trends.ts) - -The purpose of this function is to summarize the trends of the issues in a GitHub repository. - -The function performs the following steps: - -- Fetches the issues from the repository, using the GitHub REST API. -- Concatenates several pieces of information from each issue into a single string. -- Sends that string to an OpenAI model to generate a summary. -- Returns the summary as the output of the function. - - - -```ts -export function trendSummary(owner: string, repo: string): string { - const issues = getGithubIssues(owner, repo); - - const summary = issues - .map( - (issue) => - `${issue.createdAt} ${issue.user ? "From " + issue.user!.login : ""} : ${issue.title}`, - ) - .join("\n"); - - const model = models.getModel("text-generator"); - const instruction = `Provide a summary of the trends in the repository based on the issues created.`; - - const input = model.createInput([ - new SystemMessage(instruction), - new UserMessage(summary), - ]); - - input.temperature = 0.7; - - const output = model.invoke(input); - - return output.choices[0].message.content.trim(); -} -``` - - - - - -```graphql -query GetTrendSummary { - trendSummary(owner: "huggingface", repo: "transformers") -} -``` - - - -### Issue type classification - -Next let's examine the `classifyIssue` function in [functions/assembly/classify.ts](https://github.com/hypermodeAI/ship-issue-triage/blob/main/functions/assembly/classify.ts) - -The purpose of this function is to classify an issue based on its title and description. - -The function performs the following steps: - -- Concatenates the title and description of the issue into a single string. -- Sends that string to a purpose-built classification model to classify the issue. -- Returns the classification label as the output of the function. - -Along the way, it also logs some information to Hypermode that can be useful -for debugging and monitoring the function's behavior. - - - -```ts -export function classifyIssue( - id: string, - title: string, - description: string, -): string { - console.log(`Classifying issue ${id}`); - const summary = `${title}\n${description}`; - - const model = models.getModel("issue-classifier"); - const input = model.createInput([summary]); - const output = model.invoke(input).predictions[0]; - - console.log(`Issue ${id} classified as ${output.label}`); - return output.label; -} -``` - - - - - -````graphql -query ClassifyIssueText { - classifyIssue( - id: "31622" - title: "Unable to export Phi-3-vision model to PyTorch exported program" - description: "### System Info\n\n- `transformers` version: 4.41.2 -\n- Platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35 -\n- Python version: 3.10.14 -\n- Huggingface_hub version: 0.23.0 -\n- Safetensors version: 0.4.3 -\n- Accelerate version: 0.30.1 -\n- Accelerate config: not found -\n- PyTorch version (GPU?): 2.4.0.dev20240412+cu121 (True) -\n- Tensorflow version (GPU?): not installed (NA) -\n- Flax version (CPU?/GPU?/TPU?): not installed (NA) -\n- Jax version: not installed -\n- JaxLib version: not installed -\n- Using GPU in script?: yes -\n- Using distributed or parallel set-up in script?: no\n\n### Who can help?\n\n@amyeroberts\n\n### Information\n\n- [ ] The official example scripts\n- [X] My own modified scripts\n\n### Tasks\n\n- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)\n- [X] My own task or dataset (give details below)\n\n### Reproduction\n\nI'm trying to export the Phi-3-vision model to PyTorch exported program. -\n -\nRepro: -\n```python -\nimport requests -\nimport torch -\nfrom PIL import Image -\nfrom transformers import AutoModelForCausalLM, AutoProcessor -\nfrom ml_dtypes import bfloat16 -\nimport numpy as np -\n -\n -\nmodel_id = \"microsoft/Phi-3-vision-128k-instruct\" -\nprocessor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) -\nmodel = AutoModelForCausalLM.from_pretrained( -\n model_id, trust_remote_code=True, torch_dtype=\"auto\" -\n).to('cuda') -\n -\nuser_prompt = \"<|user|>\\n\" -\nassistant_prompt = \"<|assistant|>\\n\" -\nprompt_suffix = \"<|end|>\\n\" -\n -\n# single-image prompt -\nprompt = f\"{user_prompt}<|image_1|>\\nWhat is shown in this image?{prompt_suffix}{assistant_prompt}\" -\nurl = \"https://www.ilankelman.org/stopsigns/australia.jpg\" -\nprint(f\">>> Prompt\\n{prompt}\") -\n -\nimage = Image.open(requests.get(url, stream=True).raw) -\ninputs = processor(prompt, image, return_tensors=\"pt\").to(\"cuda:0\") -\n -\n# Initialize -\n# inputs.keys: dict_keys(['input_ids', 'pixel_values', 'image_sizes']) -\ninput_ids = inputs[\"input_ids\"] -\nstart_point = input_ids.shape[1] -\npixel_values = inputs[\"pixel_values\"] -\nimage_sizes = inputs[\"image_sizes\"] -\ninputs.pop(\"attention_mask\") -\n -\nwith torch.no_grad(): -\n # max=1024 has contraint violation error. https://github.com/pytorch/pytorch/issues/125604 -\n seq_len = torch.export.Dim(\"seq_len\", min=1, max=4096) -\n kwargs = {\"input_ids\": input_ids, \"pixel_values\": pixel_values, \"image_sizes\": image_sizes} -\n ep = torch.export.export( -\n model, -\n args=tuple(), -\n kwargs=kwargs, -\n dynamic_shapes=({1: seq_len}, {}, {}), -\n strict=False, -\n ) -\n``` -\n -\nError message: -\n -\n``` -\nTraceback (most recent call last): -\n File \"/home/zewenl/Documents/pytorch/TensorRT/examples/dynamo/phi3.py\", line 39, in -\n ep = torch.export.export( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/__init__.py\", line 174, in export -\n return _export( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/_trace.py\", line 900, in wrapper -\n raise e -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/_trace.py\", line 883, in wrapper -\n ep = fn(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/exported_program.py\", line 85, in wrapper -\n return fn(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/_trace.py\", line 1062, in _export -\n ep_non_strict = _export_non_strict( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/_trace.py\", line 583, in _export_non_strict -\n gm, graph_signature = transform(aot_export_module)( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/_trace.py\", line 1023, in _aot_export_non_strict -\n gm, sig = aot_export(wrapped_mod, args, kwargs=kwargs, **flags) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py\", line 1059, in aot_export_module -\n fx_g, metadata, in_spec, out_spec = _aot_export_function( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py\", line 1249, in _aot_export_function -\n fx_g, meta = create_aot_dispatcher_function( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_dynamo/utils.py\", line 265, in time_wrapper -\n r = func(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py\", line 549, in create_aot_dispatcher_function -\n fw_metadata = run_functionalized_fw_and_collect_metadata( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/collect_metadata_analysis.py\", line 150, in inner -\n flat_f_outs = f(*flat_f_args) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/utils.py\", line 174, in flat_fn -\n tree_out = fn(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/traced_function_transforms.py\", line 709, in functional_call -\n out = mod(*args[params_len:], **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1532, in _wrapped_call_impl -\n return self._call_impl(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1541, in _call_impl -\n return forward_call(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/export/_trace.py\", line 1010, in forward -\n tree_out = self._export_root(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1532, in _wrapped_call_impl -\n return self._call_impl(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1541, in _call_impl -\n return forward_call(*args, **kwargs) -\n File \"/home/zewenl/.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-vision-128k-instruct/7b92b8c62807f5a98a9fa47cdfd4144f11fbd112/modeling_phi3_v.py\", line 1301, in forward -\n outputs = self.model( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1532, in _wrapped_call_impl -\n return self._call_impl(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1541, in _call_impl -\n return forward_call(*args, **kwargs) -\n File \"/home/zewenl/.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-vision-128k-instruct/7b92b8c62807f5a98a9fa47cdfd4144f11fbd112/modeling_phi3_v.py\", line 1129, in forward -\n inputs_embeds = self.vision_embed_tokens(input_ids, pixel_values=pixel_values, image_sizes=image_sizes) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1532, in _wrapped_call_impl -\n return self._call_impl(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/nn/modules/module.py\", line 1541, in _call_impl -\n return forward_call(*args, **kwargs) -\n File \"/home/zewenl/.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-vision-128k-instruct/7b92b8c62807f5a98a9fa47cdfd4144f11fbd112/image_embedding_phi3_v.py\", line 170, in forward -\n if len(positions.tolist()) > 0: -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py\", line 219, in tolist -\n return [elem.tolist() for elem in self.elem] -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_tensor.py\", line 1066, in __iter__ -\n return iter(self.unbind(0)) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py\", line 421, in __torch_dispatch__ -\n outs_unwrapped = func._op_dk( -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/utils/_stats.py\", line 20, in wrapper -\n return fn(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py\", line 842, in __torch_dispatch__ -\n return self.dispatch(func, types, args, kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py\", line 1187, in dispatch -\n return self._cached_dispatch_impl(func, types, args, kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py\", line 920, in _cached_dispatch_impl -\n output = self._dispatch_impl(func, types, args, kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py\", line 1339, in _dispatch_impl -\n return decomposition_table[func](*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_refs/__init__.py\", line 3906, in unbind -\n if t.shape[dim] == 0: -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/__init__.py\", line 377, in __bool__ -\n return self.node.bool_() -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/fx/experimental/sym_node.py\", line 439, in bool_ -\n return self.guard_bool(\"\", 0) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/fx/experimental/sym_node.py\", line 377, in guard_bool -\n r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/fx/experimental/recording.py\", line 265, in wrapper -\n return event.run(self) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/fx/experimental/recording.py\", line 160, in run -\n return self.f(*args, **kwargs) -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py\", line 4269, in evaluate_expr -\n raise self._make_data_dependent_error( -\ntorch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 0) (unhinted: Eq(u0, 0)). (Size-like symbols: u0) -\n -\nATTENTION: guard_size_oblivious would fix the error, evaluating expression to False. -\nMaybe you need to add guard_size_oblivious to framework code, see doc below for more guidance. -\n -\nPotential framework code culprit (scroll up for full backtrace): -\n File \"/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_refs/__init__.py\", line 3906, in unbind -\n if t.shape[dim] == 0: -\n -\nFor more information, run with TORCH_LOGS=\"dynamic\" -\nFor extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL=\"u0\" -\nIf you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 -\nFor more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing -\n -\nFor C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 -\n``` -\n -\nIt seems the error is due to: -\n``` -\nFile \"/home/zewenl/.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-vision-128k-instruct/7b92b8c62807f5a98a9fa47cdfd4144f11fbd112/image_embedding_phi3_v.py\", line 170, in forward -\n if len(positions.tolist()) > 0: -\n```\n\n### Expected behavior\n\nThe code should be able to run correctly. -\nI'm not sure the issue is from huggingface or pytorch. I submitted a issue to pytorch as well [here](https://github.com/pytorch/pytorch/issues/128906) for your reference." - ) -} -```` - - - -#### Modify the function - -As an exercise, let's modify the `classifyIssue` function to only return labels -that have a confidence score of a certain threshold or higher. - -See if you can figure out how to do this. If you get stuck, ask for help, -or view the solution below. - - - -```ts -export function classifyIssue( - id: string, - title: string, - description: string, -): string { - console.log(`Classifying issue ${id}`); - const summary = `${title}\n${description}`; - - const model = models.getModel("issue-classifier"); - const input = model.createInput([summary]); - const output = model.invoke(input).predictions[0]; - - // This is the new code that checks the confidence score. - if (output.confidence < 0.6) { - console.log( - `Issue ${id} couldn't be classified. Confidence: ${output.confidence}`, - ); - return "unknown"; - } - - console.log(`Issue ${id} classified as ${output.label}`); - return output.label; -} -``` - - - -#### Deploy the changes for classification - -Commit and push the changes to the GitHub repository, using any method you prefer. - -For example, using the Git CLI: - -```sh -git add . -git commit -m "Add threshold to classification" -git push -``` - -#### Test the classification function - -Using a web browser, sign in to hypermode.com and navigate to your project. -You can test the `classifyIssue` function with sample inputs using the GraphiQL -interface provided by the Hypermode UI. - -### Issue similarity search - -In order to find similar issues, we need to: - -- Define a collection to store issues -- Create a function that knows how to compute vector embeddings -- Create a Hypermode function that uses the collection to find similar issues -- Deploy the code changes -- Insert issues data into the collection, and test the function - -#### Define a collection - -Open the `hypermode.json` (in the root of the repo directory) and add the following section: - -```json - "collections": { - "issuesCollection": { - "searchMethods": { - "byTitle": { - "embedder": "minilmEmbedder" - } - } - } - } -``` - -This tells Hypermode to create a collection named `issuesCollection`, -and to search by title using an embedder function named `minilmEmbedder`. - -In order to compute vector embeddings, we also need to declare another model -in the `hypermode.json` file. - -Add the following to the existing `"models"` section: - -```json - "minilm": { - "sourceModel": "sentence-transformers/all-MiniLM-L6-v2", - "host": "hypermode", - "provider": "hugging-face" - } -``` - - - -The complete `hypermode.json` manifest file should now be as follows: - -```json -{ - "$schema": "https://manifest.hypermode.com/hypermode.json", - - "models": { - "issue-classifier": { - "sourceModel": "AntoineMC/distilbart-mnli-github-issues", - "provider": "hugging-face", - "host": "hypermode" - }, - "text-generator": { - "sourceModel": "gpt-3.5-turbo", - "host": "openai", - "path": "v1/chat/completions" - }, - "minilm": { - "sourceModel": "sentence-transformers/all-MiniLM-L6-v2", - "host": "hypermode", - "provider": "hugging-face" - } - }, - - "hosts": { - "github": { - "baseUrl": "https://api.github.com/", - "headers": { - "Authorization": "Bearer {{AUTH_TOKEN}}" - } - }, - "openai": { - "baseUrl": "https://api.openai.com/", - "headers": { - "Authorization": "Bearer {{API_KEY}}" - } - } - }, - - "collections": { - "issuesCollection": { - "searchMethods": { - "byTitle": { - "embedder": "minilmEmbedder" - } - } - } - } -} -``` - - - -#### Create the embedder function - -Create a new file `embedder.ts` in the `functions/assembly` folder. - -This function uses the MiniLM model to create vector embeddings for the input text. - -```ts -import { models } from "@hypermode/functions-as"; -import { EmbeddingsModel } from "@hypermode/models-as/models/experimental/embeddings"; - -export function minilmEmbedder(text: string[]): f32[][] { - const model = models.getModel("minilm"); - const input = model.createInput(text); - const output = model.invoke(input); - - return output.predictions; -} -``` - -You'll also need to modify the `functions/assembly/index.ts` file to export the new function. - -Add the following line to the end of the file: - -```ts -export * from "./embedder"; -``` - -#### Create the search function - -Create a new file `similar.ts` in the `functions/assembly` folder. - -This function uses the collection to search for similar issues based on the title. - -```ts -import { collections } from "@hypermode/functions-as"; - -// Define the structure we expect for the output of the similarity search function. -@json -class SimilarIssue { - id!: string; - title!: string; - similarity!: f64; -} - -export function similarIssues(title: string): SimilarIssue[] { - const response = collections.search( - "issuesCollection", - "byTitle", - title, // the text to search for - 3, // return the top 3 results - true, // include text in the results - ); - - return response.objects.map( - (o) => - { - id: o.key, - title: o.text, - similarity: o.score, - }, - ); -} -``` - -You'll also need to modify the `functions/assembly/index.ts` file to export the new function. - -Add the following line to the end of the file: - -```ts -export * from "./similar"; -``` - -#### Deploy the changes for similarity search - -Commit and push the changes to the GitHub repository, using any method you prefer. - -For example, using the Git CLI: - -```sh -git add . -git commit -m "Add similarity search" -git push -``` - -#### Insert issues data into the collection - -- Download an [example CSV data file](https://github.com/hypermodeAI/ship-issue-triage/blob/final/extras/issues.csv), - and save it to your local machine. - -- Then, in the Hypermode UI, navigate to your project and find the collection named `issuesCollection`. - -- Click the `Upload` link and upload a CSV file containing the issues data. - -The import process starts, and should complete within a few seconds, adding all your issues -to the collection. - -Alternatively, you can add the data programmatically. -For example, you can create a new function that inserts data directly into -the collection. - -```ts -export function addIssue(id: string, title: string): string { - const result = collections.upsert("issuesCollection", id, title); - return result.status; -} -``` - -#### Test the similarity function - -Like before, you can navigate to your project at hypermode.com and test the `similarIssues` -function with sample inputs using the GraphiQL interface provided by the Hypermode UI. - - - -```graphql -query FindSimilarIssues { - similarIssues(title: "Add Spanish Readme") { - id - title - similarity - } -} -``` - - - -### Production-ready API - -As you've seen, Hypermode makes it easy to build and deploy AI-powered functions. - -When you're ready to take the next step, you can start incorporating these functions -into your own applications. Hypermode provides a GraphQL API that you can use to -interact with your functions. You can obtain an API key from the settings page -in the Hypermode UI, then use any GraphQL tool or client library you wish. - -## What's next - -You can apply the building blocks of filtering, summarization, categorization, and search -into a wide variety of applications. Hypermode makes it easy to iterate on your AI features. - -For further inspiration, explore [justship.ai](https://justship.ai) and deploy today!