Release 1.5.7 (#159)

* Pagination bug * Bug fix * chore: add docker cmd * Compatibility fixes for SDK version 2.0.0 (#69) * Pagination bug * Bug fix * Fix for schema changes * Render tool calling * Support for Langgraph, Qdrant & Groq (#73) * Pagination bug * Bug fix * Add langgraph support * QDrant support * Add Groq support * update README * update README * feat: optimise docker image for self host setup * adding api access to traces endpoint * clean up * refactor * feat: add clickhouse db create on app start (#79) * docs: add railway deploy, fix sdk badges (#81) * untrack .env * Revert "untrack .env" This reverts commit 4551d7e. * Playground and Prompt Management (#83) * Pagination bug * Bug fix * Playground - basic implementation * Playground - streaming and nonstreaming * Move playground inside project * API key flow * Api key * Playground refactor * Add chat hookup * anthropic streaming support * Bug fixes to openai playground * Anthropic bugfixes * Anthropic bugfix * Cohere first iteration * Cohere role fixes * Cohere api fix * Parallel running * Playground cost calculation non streaming * playground - streaming token calculation * latency and cost * Support for Groq * Add model name * Prompt management views * Remove current promptset flow * Prompt management API hooks * Prompt registry final * Playground bugfixes * Bug fix playground * Rearrange project nav * Fix playground * Fix prompts * Bugfixes * Minor fix * Prompt versioning bugfix * Bugfix * fix: clickhouse table find queries (#82) * Fix to surface multiple LLM requests inside LLM View (#84) * Pagination bug * Bug fix * Fix for surfacing multiple LLM requests in LLMView * Minor bugfixes (#86) * Pagination bug * Bug fix * Bugfixes * api to fetch promptset with prompt filters * bug fixes * fix invalid redirect * fix invalid status code * Project Switcher (#90) * Pagination bug * Bug fix * Project switcher * Feat: dataset download (#60) * API: download dataset * API: Download dataset * updated download-dataset api * Updated: download_dataset api * Updated download dataset API * Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500 * updated the download-dataset API: fixed the format and removed redundant lines of code * Updated download_daatset API: file name and removed 'id' param * Added the Download dataset button. * Merged developemnt into my branch * Updated button size * Fixes --------- Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> * Update prompt registry with instructions to fetch prompts (#91) * Pagination bug * Bug fix * Update prompt registry * Minor bugfix (#94) * Pagination bug * Bug fix * Minor bugfix * chore: update github repo badges * optimizing token count function * Add GPT4-O Pricing and Playground (#98) * Pagination bug * Bug fix * Add GPT4-O support * Add GPT4-O support * Update cost * Dylan/s3en 2234 add perplexity support to playground (#89) * adding perplexity to playground types * adding ui stuff:' * adding perplexity chat api * fixing perplexity model dropdown --------- Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> * api changes * add api access to get api and fix all bugs * bug fix * bug fix * updating descriptions to optional * prio python * cleanup and fixes * more bug fixes * more fixes * remove console log * updating trace_service functions * add migration * add format function, updating from day to hour * adding dropwdown menu * updating query key * updating query keys v2 * clean up * fix bug * Minor bugfix (#102) * Pagination bug * Bug fix * Minor bugfix * fixes * Bug fixes to Evaluations (#111) * Pagination bug * Bug fix * Bug fixes to evals v2 * Minor fix * upgrading tiktoken, adding omni to pricing (#110) Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> * Update package * Dataset bugfix (#113) * Pagination bug * Bug fix * Dataset bugfix * Minor linking fix (#114) * Pagination bug * Bug fix * Dataset bugfix * Minor fix * api bug fix (#116) * fixing dropdown color (#117) * Support for Image inputs (#119) * Pagination bug * Bug fix * Support image inputs * Bug fix Spans with status_code = "ERROR" (#120) * Pagination bug * Bug fix * Support image inputs * Bug fixes * Dylan/s3en 2265 save prompt and model settings directly from playground (#99) * bug fixes, adding save prompt button in playground * adding save prompt dialog * adding create registry dialog * working registry selection * Adding new prompt to exist registry * storing * button padding fix * refactor, adding combobox * increasing dropdwon width --------- Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> * adding image display * adding property filtering (#123) * optimizing cost function * optimizing per cost per hour function * removing console statements * updating queryKey for datasets * init advanced filter dialog * adding operator combobox * updating ui * adding attribute type * bug fixes * cleanup * remove console * updating queryKeys * adding missing table checks * adding more checks * chore: update clickhouse docker for user management * style: remove blank space * updating save prompt button * Redesign of Manual Evaluations (#134) * Pagination bug * Bug fix * Schema changes for evals * Fix APIs for evals * Minor bugfixes * Bugfix * Evaluation changes * Manual eval simplification * Minor fix * Manual evaluations UX changes * Chart title fix * Disable experiments * fix * Dylan/s3en 2318 add the ability to import a conversation from the traces to (#133) * adding import conversation button * init import conversation dialog * adding trace rows * fixing scroll bug * adding import prompt to playground * fixes * Minor * Bugfixes --------- Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> * storing * remove import for images * importing attribute stuff * fixing operations display * cleaning up advanced filters * cleanup * docs: update docker instructions * removing console, fixing button * add stars history (#139) * Dylan/s3en 2401 ability to filter by user for metrics and traces (#138) * adding user id combobox, api * fixing sql query * removing user filter on deselect{ * moving user combobox into filter dialog * adding filtering by user for token charts * bug fixes, more userId route, function updates * latency filtering * updating to alphanumeric * removing console, comment * updating default filter operation to and * adding clear all button, cleanup * Dylan/s3en 2350 aggregate metrics by models v2 (#141) * adding user id combobox, api * fixing sql query * removing user filter on deselect{ * moving user combobox into filter dialog * adding filtering by user for token charts * bug fixes, more userId route, function updates * latency filtering * updating to alphanumeric * removing console, comment * updating default filter operation to and * adding clear all button, cleanup * init commit model combobox * adding api, updating function, adding to metrics page * adding model to charts, apis, service * remove consoles * Automated Evaluations - Support for Inspect AI (#142) * Pagination bug * Bug fix * Experiments view * re-order * Comparison view * Experiment report * Experiments - hook up APIs * Experiments pagination * deduplicate paginated data * Fix loading and error states - experiments * Rename experiments and evaluations * Add loading spinner * Link evaluation * Link to evals * Fixes * Minor UI fixes * Bugfixes * DSPy support (#144) * Pagination bug * Bug fix * Bugfixes * DSPy support * Update README * Prompt registry and Playground integration (#146) * Pagination bug * Bug fix * Bugfixes * Playground enhancements * Fix * Playground improvment * Bug fix * Minor bug fixes (#148) * Pagination bug * Bug fix * Bugfixes * Bugfixes * feat: adding example code for starters * Bugfixes (#152) * Pagination bug * Bug fix * Bugfixes * crewai support * fix ollama bug * updating card link to traces * adding copy to clipboard for prompt id * Dylan/s3en 2415 expose project creation api for hivenetwork (#156) * adding api key auth for project, displaying team id * migrating, adding api key gen to teams page * removing comment * removing apiKeyHash from project return * Minorfixes --------- Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> * Dylan/s3en 2393 update UI tracing tab to show the prompt version being used v2 (#154) * Adding prompt id to traces table * adding prompt combobox, api route, service function * adding prompt version to traces tab * Fix --------- Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com> --------- Co-authored-by: Darshit Suratwala <darshit@scale3labs.com> Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com> Co-authored-by: dylan <dylan@scale3labs.com> Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com> Co-authored-by: Rohit Kadhe <rohit@scale3labs.com> Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com> Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com> Co-authored-by: yemiscale3 <111706151+yemiadej@users.noreply.github.com>
Scale3-Labs · Jun 28, 2024 · dd53663 · dd53663
1 parent dd838c7
commit dd53663
Show file tree

Hide file tree

Showing 15 changed files with 486 additions and 33 deletions.
diff --git a/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx b/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx
@@ -10,7 +10,7 @@ import { Switch } from "@/components/ui/switch";
 import { cn } from "@/lib/utils";
 import { Prompt } from "@prisma/client";
 import CodeEditor from "@uiw/react-textarea-code-editor";
-import { ChevronLeft } from "lucide-react";
+import { ChevronLeft, ClipboardIcon } from "lucide-react";
 import { useParams, useRouter } from "next/navigation";
 import { useState } from "react";
 import { useQuery, useQueryClient } from "react-query";
@@ -196,9 +196,24 @@ export default function Page() {
             </div>
             <div className="flex flex-col gap-2">
               <Label>Prompt Registry ID</Label>
-              <p className="p-2 rounded-md border-2 border-muted font-semibold text-md">
-                {promptsetId}
-              </p>
+              <div className="flex items-center border-2 border-muted p-2 rounded-md cursor-pointer font-semibold text-md">
+                <p
+                  onClick={() => {
+                    navigator.clipboard.writeText(promptsetId);
+                    toast.success("Copied to clipboard");
+                  }}
+                  className="flex-grow"
+                >
+                  {promptsetId}
+                </p>
+                <ClipboardIcon
+                  className="h-4 w-4 cursor-pointer text-muted-foreground"
+                  onClick={() => {
+                    navigator.clipboard.writeText(promptsetId);
+                    toast.success("Copied to clipboard");
+                  }}
+                />
+              </div>
             </div>
             <div className="flex flex-col gap-2">
               <Label>Prompt</Label>

diff --git a/app/(protected)/projects/page-client.tsx b/app/(protected)/projects/page-client.tsx
@@ -178,7 +178,7 @@ function ProjectCard({
           )}
         <Edit teamId={teamId} project={project} />
       </div>
-      <Link href={`/project/${project.id}/metrics`}>
+      <Link href={`/project/${project.id}/traces`}>
         <Card className="w-full md:w-[325px] h-[180px] shadow-md hover:cursor-pointer transition-all duration-200 ease-in-out border-muted hover:border-muted-foreground border-2 hover:shadow-lg hover:bg-muted">
           <CardHeader>
             <CardTitle className="capitalize w-1/2 truncate">

diff --git a/app/(protected)/settings/team/page-client.tsx b/app/(protected)/settings/team/page-client.tsx
@@ -11,21 +11,52 @@ import {
 import {
   Form,
   FormControl,
+  FormDescription,
   FormField,
   FormItem,
   FormLabel,
   FormMessage,
 } from "@/components/ui/form";
 import { Input } from "@/components/ui/input";
 import { zodResolver } from "@hookform/resolvers/zod";
+import { CodeIcon, RefreshCwIcon } from "lucide-react";
 import { useState } from "react";
 import { FieldValues, useForm } from "react-hook-form";
 import { toast } from "sonner";
 import { z } from "zod";
 
 export default function TeamView({ user }: { user: any }) {
-  // const queryClient = useQueryClient();
+  const [apiKey, setApiKey] = useState<string>();
   const [busy, setBusy] = useState(false);
+
+  const handleApiKeyGenerated = (newApiKey: string) => {
+    setApiKey(newApiKey);
+  };
+
+  const generateApiKey = async () => {
+    try {
+      setBusy(true);
+      const response = await fetch(`/api/api-key?team_id=${user.Team.id}`, {
+        method: "POST",
+        headers: {
+          "Content-Type": "application/json",
+        },
+      });
+      const result = await response.json();
+      setApiKey(result.data.apiKey);
+      handleApiKeyGenerated(result.data.apiKey);
+      toast("Copy your API Key!", {
+        description: "Please copy your API key. It will not be shown again.",
+      });
+    } catch (error: any) {
+      toast("Error generating API Key!", {
+        description: `There was an error generating your API Key: ${error.message}`,
+      });
+    } finally {
+      setBusy(false);
+    }
+  };
+
   const NameFormSchema = z.object({
     name: z
       .string()
@@ -89,7 +120,63 @@ export default function TeamView({ user }: { user: any }) {
                 </FormItem>
               )}
             />
+            <FormLabel>Team ID</FormLabel>
+            <div className="flex items-center bg-muted p-2 rounded-md justify-between">
+              <p
+                onClick={() => {
+                  navigator.clipboard.writeText(user.Team.id);
+                  toast.success("Copied to clipboard");
+                }}
+                className="text-sm select-all selection:bg-blue-200"
+              >
+                {user.Team.id}
+              </p>
+            </div>
+            <FormLabel>API Key</FormLabel>
+            <FormDescription className="text-red-600 font-bold">
+              {user.Team.apiKeyHash && !apiKey
+                ? "Note: If you regenerate the API key, the old key will be replaced."
+                : "Note: Click to copy this API key as it will NOT be shown again."}
+            </FormDescription>
+            {apiKey && (
+              <div className="flex items-center bg-muted p-2 rounded-md justify-between">
+                <p
+                  onClick={() => {
+                    navigator.clipboard.writeText(apiKey);
+                    toast.success("Copied to clipboard");
+                  }}
+                  className="text-sm select-all selection:bg-blue-200"
+                >
+                  {apiKey}
+                </p>
+                <button
+                  className="bg-primary-foreground rounded-md"
+                  onClick={() => {
+                    navigator.clipboard.writeText(apiKey);
+                    toast.success("Copied to clipboard");
+                  }}
+                />
+              </div>
+            )}
+            {!apiKey && (
+              <Button
+                className="w-fit"
+                variant={user.Team.apiKeyHash ? "destructive" : "default"}
+                onClick={generateApiKey}
+                disabled={busy}
+              >
+                {user.Team.apiKeyHash ? (
+                  <RefreshCwIcon className="mr-2 h-4 w-4" />
+                ) : (
+                  <CodeIcon className="mr-2 h-4 w-4" />
+                )}
+                {user.Team.apiKeyHash
+                  ? "Re-generate API Key"
+                  : "Generate API Key"}
+              </Button>
+            )}
             <Button
+              type="button"
               disabled={busy}
               onClick={NameDetailsForm.handleSubmit(saveNameDetails)}
               className="w-fit"

diff --git a/app/api/api-key/route.ts b/app/api/api-key/route.ts
@@ -1,15 +1,19 @@
+import { authOptions } from "@/lib/auth/options";
 import prisma from "@/lib/prisma";
 import { generateApiKey, hashApiKey } from "@/lib/utils";
+import { getServerSession } from "next-auth";
+import { redirect } from "next/navigation";
 import { NextRequest, NextResponse } from "next/server";
 
 // for generating new API key
 export async function POST(req: NextRequest) {
-  const id = req.nextUrl.searchParams.get("id") as string;
+  const projectId = req.nextUrl.searchParams.get("project_id") as string;
+  const teamId = req.nextUrl.searchParams.get("team_id") as string;
 
-  if (!id) {
+  if (!projectId && !teamId) {
     return NextResponse.json(
       {
-        error: "Missing project id",
+        error: "Missing project_id or team_id",
       },
       { status: 400 }
     );
@@ -18,18 +22,81 @@ export async function POST(req: NextRequest) {
   const apiKey = generateApiKey();
   const hash = hashApiKey(apiKey);
 
-  await prisma.project.update({
-    where: {
-      id,
-    },
-    data: {
-      apiKeyHash: hash,
-    },
-  });
+  if (projectId) {
+    await prisma.project.update({
+      where: {
+        id: projectId,
+      },
+      data: {
+        apiKeyHash: hash,
+      },
+    });
+  } else if (teamId) {
+    await prisma.team.update({
+      where: {
+        id: teamId,
+      },
+      data: {
+        apiKeyHash: hash,
+      },
+    });
+  }
 
   return NextResponse.json({
     data: {
       apiKey: apiKey,
     },
   });
 }
+
+export async function GET(req: NextRequest) {
+  try {
+    const session = await getServerSession(authOptions);
+    if (!session || !session.user) {
+      redirect("/login");
+    }
+
+    const teamId = req.nextUrl.searchParams.get("teamId") as string;
+    const projectId = req.nextUrl.searchParams.get("projectId") as string;
+
+    if (!teamId && !projectId) {
+      return NextResponse.json(
+        {
+          error: "No projectId or teamId provided",
+        },
+        { status: 404 }
+      );
+    }
+
+    if (teamId) {
+      const result = await prisma.team.findUnique({
+        where: {
+          id: teamId,
+        },
+      });
+
+      return NextResponse.json({
+        data: result,
+      });
+    }
+
+    if (projectId) {
+      const result = await prisma.project.findUnique({
+        where: {
+          id: projectId,
+        },
+      });
+
+      return NextResponse.json({
+        data: result,
+      });
+    }
+  } catch (error) {
+    return NextResponse.json(
+      {
+        message: "Internal server error",
+      },
+      { status: 500 }
+    );
+  }
+}
diff --git a/app/api/project/route.ts b/app/api/project/route.ts
@@ -1,6 +1,7 @@
 import { authOptions } from "@/lib/auth/options";
 import { DEFAULT_TESTS } from "@/lib/constants";
 import prisma from "@/lib/prisma";
+import { authApiKey } from "@/lib/utils";
 import { getServerSession } from "next-auth";
 import { redirect } from "next/navigation";
 import { NextRequest, NextResponse } from "next/server";
@@ -42,11 +43,18 @@ export async function GET(req: NextRequest) {
 }
 
 export async function POST(req: NextRequest) {
-  const session = await getServerSession(authOptions);
-  if (!session || !session.user) {
-    redirect("/login");
+  const apiKey = req.headers.get("x-api-key");
+  if (!apiKey) {
+    const session = await getServerSession(authOptions);
+    if (!session || !session.user) {
+      redirect("/login");
+    }
+  } else {
+    const response = await authApiKey(apiKey, true);
+    if (response.status !== 200) {
+      return response;
+    }
   }
-
   const data = await req.json();
   const { name, description, teamId } = data;
   let createDefaultTests = data.createDefaultTests;
@@ -75,8 +83,10 @@ export async function POST(req: NextRequest) {
     }
   }
 
+  const { apiKeyHash, ...projectWithoutApiKeyHash } = project;
+
   return NextResponse.json({
-    data: project,
+    data: projectWithoutApiKeyHash,
   });
 }
 

diff --git a/app/api/prompt-ids/route.ts b/app/api/prompt-ids/route.ts
@@ -0,0 +1,37 @@
+import { authOptions } from "@/lib/auth/options";
+import { TraceService } from "@/lib/services/trace_service";
+import { getServerSession } from "next-auth";
+import { redirect } from "next/navigation";
+import { NextRequest, NextResponse } from "next/server";
+
+export async function GET(req: NextRequest) {
+  const session = await getServerSession(authOptions);
+
+  if (!session || !session.user) {
+    redirect("/login");
+  }
+
+  try {
+    const projectId = req.nextUrl.searchParams.get("projectId") as string;
+
+    if (!projectId) {
+      return NextResponse.json(
+        { error: "Please provide a projectId" },
+        { status: 400 }
+      );
+    }
+
+    const traceService = new TraceService();
+    const promptIDs = await traceService.GetPromptsInProject(projectId);
+    return NextResponse.json(
+      { promptIDs },
+      {
+        status: 200,
+      }
+    );
+  } catch (error) {
+    return NextResponse.json(JSON.stringify({ error }), {
+      status: 400,
+    });
+  }
+}