Skip to content

Commit

Permalink
Release 1.5.6 (#153)
Browse files Browse the repository at this point in the history 
* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7e.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

* Dataset bugfix (#113)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor linking fix (#114)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor fix

* api bug fix (#116)

* fixing dropdown color (#117)

* Support for Image inputs (#119)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fix Spans with status_code = "ERROR" (#120)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fixes

* Dylan/s3en 2265 save prompt and model settings directly from playground (#99)

* bug fixes, adding save prompt button in playground

* adding save prompt dialog

* adding create registry dialog

* working registry selection

* Adding new prompt to exist registry

* storing

* button padding fix

* refactor, adding combobox

* increasing dropdwon width

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* adding image display

* adding property filtering (#123)

* optimizing cost function

* optimizing per cost per hour function

* removing console statements

* updating queryKey for datasets

* init advanced filter dialog

* adding operator combobox

* updating ui

* adding attribute type

* bug fixes

* cleanup

* remove console

* updating queryKeys

* adding missing table checks

* adding more checks

* chore: update clickhouse docker for user management

* style: remove blank space

* updating save prompt button

* Redesign of Manual Evaluations (#134)

* Pagination bug

* Bug fix

* Schema changes for evals

* Fix APIs for evals

* Minor bugfixes

* Bugfix

* Evaluation changes

* Manual eval simplification

* Minor fix

* Manual evaluations UX changes

* Chart title fix

* Disable experiments

* fix

* Dylan/s3en 2318 add the ability to import a conversation from the traces to (#133)

* adding import conversation button

* init import conversation dialog

* adding trace rows

* fixing scroll bug

* adding import prompt to playground

* fixes

* Minor

* Bugfixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* storing

* remove import for images

* importing attribute stuff

* fixing operations display

* cleaning up advanced filters

* cleanup

* docs: update docker instructions

* removing console, fixing button

* add stars history (#139)

* Dylan/s3en 2401 ability to filter by user for metrics and traces (#138)

* adding user id combobox, api

* fixing sql query

* removing user filter on deselect{

* moving user combobox into filter dialog

* adding filtering by user for token charts

* bug fixes, more userId route, function updates

* latency filtering

* updating to alphanumeric

* removing console, comment

* updating default filter operation to and

* adding clear all button, cleanup

* Dylan/s3en 2350 aggregate metrics by models v2 (#141)

* adding user id combobox, api

* fixing sql query

* removing user filter on deselect{

* moving user combobox into filter dialog

* adding filtering by user for token charts

* bug fixes, more userId route, function updates

* latency filtering

* updating to alphanumeric

* removing console, comment

* updating default filter operation to and

* adding clear all button, cleanup

* init commit model combobox

* adding api, updating function, adding to metrics page

* adding model to charts, apis, service

* remove consoles

* Automated Evaluations - Support for Inspect AI (#142)

* Pagination bug

* Bug fix

* Experiments view

* re-order

* Comparison view

* Experiment report

* Experiments - hook up APIs

* Experiments pagination

* deduplicate paginated data

* Fix loading and error states - experiments

* Rename experiments and evaluations

* Add loading spinner

* Link evaluation

* Link to evals

* Fixes

* Minor UI fixes

* Bugfixes

* DSPy support (#144)

* Pagination bug

* Bug fix

* Bugfixes

* DSPy support

* Update README

* Prompt registry and Playground integration (#146)

* Pagination bug

* Bug fix

* Bugfixes

* Playground enhancements

* Fix

* Playground improvment

* Bug fix

* Minor bug fixes (#148)

* Pagination bug

* Bug fix

* Bugfixes

* Bugfixes

* feat: adding example code for starters

* Bugfixes (#152)

* Pagination bug

* Bug fix

* Bugfixes

* crewai support

* fix ollama bug

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>
Co-authored-by: yemiscale3 <111706151+yemiadej@users.noreply.github.com>
  • Loading branch information
@karthikscale3 @darshit-s3
@dylanzuber-scale3 @rohit-kadhe @MayuriS24 @yemiadej
9 people authored Jun 25, 2024
1 parent 00cb17a commit dd838c7
Show file tree
Hide file tree
Showing 7 changed files with 258 additions and 110 deletions.
26 changes: 19 additions & 7 deletions components/shared/llm-view.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ export const LLMView = ({
const [selectedTab, setSelectedTab] = useState(0);
return (
<div className="flex flex-col gap-6 p-4 border-[1px] border-muted rounded-lg shadow-md bg-primary-foreground">
<div className="flex items-center gap-2">
<div className="flex flex-wrap items-center gap-2">
{prompts.map((prompt: any, i: number) => (
<Button
key={i}
Expand Down Expand Up @@ -72,12 +72,24 @@ export const LLMView = ({
</div>
{prompts?.length > 0 &&
JSON.parse(prompts[selectedTab]).map((prompt: any, i: number) => {
const role = prompt?.role ? prompt?.role?.toLowerCase() : "User";
const content = prompt?.content
? prompt?.content
: prompt?.function_call
? prompt?.function_call
: "";
let role;
let content;
// check if prompt is a list
if (Array.isArray(prompt) && prompt.length > 0) {
role = prompt[0]?.role ? prompt[0]?.role?.toLowerCase() : "User";
content = prompt[0]?.content
? prompt[0]?.content
: prompt[0]?.function_call
? prompt[0]?.function_call
: "";
} else {
role = prompt?.role ? prompt?.role?.toLowerCase() : "User";
content = prompt?.content
? prompt?.content
: prompt?.function_call
? prompt?.function_call
: "";
}

// will add once edit image wrapper has been added in sdks
// const url = prompt?.content?.url;
Expand Down
6 changes: 4 additions & 2 deletions components/shared/project-switcher.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,15 @@ export function ProjectSwitcher({ email }: { email: string }) {
variant="outline"
role="combobox"
aria-expanded={open}
className="w-[200px] justify-between"
className="flex w-[200px] justify-between"
>
<p className="truncate">
{
projects?.projects.filter(
(project: any) => project.id === pathname.split("/")[2]
)[0]?.name
}
</p>
<CaretSortIcon className="ml-2 h-4 w-4 shrink-0 opacity-50" />
</Button>
</PopoverTrigger>
Expand All @@ -78,7 +80,7 @@ export function ProjectSwitcher({ email }: { email: string }) {
onSelect={() => setOpen(false)}
>
<div className="flex justify-between w-full cursor-pointer">
{project.name}
<p className="truncate">{project.name}</p>
<CheckIcon
className={cn(
"ml-auto h-4 w-4",
Expand Down
298 changes: 197 additions & 101 deletions components/shared/setup-instructions.tsx
View file
Original file line number Diff line number Diff line change
@@ -1,115 +1,211 @@
import { useState } from "react";
import { toast } from "sonner";
import {
Accordion,
AccordionContent,
AccordionItem,
AccordionTrigger,
} from "../ui/accordion";
import { Button } from "../ui/button";
import GenerateApiKey from "./api-key";
import { Separator } from "../ui/separator";

type SdkType = "python" | "typescript";

const pythonCodeTemplate = (apiKey: string) =>
`# Must precede any llm module imports
from langtrace_python_sdk import langtrace
langtrace.init(api_key = '${apiKey ?? "<LANGTRACE_API_KEY>"}')`;

const typescriptCodeTemplate = (apiKey: string) =>
`// Must precede any llm module imports
import * as Langtrace from '@langtrace/typescript-sdk'
Langtrace.init({ api_key: '${apiKey ?? "<LANGTRACE_API_KEY>"}' })`;

const installCommands = {
python: "pip install langtrace-python-sdk",
typescript: "npm i @langtrase/typescript-sdk",
};

const pythonExampleCode = (apiKey: string) =>
`from langtrace_python_sdk import langtrace
from langtrace_python_sdk.utils.with_root_span import with_langtrace_root_span
from openai import OpenAI
langtrace.init(
api_key="${apiKey ?? "<YOUR API KEY>"}"
)
@with_langtrace_root_span()
def example():
client = OpenAI()
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{
"role": "system",
"content": "How many states of matter are there?"
}
],
)
print(response.choices[0].message.content)
example()`;

const typescriptExampleCode = (apiKey: string) =>
`import * as Langtrace from "@langtrase/typescript-sdk";
import OpenAI from "openai";
Langtrace.init({
api_key: "${apiKey ?? "<YOUR API KEY>"}",
batch: false,
instrumentations: {
openai: OpenAI,
},
});
const openai = new OpenAI();
async function example() {
const completion = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content: "How many states of matter are there?",
},
],
});
console.log(completion.choices[0]);
}
example().then(() => {
console.log("done");
});`;

const CodeBlock = ({ code, onClick }: { code: string; onClick: any }) => (
<pre
className="text-xs p-2 rounded-md bg-muted select-all selection:bg-orange-400 dark:selection:bg-orange-600"
onClick={onClick}
>
{code}
</pre>
);

const copyToClipboard = (text: string) => {
navigator.clipboard
.writeText(text)
.then(() => {
toast.success("Copied to clipboard");
})
.catch((err) => {
console.error("Failed to copy!", err);
});
};

export function SetupInstructions({ project_id }: { project_id: string }) {
const [sdk, setSdk] = useState("python");
const [apiKey, setApiKey] = useState("<LANGTRACE_API_KEY>");
const [sdk, setSdk] = useState<SdkType>("python");
const [apiKey, setApiKey] = useState("<LANGTRACE_API_KEY>");

const handleApiKeyGenerated = (newApiKey: string) => {
setApiKey(newApiKey);
};
const copyToClipboard = (code: string) => {
navigator.clipboard.writeText(code);
return toast.success("Copied to clipboard");
};
const handleApiKeyGenerated = (newApiKey: string) => {
setApiKey(newApiKey);
};

return (
<div className="flex flex-col gap-6 border rounded-md p-4">
<div className="flex flex-row items-center gap-3">
<Button
onClick={() => setSdk("python")}
variant={sdk === "python" ? "default" : "ghost"}
size={"sm"}
>
Python
</Button>
<Button
onClick={() => setSdk("typescript")}
variant={sdk === "typescript" ? "default" : "ghost"}
size={"sm"}
>
TypeScript
</Button>
</div>
<div className="flex flex-col gap-3">
<p className="text-sm text-muted-foreground">
1. Install the Langtrace SDK in your application.
</p>
<pre
className="text-xs p-2 rounded-md bg-muted select-all selection:bg-orange-400 dark:selection:bg-orange-600"
onClick={() => {
copyToClipboard(
sdk === "typescript"
? "npm i @langtrase/typescript-sdk"
: "pip install langtrace-python-sdk"
);
}}
>
{sdk === "typescript" && "npm i @langtrase/typescript-sdk"}
{sdk === "python" && "pip install langtrace-python-sdk"}
</pre>
</div>
<div className="flex flex-col items-start gap-3">
<p className="text-sm text-muted-foreground">
2. Generate and copy your API key.
</p>
<GenerateApiKey
projectId={project_id}
onApiKeyGenerated={handleApiKeyGenerated}
/>
</div>
<div className="flex flex-col gap-3">
<p className="text-sm text-muted-foreground">
3. Initialize the SDK with the following code snippet
</p>
{sdk === "typescript" && (
<pre
className="text-xs p-2 rounded-md bg-muted select-all selection:bg-orange-400 dark:selection:bg-orange-600"
onClick={() => {
copyToClipboard(
`//Must precede any llm module imports\n\nimport * as Langtrace from '@langtrase/typescript-sdk'\n\nLangtrace.init({ api_key: '${
apiKey ?? "<LANGTRACE_API_KEY>"
}' })`
);
}}
>
{"// Must precede any llm module imports\n\n"}
{"import * as Langtrace from '@langtrase/typescript-sdk'\n\n"}
{`Langtrace.init({ api_key: '${
apiKey ?? "<LANGTRACE_API_KEY>"
}' })`}
</pre>
)}
{sdk === "python" && (
<pre
className="text-xs p-2 rounded-md bg-muted select-all selection:bg-orange-400 dark:selection:bg-orange-600"
onClick={() => {
copyToClipboard(
`# Must precede any llm module imports\n\nfrom langtrace_python_sdk import langtrace\n\nlangtrace.init(api_key = '${
apiKey ?? "<LANGTRACE_API_KEY>"
}')`
);
}}
>
{"# Must precede any llm module imports\n\n"}
{"from langtrace_python_sdk import langtrace\n\n"}
{`langtrace.init(api_key = '${apiKey ?? "<LANGTRACE_API_KEY>"}')`}
</pre>
)}
</div>
</div>
);
const handleCodeCopy = (code: string) => {
copyToClipboard(code);
};

const sdkCodeTemplates = {
python: pythonCodeTemplate(apiKey),
typescript: typescriptCodeTemplate(apiKey),
};

const exampleCodeTemplates = {
python: pythonExampleCode(apiKey),
typescript: typescriptExampleCode(apiKey),
};

return (
<div className="flex flex-col gap-6 border rounded-md p-4">
<div className="flex flex-row items-center gap-3">
<Button
onClick={() => setSdk("python")}
variant={sdk === "python" ? "default" : "ghost"}
size={"sm"}
>
Python
</Button>
<Button
onClick={() => setSdk("typescript")}
variant={sdk === "typescript" ? "default" : "ghost"}
size={"sm"}
>
TypeScript
</Button>
</div>
<div className="flex flex-col gap-3">
<p className="text-sm text-muted-foreground">
1. Install the Langtrace SDK in your application.
</p>
<CodeBlock
code={installCommands[sdk]}
onClick={() => handleCodeCopy(installCommands[sdk])}
/>
</div>
<div className="flex flex-col items-start gap-3">
<p className="text-sm text-muted-foreground">
2. Generate and copy your API key.
</p>
<GenerateApiKey
projectId={project_id}
onApiKeyGenerated={handleApiKeyGenerated}
/>
</div>
<div className="flex flex-col gap-3">
<p className="text-sm text-muted-foreground">
3. Initialize the SDK with the following code snippet
</p>
<CodeBlock
code={sdkCodeTemplates[sdk]}
onClick={() => handleCodeCopy(sdkCodeTemplates[sdk])}
/>

<Accordion type="single" collapsible className="w-full">
<AccordionItem value="item-1">
<AccordionTrigger>
Want an example to generate traces?
</AccordionTrigger>
<AccordionContent>
<CodeBlock
code={exampleCodeTemplates[sdk]}
onClick={() => handleCodeCopy(exampleCodeTemplates[sdk])}
/>
<Separator className="my-4" />
<p>
Checkout{" "}
<a
href="https://docs.langtrace.ai/how-to-guides"
target="_blank"
rel="noopener noreferrer"
className="text-blue-600"
>
Langtrace Docs
</a>{" "}
for detailed information on how to instrument your code.
</p>
</AccordionContent>
</AccordionItem>
</Accordion>
</div>
</div>
);
}

export function PromptInstructions({ id }: { id: string }) {
const [sdk, setSdk] = useState("python");

const copyToClipboard = (code: string) => {
navigator.clipboard.writeText(code);
return toast.success("Copied to clipboard");
};
const [sdk, setSdk] = useState<SdkType>("python");

return (
<div className="flex flex-col gap-6 border rounded-md p-4">
Expand Down
Loading

0 comments on commit dd838c7

Please sign in to comment.