Skip to content

Dev #1545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 150 commits into from
May 27, 2025
Merged

Dev #1545

Changes from 7 commits
Commits
Show all changes
150 commits
Select commit Hold shift + click to select a range
ab5142f
disk space script
pelikhan May 9, 2025
53768f2
add glossary script
pelikhan May 10, 2025
cbbc919
tweak
pelikhan May 10, 2025
5bbd404
✨ feat: improve glossary and add "tiny" model alias
pelikhan May 10, 2025
7af903a
updated glossary
pelikhan May 10, 2025
3f5fb78
✨ feat: revamp glossary and enhance GenAIScript features
pelikhan May 10, 2025
6e56ca2
✨ chore: Upgrade dependencies across multiple packages
pelikhan May 10, 2025
1aea590
Merge pull request #1542 from microsoft/glossary
pelikhan May 10, 2025
43af316
render description as markdown
pelikhan May 10, 2025
46e7b24
Add package manager selection support in CLI and refine Azure credent…
pelikhan May 12, 2025
8dbc1f0
Mcp-images (#1543)
pelikhan May 12, 2025
42473ab
Add metadata support to requests (#1548)
pelikhan May 13, 2025
a6219a2
Call tools (#1546)
pelikhan May 13, 2025
2472e30
✨: Enhance stored completions and metadata handling
pelikhan May 13, 2025
5b507b0
new line
pelikhan May 13, 2025
9f07efc
Release 1.136.0
pelikhan May 13, 2025
fe36d61
[skip ci] updated version numbers
pelikhan May 13, 2025
cbdc102
fix: regression in vscode copuilot chat support
pelikhan May 13, 2025
b2911b2
Release 1.136.1
pelikhan May 13, 2025
3bab921
[skip ci] updated version numbers
pelikhan May 13, 2025
bb84e21
✨: Add metadata handling support across components
pelikhan May 13, 2025
307f8eb
Release 1.136.2
pelikhan May 13, 2025
a99c748
[skip ci] updated version numbers
pelikhan May 13, 2025
bb9c94e
normalize metadata
pelikhan May 13, 2025
733d8c9
missing --yes
pelikhan May 13, 2025
e2d2288
Release 1.136.3
pelikhan May 13, 2025
2c10981
[skip ci] updated version numbers
pelikhan May 13, 2025
4373706
✨ feat: add runId to expansion variables for unique tracking
pelikhan May 13, 2025
e8c71fa
emphasize on create pr
pelikhan May 14, 2025
0980008
typo
pelikhan May 14, 2025
f5cc9b7
Use CLI for GitHub token resolution (#1550)
pelikhan May 14, 2025
e573385
3rd party license update
pelikhan May 14, 2025
53f6461
✨ chore: Update CI workflow and script configurations
pelikhan May 15, 2025
2c77279
:truck: Rename job to `generate-alt-text`
pelikhan May 15, 2025
8636654
fix image encoding
pelikhan May 15, 2025
6900073
Release 1.136.4
pelikhan May 15, 2025
aa385ba
[skip ci] updated version numbers
pelikhan May 15, 2025
dd49d51
:sparkles: Update workflows and refine logging mechanism
pelikhan May 15, 2025
b52df92
updated action
pelikhan May 15, 2025
c18f9d9
updated workflow
pelikhan May 15, 2025
7bbcf1d
handle github images
pelikhan May 15, 2025
380fd03
duplicate section
pelikhan May 15, 2025
4ab22ee
:fire: Remove unused file pattern in GitHub Action
pelikhan May 15, 2025
51df602
:construction: Adjust trace logic for CI environment
pelikhan May 15, 2025
a2e5d9f
Show UI element when no script is selected (#1551)
pelikhan May 15, 2025
207df95
es2024 (#1552)
pelikhan May 15, 2025
773b4ec
fix tokens
pelikhan May 15, 2025
fb9fd6f
Release 1.136.5
pelikhan May 15, 2025
39c1dc8
link to azure openai
pelikhan May 15, 2025
42bf584
✨ enhance: Add await to promiseExample invocation
pelikhan May 15, 2025
02ac6d1
Release 1.136.6
pelikhan May 15, 2025
34c6673
[skip ci] updated version numbers
pelikhan May 15, 2025
d10944a
:sparkles: Enhance logging for Azure token generation
pelikhan May 15, 2025
2137e97
fix docker docs
pelikhan May 15, 2025
26d7cb4
typos
pelikhan May 15, 2025
25de583
make iat more resilient
pelikhan May 15, 2025
2863098
updated prompts
pelikhan May 15, 2025
3838c9f
removing promptfoo dependency
pelikhan May 15, 2025
e35ef69
Release 1.136.7
pelikhan May 15, 2025
4b21c59
[skip ci] updated version numbers
pelikhan May 15, 2025
b6c6dbb
updated docs
pelikhan May 15, 2025
fc5f869
Removing-transformers (#1553)
pelikhan May 15, 2025
d4a87a2
add clean option
pelikhan May 15, 2025
d0413fe
✨ Add TestHost setup with beforeEach in tests
pelikhan May 15, 2025
4365858
Release 1.137.0
pelikhan May 15, 2025
d77039e
[skip ci] updated version numbers
pelikhan May 15, 2025
b37ef2a
updated pdf rendering outdated dependency (#1554)
pelikhan May 16, 2025
73aa1bb
fixing permissions
pelikhan May 16, 2025
d10b1e5
:sparkles: Enhance prettyValue API and update model selection
pelikhan May 16, 2025
e817c85
Release 1.137.1
pelikhan May 16, 2025
0cffe65
[skip ci] updated version numbers
pelikhan May 16, 2025
d6703b5
🎨 chore: add private field to package.json
pelikhan May 16, 2025
c6f0ca6
Support for parsing mermaid diagrams (#1538)
pelikhan May 16, 2025
66a4afd
Release 1.137.2
pelikhan May 16, 2025
5e98132
[skip ci] updated version numbers
pelikhan May 16, 2025
71fc490
clean command
pelikhan May 16, 2025
6f977a4
mermaid blog post
pelikhan May 16, 2025
59087b4
Release 1.137.3
pelikhan May 16, 2025
bf5e617
[skip ci] updated version numbers
pelikhan May 16, 2025
0a4dfce
✨ feat: add support for downloading workflow artifacts (#1555)
pelikhan May 16, 2025
c20cb2c
🚀: Refactor GitHub client to use a `normalizeInt` utility (#1556)
pelikhan May 16, 2025
2c096a6
:sparkles: Standardize blog metadata and enhance clarity
pelikhan May 16, 2025
53e25c0
✨ refactor CLI and VSCode functionalities
pelikhan May 16, 2025
10da5ff
Release 1.137.4
pelikhan May 16, 2025
58e6c3d
[skip ci] updated version numbers
pelikhan May 16, 2025
4090a64
Generate-image-retry (#1557)
pelikhan May 16, 2025
8c11f0f
Release 1.137.5
pelikhan May 16, 2025
102a4e4
[skip ci] updated version numbers
pelikhan May 16, 2025
eb49188
update prd
pelikhan May 16, 2025
c932e7e
renamed file
pelikhan May 16, 2025
7ac5cdf
move cross-fetch-oai in dependencies
pelikhan May 16, 2025
33d7cb5
✨: Update GenAIScript instructions to include ESM support
pelikhan May 16, 2025
383a97f
Release 1.137.6
pelikhan May 16, 2025
b88e21b
[skip ci] updated version numbers
pelikhan May 16, 2025
9a20104
more docs
pelikhan May 16, 2025
5df5547
rotate labels
pelikhan May 16, 2025
e9d79df
Release 1.137.7
pelikhan May 16, 2025
78e34e1
[skip ci] updated version numbers
pelikhan May 16, 2025
4038523
🎉 feat: add CI check to skip CLI trace generation
pelikhan May 17, 2025
ae35309
migration github models (#1558)
pelikhan May 17, 2025
7812af7
refresh lock files (#1559)
pelikhan May 17, 2025
dc134e4
✨ feat: clarify error handling for mermaid diagrams
pelikhan May 17, 2025
f6954ab
Release 1.138.0
pelikhan May 17, 2025
712967c
[skip ci] updated version numbers
pelikhan May 17, 2025
a77781c
approximate tokens for large strings (#1560)
pelikhan May 17, 2025
7c7cbf1
[genai] generated blog posts
pelikhan May 18, 2025
1914bb2
[genai] generated blog posts
pelikhan May 19, 2025
1b8028c
✨ Enhance message conversion logic with filtering (#1562)
pelikhan May 19, 2025
ca67735
Release 1.138.1
pelikhan May 19, 2025
d5d83db
[skip ci] updated version numbers
pelikhan May 19, 2025
51d6ba0
don't estimate tokens on message - too much compute
pelikhan May 19, 2025
59c9bfb
git pull (#1563)
pelikhan May 19, 2025
239cb54
Release 1.138.2
pelikhan May 19, 2025
fbdeb40
[skip ci] updated version numbers
pelikhan May 19, 2025
75f827f
wording
pelikhan May 19, 2025
5e75621
[genai] generated blog posts
pelikhan May 20, 2025
09284ff
✨ feat: add support for updating GitHub issues
pelikhan May 20, 2025
99ba7ed
Release 1.138.3
pelikhan May 20, 2025
ed0ab1e
[skip ci] updated version numbers
pelikhan May 20, 2025
b1f8c56
ignore .http files
pelikhan May 20, 2025
8774696
[genai] generated blog posts
pelikhan May 21, 2025
83f1532
✨ chore: update dependencies across multiple packages
pelikhan May 21, 2025
f35b8c5
🩹: Remove sensitive debug logging for tokens
pelikhan May 21, 2025
47f7941
✨: Improve body formatting in traceFetchPost
pelikhan May 21, 2025
744c97c
Release 1.138.4
pelikhan May 21, 2025
e294b8f
[skip ci] updated version numbers
pelikhan May 21, 2025
d4ec1fc
✨ chore: Update dependencies across multiple packages
pelikhan May 21, 2025
fad1a6a
Release 1.138.5
pelikhan May 21, 2025
e4c8567
[skip ci] updated version numbers
pelikhan May 21, 2025
8d81ea8
[genai] generated blog posts
pelikhan May 22, 2025
2027081
✨ refactor collectFolders to specify return type
pelikhan May 22, 2025
49e3fd2
don't fix node_modules (#1568)
pelikhan May 22, 2025
cf3ba03
✨ feat: add asset URL resolution for GitHub uploads (#1569)
pelikhan May 22, 2025
834b673
OpenAPI (#1571)
pelikhan May 23, 2025
e2ff5e8
Release 1.139.0
pelikhan May 23, 2025
7bd372c
[skip ci] updated version numbers
pelikhan May 23, 2025
1335c46
complete the sentence (#1572)
mrjf May 24, 2025
7b96f4c
web api improvements (#1573)
pelikhan May 24, 2025
750a135
✨ refactor: simplify OpenAPI server schema check logic
pelikhan May 24, 2025
c6bad43
[genai] generated blog posts
pelikhan May 25, 2025
30c850a
[genai] generated blog posts
pelikhan May 26, 2025
a199d13
use query argument in openapi
pelikhan May 26, 2025
da4a1a8
✨ enhance: refine OpenAPI server schemas and handlers
pelikhan May 26, 2025
03d5ed8
[genai] generated blog posts
pelikhan May 27, 2025
35cc515
require node v22
pelikhan May 27, 2025
d984d71
add dev container info
pelikhan May 27, 2025
e8966c1
split configuration page (#1576)
pelikhan May 27, 2025
c34948e
added docker model runner
pelikhan May 27, 2025
900f286
Release 1.140.0
pelikhan May 27, 2025
b674bab
[skip ci] updated version numbers
pelikhan May 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/licenses.yml
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ permissions:
on:
workflow_dispatch:
push:
branches: [main, dev]
branches: [dev]
paths:
- "yarn.lock"
concurrency:
1 change: 1 addition & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -109,6 +109,7 @@
"githubclient",
"gitmoji",
"gitmojis",
"glossarify",
"greyscale",
"GROQ",
"Gruber",
16 changes: 8 additions & 8 deletions THIRD_PARTY_LICENSES.md
Original file line number Diff line number Diff line change
@@ -2925,7 +2925,7 @@ MIT License

The following npm package may be included in this product:

- genaiscript-vscode@1.134.4
- genaiscript-vscode@1.135.0

This package contains the following license:

@@ -7590,8 +7590,8 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI

The following npm packages may be included in this product:

- @anthropic-ai/bedrock-sdk@0.21.1
- @anthropic-ai/sdk@0.50.2
- @anthropic-ai/bedrock-sdk@0.21.2
- @anthropic-ai/sdk@0.50.3

These packages each contain the following license:

@@ -8560,10 +8560,10 @@ The following npm packages may be included in this product:
- @napi-rs/canvas-linux-x64-musl@0.1.70
- @tokenizer/token@0.3.0
- eastasianwidth@0.2.0
- genaiscript-core-internal@1.134.4
- genaiscript-sample@1.134.4
- genaiscript-web@1.134.4
- genaiscript@1.134.4
- genaiscript-core-internal@1.135.0
- genaiscript-sample@1.135.0
- genaiscript-web@1.135.0
- genaiscript@1.135.0
- isarray@1.0.0
- javascript-natural-sort@0.7.1
- keyv@4.5.4
@@ -13061,7 +13061,7 @@ OTHER DEALINGS IN THE SOFTWARE.

The following npm package may be included in this product:

- z3-solver@4.14.1
- z3-solver@4.15.0

This package contains the following license:

1 change: 1 addition & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
@@ -20,3 +20,4 @@ pnpm-debug.log*

# macOS-specific files
.DS_Store
src/content/docs/glossary.temp.json
2 changes: 1 addition & 1 deletion docs/astro.config.mjs
Original file line number Diff line number Diff line change
@@ -57,11 +57,11 @@ is at https://microsoft.github.io/genaiscript/reference/scripts.md
- you use the global types in genaiscript.d.ts are already loaded in the global context, no need to import them.
- save generated code in the "./genaisrc" folder with ".genai.mts" extension
`,
pageSeparator: "\n\n=|=|=|=|=|=\n\n",
minify: {
customSelectors: ["picture"],
},
promote: ["index*", "getting-started*", "!*/*"],
exclude: ["glossary", "slides"],
customSets: [
{
label: "Getting Started",
136 changes: 136 additions & 0 deletions docs/genaisrc/glossary.genai.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
// Build a consolidated glossary (max 100 terms) from docs/**/*.md,
// preserving any Markdown links in definitions.
script({
title: "build-glossary",
accept: "none",
})
const { output } = env
const docs = await workspace.readText("docs/dist/llms-full.txt")
const chunks = await tokenizers.chunk(docs, {
model: "openai:gpt-4o-mini",
chunkSize: 12000,
chunkOverlap: 1000,
})
const glossaryFilename = "docs/src/content/docs/glossary.json"
const originalGlossary = (await workspace.readJSON(glossaryFilename)) || {}
const entries = {}
for (const chunk of chunks) {
output.heading(3, `line ${chunk.lineStart}`)

const res = await runPrompt(
(ctx) => {
const fileRef = ctx.def("CHUNK", chunk)
ctx.$`
## Role

You are a documentation specialist, specializing in indexing and understanding documentation.

## Task
Extract all glossary terms and their definitions from ${fileRef}.
- **Preserve any Markdown-formatted links** (e.g., [text](url)) in definitions exactly as they appear.

## Existing glossary

${YAML.stringify(entries)}.

## Instructions

DO NOT repeat existing terms that are already in the glossary.
ONLY respond with new terms.
Term names are case sensitive and should be unique.
If there are no new terms, respond with an empty INI block.

## Output

Respond with name=definition pairs, one per line.

term=definition
term2=definition2
term3=definition3
...

Remember DO NOT repeat existing terms that are already in the glossary.
`
},
{
cache: "glossary",
model: "small",
label: `glossarify ${chunk.lineStart}`,
system: ["system.assistant"],
}
)
const entry = INI.parse(
res.fences.find((f) => f.language === "ini")?.content ||
parsers.unfence(res.text, "ini")
)
const oldGlossary = structuredClone(entries)
Object.assign(entries, entry)
output.diff(YAML.stringify(oldGlossary), YAML.stringify(entries))
await workspace.writeText(
path.changeext(glossaryFilename, ".temp.json"),
JSON.stringify(entries, null, 2)
)
}

const MAX = 256
const res = await runPrompt(
(ctx) => {
const oldTermsRef = ctx.defData("OLD_TERMS", originalGlossary)
const termsRef = ctx.defData("TERMS", entries)
ctx.$`
You may have at most ${MAX} glossary entries.
From ${termsRef}, consolidate and cap at ${MAX} entries.

- **Preserve any Markdown-formatted links** (e.g., [text](url)) in definitions exactly as they appear.
- Ensure proper spelling and grammar.

## Task:
1. Merge synonymous or highly related terms. If possible, reuse terms from the existing glossary ${oldTermsRef}.
2. Refine definitions for clarity and brevity. Fix spelling and grammar.
3. **Keep all Markdown links** (e.g., [text](url)) intact.
4. Return at most ${MAX} items.

## Output
Respond in INI format:
\`\`\`ini
term=definition
term2=definition2
...
\`\`\``
},
{
cache: "glossary",
systemSafety: false,
}
)
const glossary = INI.parse(
res.fences.find((f) => f.language === "ini")?.content ||
parsers.unfence(res.text, "ini")
)
output.heading(3, "Final glossary")
env.output.fence(glossary, "yaml")
await workspace.writeText(glossaryFilename, JSON.stringify(glossary, null, 2))

await workspace.writeText(
`docs/src/content/glossary.md`,
`---
title: Glossary
description: A glossary of terms used in the GenAI project.
keywords: glossary, terms, definitions
sidebar:
order: 200
---

This glossary provides definitions for terms used in the project.
Each term is linked to its corresponding section in the documentation for easy reference.

> This glossary is auto-generated from the source files.

## Terms

${Object.entries(glossary)
.sort((l, r) => l[0].toUpperCase().localeCompare(r[0].toUpperCase()))
.map(([term, definition]) => `- **${term}**: ${definition}`)
.join("\n")}
`
)
8 changes: 4 additions & 4 deletions docs/package.json
Original file line number Diff line number Diff line change
@@ -20,15 +20,15 @@
},
"devDependencies": {
"@astrojs/check": "^0.9.4",
"@astrojs/starlight": "^0.34.2",
"astro": "^5.7.10",
"@astrojs/starlight": "^0.34.3",
"astro": "^5.7.12",
"astro-embed": "^0.9.0",
"rehype-mermaid": "^3.0.0",
"starlight-blog": "^0.21.0",
"starlight-blog": "^0.23.0",
"starlight-links-validator": "^0.16.0",
"starlight-llms-txt": "^0.5.1",
"starlight-package-managers": "^0.11.0",
"zx": "^8.5.3"
"zx": "^8.5.4"
},
"resolutions": {
"marked": "15.0.8"
Loading
Loading