v0.5.0-beta.10: @prompd/test framework and Test Explorer UI#43
Merged
Conversation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update licenses.json to reflect @prompd/app@0.5.0-beta.6 (previously beta.5). Keeps the public license manifest in sync with the package version change.
Introduce a disk-based package cache and UI to browse/use cached packages. Adds CacheIpcRegistration (cache:list, cache:readFile, cache:download, cache:delete, cache:getPath, cache:fileTree) to manage ~/.prompd/cache, with ZIP download, extraction, path-safety and file-tree building. Exposes cache APIs in preload and updates electron.d.ts types. Adds RegistrySearchBar for package discovery on the WelcomeView (instant suggestions + debounced search) and a full PackageExplorerPanel to browse workspace/global/cache packages with actions (open, copy, inherit, install, delete, view on hub). Wire the new panel into App.tsx (auto-expand after download, pass packageSource through execution flow) and register CacheIpc in main.js. Several UI tweaks: ActivityBar label updated and analytics sync location noted. New/modified files include CacheIpcRegistration.js, RegistrySearchBar.tsx, PackageExplorerPanel.tsx, preload/main/App/electron.d.ts updates and related wiring.
- Registry search bar: descriptions from registry, contextual placeholder, hint text, chevron affordance, offline state, keyboard hints, wider layout - Registry search overlay (Ctrl+Shift+D): search packages from anywhere with tabs open, focuses WelcomeView search when no tabs - Package install: fix global install failing due to workspace path validation, pass global flag through to CLI - /install command: add --global/-g flag support, remove workspace requirement for global installs - Bump version to 0.5.0-beta.7 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Reject absolute ZIP entry paths during cache extraction (P1) - Use path separator boundary check for cache delete validation (P2) - Fix stale openRegistrySearch closure in menu IPC listener (P2) - Remove broken UpdateBanner component Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Override file: symlink with npm version for CI TypeScript checks - Exclude WysiwygEditor, WysiwygToolbar, and tiptap extension from tsc (depends on @tiptap packages not yet in package.json) - Stop deleting package-lock.json in CI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…vements Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pin @tiptap packages to 3.19.0 (3.20.x ships without dist/) - Add ts-nocheck to WIP WysiwygEditor/Toolbar (API compat issues) - Restore clean tsconfig.json without excludes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New @prompd/test package: TestRunner, TestParser, TestDiscovery, evaluators (NLP, Script, Prmd), reporters (Console, JSON, JUnit) - .test.prmd sidecar convention with auto-discovery, parameterized test cases, and LLM-based evaluation - Test Explorer sidebar panel with expandable tree view, run all/no-llm actions - Test Results bottom panel with collapsible suite groups, expandable test cards, and assertion details - Provider/Model selector for test execution independent from editor - IPC bridge (TestIpcRegistration) for Electron main process test execution - Monaco IntelliSense: skip validation warnings for .test.prmd built-in vars (prompt, response, params) - Context menu "Create Test File" on .prmd files in file explorer - Flask icon in editor header only for .test.prmd files - Status bar test run summary - Update @prompd/cli to ^0.5.0-beta.9 across all packages (frontend, backend, scheduler, test) - Add ModelCapabilities system to CLI provider layer (useMaxCompletionTokens, noTemperature, noSystemMessage) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…st.prmd - Update @prompd/test to ^0.5.0-beta.9 from npm (was file: symlink) - Re-include node_modules/@prompd/test in electron-builder files (excluded by !**/test/** glob) - Hide execute button in editor header for .test.prmd files - Bump @prompd/test version to 0.5.0-beta.9 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Bump all package versions to 0.5.0-beta.10 - Test explorer, evaluator, and store updates for beta.10 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update @prompd/cli to 0.5.0-beta.10 across frontend, backend, and test packages - Fix test file scaffold: add missing version field, fix comment syntax to HTML comments - Disable Design view for .test.prmd files (tiptap strips HTML comments) - Update prmd-test.svg icon border to match green theme - Fix typo: "defults" -> "defaults" in scaffold comments Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b220175dd2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… checks - Thread AbortSignal from IPC into TestRunner via options.signal for proper cancellation - Preserve evaluate field in parser for NLP, Script, and Prmd assertions - Script evaluator: exit 0=pass, exit 1=fail, other=error (was treating all non-zero as fail) - Add min_words/max_words to parser NLP check allowlist Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@prompd/testpackage — prompt testing and evaluation framework with.test.prmdsidecar convention, three evaluator types (NLP, Script, Prmd/LLM-as-judge), fail-fast ordering, and No LLM mode for CI@prompd/cliprovider layer handlingmax_completion_tokens,noTemperature,noSystemMessagefor OpenAI o1/o3/o4-mini and gpt-4.1 family@prompd/clibumped to 0.5.0-beta.10 across all packagesTest plan
.test.prmddiscovery and execution with NLP assertions (contains, min/max tokens/words, regex){{ prompt }},{{ response }},{{ params }}).prmd.test.prmd(prevents tiptap stripping HTML comments)max_completion_tokens, o1/o3 skip temperature and system messages🤖 Generated with Claude Code