-
Notifications
You must be signed in to change notification settings - Fork 7
Issues: microsoft/promptpex
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Using paper script overview.csv and readme.md contains a couple of issues
bug
Something isn't working
#120
opened Apr 4, 2025 by
bzorn
Update readme.md in evals directory when processing multiple prompt inputs with paper script
bug
Something isn't working
#119
opened Apr 4, 2025 by
bzorn
Make non-compliance eval results graphical in evals/readme.md
documentation
Improvements or additions to documentation
enhancement
New feature or request
#117
opened Apr 2, 2025 by
bzorn
Regenerate tests that describe the test content instead of creating concrete instance
enhancement
New feature or request
#106
opened Mar 28, 2025 by
bzorn
Add error message, checking, and logging when processing prompt file on command line
enhancement
New feature or request
#104
opened Mar 28, 2025 by
bzorn
Labels overlap in the bar graph of component timing at end of run in UI
bug
Something isn't working
#103
opened Mar 27, 2025 by
bzorn
Using cache=true did not prevent calls to the models
bug
Something isn't working
#102
opened Mar 27, 2025 by
bzorn
Using ollama:deepseek-r1:32b as the model-under-test can hang the UI
bug
Something isn't working
#101
opened Mar 27, 2025 by
bzorn
Current version fails to run and record baseline compliance statistics
bug
Something isn't working
#99
opened Mar 27, 2025 by
bzorn
Provide way to give example inputs and outputs to promptpex
enhancement
New feature or request
#95
opened Mar 21, 2025 by
bzorn
Observed very concise test generation when a paragraph of test was specified
enhancement
New feature or request
#92
opened Mar 20, 2025 by
bzorn
Add timestamps for starting and ending processing phases
enhancement
New feature or request
#90
opened Mar 19, 2025 by
bzorn
Generation execution may lead to 408 causing script termination
bug
Something isn't working
#87
opened Mar 18, 2025 by
bzorn
Seeing empty entries in the "expected output" column in the generated test file
bug
Something isn't working
#80
opened Mar 11, 2025 by
bzorn
Produce non-compliance results as table/figure for single prompt with multiple models in readme.md file
#77
opened Mar 10, 2025 by
bzorn
Change effort of GENAISCRIPT_DEBUG=1 so that it only generates 2 tests in test.csv to make debugging a sample faster
enhancement
New feature or request
#74
opened Mar 7, 2025 by
bzorn
Tests are run where the test input is an empty string
bug
Something isn't working
#73
opened Mar 7, 2025 by
bzorn
Add method to roll up evaluation results across multiple test prompts
documentation
Improvements or additions to documentation
enhancement
New feature or request
#71
opened Mar 7, 2025 by
bzorn
When logging execution times, include more information in the log record
enhancement
New feature or request
#67
opened Mar 7, 2025 by
bzorn
support output specification in prompty
enhancement
New feature or request
#51
opened Feb 20, 2025 by
pelikhan
Create a documentation markdown file indicating what models we've tried and whether promptpex works with them
documentation
Improvements or additions to documentation
#48
opened Feb 19, 2025 by
bzorn
Add timestamps for major sections (such as rule gen, test gen, test run, and test eval
enhancement
New feature or request
#43
opened Feb 14, 2025 by
bzorn
Previous Next
ProTip!
Mix and match filters to narrow down what you’re looking for.