-
Notifications
You must be signed in to change notification settings - Fork 2
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
temperature and maxTokens are plumbed through the provider but cannot be set by a suite
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomersStatus: Open.#17 In AgentPostmortem/Evalgate;Cost is one hardcoded rate per provider, applied to every model and both token directions
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#16 In AgentPostmortem/Evalgate;Provider calls have no timeout and no retry, so a 429 fails the build
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#15 In AgentPostmortem/Evalgate;One provider or scorer error discards the entire run
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#14 In AgentPostmortem/Evalgate;Deleting a failing case makes the gate green: removed cases are never regressions
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#13 In AgentPostmortem/Evalgate;exact-match always collapses internal whitespace, and trim: false does not turn it off
good first issueGood for newcomersGood for newcomersStatus: Open.#9 In AgentPostmortem/Evalgate;An invalid pattern in a JSON schema throws and kills the whole run
bugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomersStatus: Open.#8 In AgentPostmortem/Evalgate;cost scorer passes with a perfect score when the provider reports no cost
bugSomething isn't workingSomething isn't workingStatus: Open.#7 In AgentPostmortem/Evalgate;Bug: a run with zero cases reports passed and exits 0, so the gate silently disables itself
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededStatus: Open.#6 In AgentPostmortem/Evalgate;--tags and --concurrency are ignored by baseline and compare
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededStatus: Open.#5 In AgentPostmortem/Evalgate;Docs: --concurrency, --junit and --degrade are missing from evalgate --help
documentationImprovements or additions to documentationImprovements or additions to documentationgood first issueGood for newcomersGood for newcomersStatus: Open.#4 In AgentPostmortem/Evalgate;evalgate list hardcodes the provider names instead of asking the registry
claimedSomeone asked first and is working on thisSomeone asked first and is working on thisenhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomersStatus: Open.#3 In AgentPostmortem/Evalgate;