Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Apr 6, 2026

• [codex] Pin GitHub Actions workflow references (#1637)

  Show description for dbb1a20
  hintz-openai

  authored
  dbb1a20

  Copy full SHA for dbb1a20

Commits on Nov 3, 2025

• Remove incontext_rl suite with defunct dependencies (#1605)

  Show description for 4bfc1f5
  maxb-openai

  authored
  4bfc1f5

  Copy full SHA for 4bfc1f5

Commits on Dec 18, 2024

• Updating readme to link to OpenAI hosted evals experience (#1572)

  Show description for cdb8ce9
  dmitry-openai

  authored
  cdb8ce9

  Copy full SHA for cdb8ce9

Commits on Sep 30, 2024

• 20240930 steven exception handling usage tokens (#1560)

  Show description for a32c982
  sjadler2004

  authored
  a32c982

  Copy full SHA for a32c982

Commits on Jul 13, 2024

• [eval] Add IMO problems with exact answers (#1528)

  

  Justin W. Lin

  authored
  234bcde

  Copy full SHA for 234bcde

Commits on May 1, 2024

• Release 3.0.1 (#1525)

  Show description for d3dc890
  etr2460

  authored
  d3dc890

  Copy full SHA for d3dc890

• Make the torch dep optional (#1524)

  Show description for 1d3f11c
  etr2460

  authored
  1d3f11c

  Copy full SHA for 1d3f11c

Commits on Apr 17, 2024

• Release 3.0.0 (#1520)

  Show description for 778caa6
  etr2460

  authored
  778caa6

  Copy full SHA for 778caa6

• Unpin dependencies (#1519)

  hauntsaninja

  authored
  518a9a8

  Copy full SHA for 518a9a8

Commits on Apr 5, 2024

• Remove citation prediction eval (#1512)

  Show description for c124f98
  ojaffe

  authored
  c124f98

  Copy full SHA for c124f98

• Allow for evals with no args (#1517)

  Show description for 4ed2f6f
  thesofakillers

  authored
  4ed2f6f

  Copy full SHA for 4ed2f6f

Commits on Apr 4, 2024

• Relax version constraint for `playwright` module (#1516)

  Show description for 20de8c5
  danesherbs

  authored
  20de8c5

  Copy full SHA for 20de8c5

Commits on Apr 2, 2024

• Switch from pyzstd to zstandard (#1514)

  Show description for b5853eb
  josnyder-2

  authored
  b5853eb

  Copy full SHA for b5853eb

Commits on Mar 28, 2024

• Updates on existing solvers and bugged tool eval (#1506)

  Show description for 2420c62
  

  4 peopleauthored
  2420c62

  Copy full SHA for 2420c62

Commits on Mar 27, 2024

• Update ReadMe with New Cookbook link (#1507)

  Show description for d9d2f5f
  royziv11

  authored
  d9d2f5f

  Copy full SHA for d9d2f5f

Commits on Mar 26, 2024

• Add Gemini Solver (#1503)

  Show description for 5a92ac3
  

  ojaffe

  and

  JunShern

  authored
  5a92ac3

  Copy full SHA for 5a92ac3

• Unified create_retrying for all solvers (#1501)

  Show description for 150dcb9
  ojaffe

  authored
  150dcb9

  Copy full SHA for 150dcb9

Commits on Mar 25, 2024

• Add info about logging and link to logviz (#1480)

  Show description for ac44aae
  JunShern

  authored
  ac44aae

  Copy full SHA for ac44aae

• Log model and usage stats in `record.sampling` (#1449)

  Show description for 9b2e1b1
  JunShern

  authored
  9b2e1b1

  Copy full SHA for 9b2e1b1

• Address sporadic hanging of evals on certain samples (#1482)

  Show description for bfe3925
  thesofakillers

  authored
  bfe3925

  Copy full SHA for bfe3925

Commits on Mar 22, 2024

• TogetherSolver (#1502)

  Show description for 5805c20
  

  3 peopleauthored
  5805c20

  Copy full SHA for 5805c20

Commits on Mar 21, 2024

• AnthropicSolver (#1498)

  Show description for e30e141
  thesofakillers

  authored
  e30e141

  Copy full SHA for e30e141

• Add Human-Relative MLAgentBench (#1496)

  Show description for 4f97ce6
  danesherbs

  authored
  4f97ce6

  Copy full SHA for 4f97ce6

• Add Multi-Step Web Tasks (#1500)

  Show description for 5b84993
  

  danesherbs

  and

  ianmckenzie-oai

  authored
  5b84993

  Copy full SHA for 5b84993

Commits on Mar 19, 2024

• Add In-Context RL eval (#1491)

  Show description for ff994b5
  james-aung-aisi

  authored
  ff994b5

  Copy full SHA for ff994b5

• Add Function Deduction eval (#1492)

  Show description for dfeaac4
  

  james-aung-aisi

  and

  johny-b

  authored
  dfeaac4

  Copy full SHA for dfeaac4

• Identifying Variables Eval (#1488)

  Show description for c207dba
  thesofakillers

  authored
  c207dba

  Copy full SHA for c207dba

• Track the Stat Eval (#1489)

  Show description for 99bfada
  thesofakillers

  authored
  99bfada

  Copy full SHA for 99bfada

• Already Said That Eval (#1490)

  Show description for baa12d0
  thesofakillers

  authored
  baa12d0

  Copy full SHA for baa12d0

• Add 20 questions eval (#1499)

  Show description for bd1736e
  inwaves

  authored
  bd1736e

  Copy full SHA for bd1736e

• Add skill acquisition eval (#1497)

  Show description for 76a9f4e
  inwaves

  authored
  76a9f4e

  Copy full SHA for 76a9f4e

• Error Recovery Eval (#1485)

  Show description for 80ac60d
  

  ojaffe

  and

  ianmckenzie-oai

  authored
  80ac60d

  Copy full SHA for 80ac60d

• Can't Do That Anymore Eval (#1487)

  Show description for f72afb9
  ojaffe

  authored
  f72afb9

  Copy full SHA for f72afb9

• Bugged Tools Eval (#1486)

  Show description for ad377e4
  ojaffe

  authored
  ad377e4

  Copy full SHA for ad377e4

Commits on Mar 13, 2024

• Updates on existing evals; readmes; solvers (#1483)

  Show description for 11c30b2
  ojaffe

  authored
  11c30b2

  Copy full SHA for 11c30b2