Make expression comparison more thorough by benchristel · Pull Request #3503 · Khan/perseus

benchristel · 2026-04-16T15:56:59Z

Summary:

Content creators reported a bug in an expression widget where the correct
answer was -x / sqrt(1-x^2). You could put anything in the numerator and your
answer would (usually) be marked correct.

The bug happened because of the nondeterministic (and not very thorough) way
that we compared expressions. Previously, we selected 12 random variable
bindings and checked that the two expressions evaluated the same for all of
them. Some effort was made to cover different orders of magnitude, but simple
values like -1, 0, and 1 were not always checked.

This meant that for expressions with a restricted domain, like -x / sqrt(1 - x^2) (which is only defined in the interval [-1, 1]), we might check only
variable bindings for which the expression was not defined.

Expressions are now compared more thoroughly. Now we always check that the
expressions evaluate the same with all variables bound to -1, 0, and 1. We also
check more randomly-chosen values: 28 instead of 12.

Summary:

Issue: LEMS-4042

Test plan:

The expression in this article should be scored correctly:
https://www.khanacademy.org/devadmin/content/articles/arc-length-of-function-graphs-examples/x615e6443

…ough

… compared more thoroughly. Now we always check that the expressions evaluate the same with all variables bound to -1, 0, and 1. We also check more randomly-chosen values: 28 instead of 12.

claude

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

_{Tip: disable this comment in your organization's Code Review settings.}

github-actions · 2026-04-16T15:58:01Z

🗄️ Schema Change: No Changes ✅

github-actions · 2026-04-16T15:58:21Z

🛠️ Item Splitting: No Changes ✅

github-actions · 2026-04-16T15:58:34Z

Size Change: +100 B (+0.02%)

Total Size: 499 kB

📦 View Changed

Filename	Size	Change
`packages/kas/dist/es/index.js`	20.6 kB	+100 B (+0.49%)

ℹ️ View Unchanged

Filename	Size
`packages/keypad-context/dist/es/index.js`	1 kB
`packages/kmath/dist/es/index.js`	6.36 kB
`packages/math-input/dist/es/index.js`	98.5 kB
`packages/math-input/dist/es/strings.js`	1.61 kB
`packages/perseus-core/dist/es/index.item-splitting.js`	11.9 kB
`packages/perseus-core/dist/es/index.js`	25.1 kB
`packages/perseus-editor/dist/es/index.js`	102 kB
`packages/perseus-linter/dist/es/index.js`	9.3 kB
`packages/perseus-score/dist/es/index.js`	9.7 kB
`packages/perseus-utils/dist/es/index.js`	403 B
`packages/perseus/dist/es/index.js`	196 kB
`packages/perseus/dist/es/strings.js`	8.27 kB
`packages/pure-markdown/dist/es/index.js`	1.39 kB
`packages/simple-markdown/dist/es/index.js`	6.71 kB

_{compressed-size-action}

github-actions · 2026-04-16T15:59:22Z

npm Snapshot: Published

Good news!! We've packaged up the latest commit from this PR (6835eb5) and published it to npm. You
can install it using the tag PR3503.

Example:

pnpm add @khanacademy/perseus@PR3503

If you are working in Khan Academy's frontend, you can run the below command.

./dev/tools/bump_perseus_version.ts -t PR3503

If you are working in Khan Academy's webapp, you can run the below command.

./dev/tools/bump_perseus_version.js -t PR3503

SonicScrewdriver

From a code standpoint, I think this looks like a great improvement and all the tests pass. Thank you for including such detailed comments!

It wouldn't be a bad plan to get the Math Content team to double check that everything looks right, but I'm happy to approve myself!

SonicScrewdriver · 2026-04-21T21:53:43Z

-        expect("sin^2(x)+cos^2(x)").toEqualExpr("x/x");
-        expect("y = sin^2(x)+cos^2(x)").toEqualExpr("y = x/x");
+        expect("sin^2(x)+cos^2(x)").toEqualExpr("1");
+        expect("y = sin^2(x)+cos^2(x)").toEqualExpr("y = 1");


This seems much clearer!

SonicScrewdriver · 2026-04-21T21:57:42Z

+        const variableValueGenerators = [
+            // Always check x = -1, x = 0, and x = 1.
+            // Requested by content creators:
+            // https://khanacademy.slack.com/archives/CJDRXTGQ7/p1776180198083239?thread_ts=1776096483.309839&cid=CJDRXTGQ7


This was honestly very useful, but I wonder if we want to permanently include our slack link in our open source repo. Maybe we could add the ticket ID and a comment on the ticket with the slack link?

This is truly just a nit / musing, so feel free to disregard at your discretion.

SonicScrewdriver · 2026-04-21T21:58:27Z

+}
+
+function intGenerator(min: number, max: number): () => number {
+    return () => _.random(min, max);


I'd love it if we could clear out some of the underscore while we're in here, but that juice might not be worth the squeeze right now.

@benchristel

This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @khanacademy/perseus-core@25.0.0 ### Major Changes - [#3511](#3511) [`15b0193db5`](15b0193) Thanks [@benchristel](https://github.com/benchristel)! - Remove unused `static` field from PerseusCSProgramWidgetOptions. Callers should update test data that constructs `PerseusCSProgramWidgetOptions` to remove the static field. ### Minor Changes - [#3441](#3441) [`de45286302`](de45286) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of new Vector graph and subcomponents. - [#3433](#3433) [`b4bb6e2f42`](b4bb6e2) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of initial types and stubs for Vector graph ### Patch Changes - [#3434](#3434) [`de2dda0258`](de2dda0) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Implementation of state management logic for new Vector graph - Updated dependencies \[[`a5b9105c28`](a5b9105)]: - @khanacademy/kas@2.2.2 ## @khanacademy/perseus@77.3.0 ### Minor Changes - [#3441](#3441) [`de45286302`](de45286) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of new Vector graph and subcomponents. - [#3433](#3433) [`b4bb6e2f42`](b4bb6e2) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of initial types and stubs for Vector graph - [#3494](#3494) [`8fb79829d0`](8fb7982) Thanks [@ivyolamit](https://github.com/ivyolamit)! - Interactive Graph: change asymptote line to dashed for both exponential and logarithm based on user feedback ### Patch Changes - [#3523](#3523) [`c89cdbe2aa`](c89cdbe) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Ensure the default coords for Exponential and Logarithm are slightly further away from the asymptote. - [#3508](#3508) [`16f7f77ba1`](16f7f77) Thanks [@nishasy](https://github.com/nishasy)! - [Image] | (CX) | Give explore modal launcher a label saying it has description - [#3496](#3496) [`4d923417cd`](4d92341) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Add fallback label to Numeric Inputs and a linter warning for missing labels in the editor. - [#3511](#3511) [`15b0193db5`](15b0193) Thanks [@benchristel](https://github.com/benchristel)! - Remove unused `static` field from PerseusCSProgramWidgetOptions. Callers should update test data that constructs `PerseusCSProgramWidgetOptions` to remove the static field. - [#3504](#3504) [`b8178b52e7`](b8178b5) Thanks [@nishasy](https://github.com/nishasy)! - [Image] | (a11y) | Add aria-describedby to Explore Image modal - [#3488](#3488) [`3abc5e8277`](3abc5e8) Thanks [@mark-fitzgerald](https://github.com/mark-fitzgerald)! - [Free Response] Add visual regression tests - [#3493](#3493) [`11742c2cff`](11742c2) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Implementing bug fix for jumping MovableLines in the Correct Answer graph in the editor - [#3483](#3483) [`7794943ec7`](7794943) Thanks [@nishasy](https://github.com/nishasy)! - [ColorSync] Numeric Input - update visual regression tests - [#3434](#3434) [`de2dda0258`](de2dda0) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Implementation of state management logic for new Vector graph - Updated dependencies \[[`de45286302`](de45286), [`4d923417cd`](4d92341), [`15b0193db5`](15b0193), [`b4bb6e2f42`](b4bb6e2), [`d3ef4dbcc2`](d3ef4db), [`a5b9105c28`](a5b9105), [`de2dda0258`](de2dda0)]: - @khanacademy/perseus-core@25.0.0 - @khanacademy/perseus-linter@4.9.5 - @khanacademy/perseus-score@8.7.0 - @khanacademy/kas@2.2.2 - @khanacademy/keypad-context@3.2.44 - @khanacademy/kmath@2.4.2 - @khanacademy/math-input@26.4.15 ## @khanacademy/perseus-editor@30.4.0 ### Minor Changes - [#3441](#3441) [`de45286302`](de45286) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of new Vector graph and subcomponents. - [#3443](#3443) [`a3396604a7`](a339660) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of new Vector Graph Editor - [#3433](#3433) [`b4bb6e2f42`](b4bb6e2) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Creation of initial types and stubs for Vector graph - [#3492](#3492) [`883133a28f`](883133a) Thanks [@jeremywiebe](https://github.com/jeremywiebe)! - Add preview data sanitizer to strip non-serializable values before postMessage ### Patch Changes - [#3522](#3522) [`19911cc966`](19911cc) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Label absolute-value graph start coordinates as "Vertex" and "Arm" in the editor instead of "Point 1" and "Point 2". - [#3505](#3505) [`1ab914fc41`](1ab914f) Thanks [@nishasy](https://github.com/nishasy)! - [Image] | (CX) | Add warning for large images - [#3530](#3530) [`b5e918e8b3`](b5e918e) Thanks [@nishasy](https://github.com/nishasy)! - [Image] Update copy of recalculate button in editor - [#3434](#3434) [`de2dda0258`](de2dda0) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Implementation of state management logic for new Vector graph - Updated dependencies \[[`de45286302`](de45286), [`c89cdbe2aa`](c89cdbe), [`16f7f77ba1`](16f7f77), [`4d923417cd`](4d92341), [`15b0193db5`](15b0193), [`b8178b52e7`](b8178b5), [`b4bb6e2f42`](b4bb6e2), [`d3ef4dbcc2`](d3ef4db), [`3abc5e8277`](3abc5e8), [`a5b9105c28`](a5b9105), [`11742c2cff`](11742c2), [`7794943ec7`](7794943), [`de2dda0258`](de2dda0), [`8fb79829d0`](8fb7982)]: - @khanacademy/perseus@77.3.0 - @khanacademy/perseus-core@25.0.0 - @khanacademy/perseus-linter@4.9.5 - @khanacademy/perseus-score@8.7.0 - @khanacademy/kas@2.2.2 - @khanacademy/keypad-context@3.2.44 - @khanacademy/kmath@2.4.2 - @khanacademy/math-input@26.4.15 ## @khanacademy/perseus-score@8.7.0 ### Minor Changes - [#3442](#3442) [`d3ef4dbcc2`](d3ef4db) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Added ability to score Vector Interactive Graphs ### Patch Changes - Updated dependencies \[[`de45286302`](de45286), [`15b0193db5`](15b0193), [`b4bb6e2f42`](b4bb6e2), [`a5b9105c28`](a5b9105), [`de2dda0258`](de2dda0)]: - @khanacademy/perseus-core@25.0.0 - @khanacademy/kas@2.2.2 - @khanacademy/kmath@2.4.2 ## @khanacademy/kas@2.2.2 ### Patch Changes - [#3503](#3503) [`a5b9105c28`](a5b9105) Thanks [@benchristel](https://github.com/benchristel)! - Expressions are now compared more thoroughly. Now we always check that the expressions evaluate the same with all variables bound to -1, 0, and 1. We also check more randomly-chosen values: 28 instead of 12. ## @khanacademy/keypad-context@3.2.44 ### Patch Changes - Updated dependencies \[[`de45286302`](de45286), [`15b0193db5`](15b0193), [`b4bb6e2f42`](b4bb6e2), [`de2dda0258`](de2dda0)]: - @khanacademy/perseus-core@25.0.0 ## @khanacademy/kmath@2.4.2 ### Patch Changes - Updated dependencies \[[`de45286302`](de45286), [`15b0193db5`](15b0193), [`b4bb6e2f42`](b4bb6e2), [`de2dda0258`](de2dda0)]: - @khanacademy/perseus-core@25.0.0 ## @khanacademy/math-input@26.4.15 ### Patch Changes - Updated dependencies \[[`de45286302`](de45286), [`15b0193db5`](15b0193), [`b4bb6e2f42`](b4bb6e2), [`de2dda0258`](de2dda0)]: - @khanacademy/perseus-core@25.0.0 - @khanacademy/keypad-context@3.2.44 ## @khanacademy/perseus-linter@4.9.5 ### Patch Changes - [#3496](#3496) [`4d923417cd`](4d92341) Thanks [@SonicScrewdriver](https://github.com/SonicScrewdriver)! - Add fallback label to Numeric Inputs and a linter warning for missing labels in the editor. - Updated dependencies \[[`de45286302`](de45286), [`15b0193db5`](15b0193), [`b4bb6e2f42`](b4bb6e2), [`a5b9105c28`](a5b9105), [`de2dda0258`](de2dda0)]: - @khanacademy/perseus-core@25.0.0 - @khanacademy/kas@2.2.2 - @khanacademy/kmath@2.4.2

benchristel added 2 commits April 16, 2026 08:44

[benc/expression-widget-scoring] Make expression comparison more thor…

1e4328f

…ough

[benc/expression-widget-scoring] docs(changeset): Expressions are now…

179f527

… compared more thoroughly. Now we always check that the expressions evaluate the same with all variables bound to -1, 0, and 1. We also check more randomly-chosen values: 28 instead of 12.

benchristel self-assigned this Apr 16, 2026

benchristel requested a review from SonicScrewdriver April 16, 2026 15:57

claude Bot reviewed Apr 16, 2026

View reviewed changes

benchristel added the olc-5.0.f97a8 label Apr 16, 2026

[benc/expression-widget-scoring] Refactor

821b13d

benchristel requested a review from catandthemachines April 16, 2026 19:14

[benc/expression-widget-scoring] Fix formatting

6835eb5

SonicScrewdriver approved these changes Apr 21, 2026

View reviewed changes

benchristel merged commit a5b9105 into main Apr 27, 2026
11 checks passed

benchristel deleted the benc/expression-widget-scoring branch April 27, 2026 16:44

khan-actions-bot mentioned this pull request Apr 24, 2026

Version Packages #3498

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make expression comparison more thorough#3503

Make expression comparison more thorough#3503
benchristel merged 4 commits intomainfrom
benc/expression-widget-scoring

benchristel commented Apr 16, 2026

Uh oh!

claude Bot left a comment

Uh oh!

github-actions Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

SonicScrewdriver left a comment

Uh oh!

SonicScrewdriver Apr 21, 2026

Uh oh!

SonicScrewdriver Apr 21, 2026

Uh oh!

SonicScrewdriver Apr 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

benchristel commented Apr 16, 2026

Summary:

Summary:

Test plan:

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

github-actions Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🗄️ Schema Change: No Changes ✅

Uh oh!

github-actions Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠️ Item Splitting: No Changes ✅

Uh oh!

github-actions Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

npm Snapshot: Published

Uh oh!

SonicScrewdriver left a comment

Choose a reason for hiding this comment

Uh oh!

SonicScrewdriver Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

SonicScrewdriver Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

SonicScrewdriver Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Apr 16, 2026 •

edited

Loading

github-actions Bot commented Apr 16, 2026 •

edited

Loading

github-actions Bot commented Apr 16, 2026 •

edited

Loading

github-actions Bot commented Apr 16, 2026 •

edited

Loading