Skip to content

Fix UInt8/UInt16 masking: switch scrutinee, ETernary, ECast - #741

Merged
mtzguido merged 1 commit into
FStarLang:masterfrom
mtzguido:707
May 20, 2026
Merged

Fix UInt8/UInt16 masking: switch scrutinee, ETernary, ECast#741
mtzguido merged 1 commit into
FStarLang:masterfrom
mtzguido:707

Conversation

@mtzguido

@mtzguido mtzguido commented May 3, 2026

Copy link
Copy Markdown
Member

Three issues:
1- The ESwitch case passed the scrutinee through mk_expr, bypassing
mk_arith masking.
2- mk_arith's catch-all marked ETernary as atomic, hiding non-atomic
arithmetic branches from consumers (switch, comparisons).
3- ECast applied the cast to the widened uint32 value without masking,
leaking overflow bits on widening casts.

Fix: for UInt8/UInt16 switch scrutinees, route through mk_arith and mask non-atomic results. Add an ETernary case to mk_arith that recurses into both branches, returning is_atomic only when both are atomic. For ECast, route UInt8/UInt16 sources through mk_arith and mask before casting.

Add SmallIntSwitch.fst, SmallIntTernary.fst, SmallIntCast.fst regression tests.

Fixes #707

--

Hi Jonathan, this is coauthored with Claude. It fixes #707 (which is legit) and a few other variants. The three tests fail before this patch. I'm not sure the style of the patch is that pleasant, since it seems easy to forget about this mk_arith dance and reintroduce variants of this bug later.

@protz protz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks.

Thinking more about my comment regarding the ternary operator, e.g. (false ? f () : 1) >> 1. I think it should be have like a binary operator and I think w1 || w2 is correct.

Comment thread lib/AstToCStar.ml
that every subexpression operates over unsigned int until the final
cast, or until a mask is needed to preserve semantics. *)
mk_expr env false false e, true, false (* C++: a constant that is wider than the intended type, but in an initializer list, is ok *)
| ETernary (c, t, f) ->

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you expecting things of the form return e1 + e2 ? e3 : e4?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean with an operation in the condition? I don't think so, they must have bool type.

Comment thread lib/AstToCStar.ml
let c = mk_expr env false false c in
let t, a1, w1 = mk_arith env t in
let f, a2, w2 = mk_arith env f in
CStar.Ternary (c, t, f), a1 && a2, w1 || w2

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if one of the branches was widened but not the other, shouldn't we widen both? feels like we should assert that w1 = w2 or do something

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a conservative choice so I think it is correct

Comment thread lib/AstToCStar.ml
CStar.Op (K.op_of_poly_comp c)
| ECast (e, t) ->
CStar.Cast (mk_expr env false e, mk_type env t)
(* When the source is UInt8/UInt16 and contains arithmetic, the widened

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@mtzguido

mtzguido commented May 9, 2026

Copy link
Copy Markdown
Member Author

I have to say I'm a bit confused about the widening. It's only relevant for C++ initializers if I understand correctly? And it seems like an addition of u8 constants is not considered widened, but the addition would be promoted to int?

This generates a bad diff in Eurydice, I'll look into it.

tests>  size_t where_clauses_simple_add_81_6c(Eurydice_arr_96 x, size_t y)
tests>  {
tests> -  return (size_t)x.data[0U] + y + (size_t)1U;
tests> +  return (size_t)(uint32_t)x.data[0U] + y + (size_t)1U;
tests>  }

I wonder if a different design for all this would be more desirable, actually: keep all casts explicit during AST->C*, and then do a pass to remove the superfluous ones. With enough rules, I think we could remove all the casts in the expression above.

@protz

protz commented May 19, 2026

Copy link
Copy Markdown
Collaborator

Sorry I forgot to catch up on this one.

I have to say I'm a bit confused about the widening. It's only relevant for C++ initializers if I understand correctly?

Yes it happens when Eurydice compiles in C++ mode and relies on a C++ gadget for the construction of tagged unions. I'm re-reading https://en.cppreference.com/cpp/language/list_initialization and I can't seem to either remember or reproduce the issue, but I know that at the time it was necessary. Maybe this is it:

#include <inttypes.h>

class S {
  uint32_t x;
  public:
  S(uint32_t x) :x(x) {}
};

int main() {
  uint64_t x = 0;
  auto y = S { x }; // ERROR: narrowing conversion not allowed
}

I think the logic is that if this is directly under a context that sets the type (like in an initializer list), there's no need to cast to the destination type, UNLESS we are in that particular C++ case.

And it seems like an addition of u8 constants is not considered widened, but the addition would be promoted to int?

I think the intent is that yes, if this is an addition, it works differently? The example above succeeds with 1ULL (or 1ULL+1) in the initializer list.

This generates a bad diff in Eurydice, I'll look into it.

Thanks

I wonder if a different design for all this would be more desirable, actually: keep all casts explicit during AST->C*, and then do a pass to remove the superfluous ones. With enough rules, I think we could remove all the casts in the expression above.

Yes, I'm willing to be convinced? It might be easier to prove the correctness via rewriting rules (rather than with the on-the-fly transformation).

@mtzguido

Copy link
Copy Markdown
Member Author

Thanks for the comments, that helps. I'll probably add some tests for that.

I wonder if a different design for all this would be more desirable, actually: keep all casts explicit during AST->C*, and then do a pass to remove the superfluous ones. With enough rules, I think we could remove all the casts in the expression above.

Yes, I'm willing to be convinced? It might be easier to prove the correctness via rewriting rules (rather than with the on-the-fly transformation).

I did start doing this, with encouraging results, but did not finish yet. I could probably get it done in the next week or so. Would you like to merge this PR meanwhile or should we wait for the alternative?

Three issues:
1- The ESwitch case passed the scrutinee through mk_expr, bypassing
   mk_arith masking.
2- mk_arith's catch-all marked ETernary as atomic, hiding non-atomic
   arithmetic branches from consumers (switch, comparisons).
3- ECast applied the cast to the widened uint32 value without masking,
   leaking overflow bits on widening casts.

Fix: for UInt8/UInt16 switch scrutinees, route through mk_arith and mask
non-atomic results. Add an ETernary case to mk_arith that recurses into
both branches, returning is_atomic only when both are atomic. For ECast,
route UInt8/UInt16 sources through mk_arith and mask before casting.

Add SmallIntSwitch.fst, SmallIntTernary.fst, SmallIntCast.fst regression
tests.

Fixes FStarLang#707

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@protz

protz commented May 19, 2026

Copy link
Copy Markdown
Collaborator

Let's just merge this on the basis that we don't want to leave something incorrect in the tree if we have a fix for it.

@mtzguido

Copy link
Copy Markdown
Member Author

AeneasVerif/eurydice#407

protz pushed a commit to AeneasVerif/eurydice that referenced this pull request May 19, 2026
Generated code has a few more casts. This will be improved in a follow
up karamel patch.
protz added a commit to AeneasVerif/eurydice that referenced this pull request May 19, 2026
@mtzguido
mtzguido merged commit 3611ae4 into FStarLang:master May 20, 2026
29 of 31 checks passed
@mtzguido
mtzguido deleted the 707 branch May 20, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DeepTest] UInt8/UInt16 switch scrutinee not masked after wrapping arithmetic

2 participants