builtins.substring: fix int overflow

[yorickvP]

repro: builtins.substring 4294967296 1 "umu"

[yorickvP]

cc @puckipedia

[thufschmitt]

Care to add a little testcase for it? (here is probably the best place to do it)

Looks good otherwise 👍

[edolstra]

This causes a compilation failure on macOS. Wouldn't it make more sense to cast it to size_t, since we're comparing to s->size()?

[yorickvP]

That's annoying, a cast to size_t would overflow on 32-bit systems. I've now added a forceIntChecked function that uses boost::numeric_cast to do a checked cast to size_t where applicable in the primops.

[thufschmitt]

The change looks good overall, but it feels like the error messages are slightly worse than what they used to be (when evaluating something like builtins.genList (x: x) (-1) for example).

It might be nicer to make forceIntChecked return an optional or equivalent rather than throwing, and keeping the error handling on the client side where we can get some better messages

[yorickvP]

I spent a while converting to a std::optional, but:

• it just duplicates a lot of the error handling, since you still want to add the context to out-of-bounds errors
• it doesn't separate out the negative cases from the overflow cases, meaning errors are still less clear

I've added a commit to separate overflows from underflows.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-03-03-nix-team-meeting-minutes-37/25998/1

[fricklerhandwerk]

The additions to the error messages are sensible, very good. Only added suggestions on phrasing, but this is cherry on top.

[fricklerhandwerk]

I think it's integer values that can be high or low, but integers themselves can be large or small.

That's the only thing I see that could be improved immediately. Ideally though we would determine the bound and do the fully explicit thing saying something like: integer %1 too small, must be at least %2

Or, depending on your taste, and I don't have an opinion on this: The value %1 is too low, it must be at least %2
Using "value" and avoiding "integer" allows reuse of that procedure for other numeric type.

We may also use the well-known phrase "out of bounds" so it can be immediately recognised for that class of error: out of bounds: the value %1 is too low, it must be at least %2

Pick what feels right to you.

https://www.boost.org/doc/libs/1_34_1/libs/numeric/conversion/doc/bounds.html

[thufschmitt]

On the topic of error messages, I'd just like to point out a marginal regression wrt what we have on master (I don't have a strong opinion as to whether that should be a blocker or not, just want to make sure that's not an overlook):

$ # On master                                   
$ nix eval --expr 'builtins.genList (x: x) (-1)'
error:
       … while calling the 'genList' builtin

         at «string»:1:1:

            1| builtins.genList (x: x) (-1)
             | ^

       error: cannot create list of size -1
$ # On this branch
$ nix run github:yorickvP/nix/fix-substring-int -- eval --expr 'builtins.genList (x: x) (-1)'
error:
       … while calling the 'genList' builtin

         at «string»:1:1:

            1| builtins.genList (x: x) (-1)
             | ^

       … while evaluating the second argument passed to builtins.genList

         at «none»:0: (source not available)

       error: integer -1 is too low

[roberth]

Suggested change

	NixInt len_ = state.forceInt(*args[1], pos, "while evaluating the second argument (the substring length) passed to builtins.substring");
	NixInt len_ = state.forceInt(*args[1], pos, "while evaluating the second argument (the start offset) passed to builtins.substring");

[roberth]

Are you still interested to work on this problem?
The code has changed quite a bit due to error message improvements. We now also have a type wrapper for checked arithmetic in src/libutil/checked-arithmetic.hh.
I'd be happy to review it.