implement array slicing

[mikedanese]

Incomplete pass at #101

$ cat exp.jsonnet 
local arr = std.range(0, 10);
arr[5:7]
$ ./jsonnet exp.jsonnet
[
   5,
   6
]

[mikedanese]

Just pushed support for [:4] and [4:] notations

$  jsonnet git:(slice) ✗ cat exp.jsonnet 
local arr = std.range(0, 10);
{
    foo: arr[5:7],
    bar: arr[:7],
    baz: arr[5:],
}
$  jsonnet git:(slice) ✗ ./jsonnet exp.jsonnet 
{
   "bar": [
      0,
      1,
      2,
      3,
      4,
      5,
      6
   ],
   "baz": [
      5,
      6,
      7,
      8,
      9,
      10
   ],
   "foo": [
      5,
      6
   ]
}

[mikedanese]

This doesn't support step parameter yet.

[sparkprime]

Is there an advantage to having this as a separate AST if it can only be used inside the [ ] of an Index AST?

[sparkprime]

Basically you could extend Index to add the last and step ASTs which are null if they're not provided.

[mikedanese]

Done

[sparkprime]

Good work! Sorry it took me a few days to get around to reviewing it.

[sparkprime]

This should also work on strings and fail explicitly if arr is not a string or array type. There is already str.substr but its semantics aren't right (see comment below). So we should deprecate str.substr() and use this instead.

[mikedanese]

Ok, I need to implement the python behavior in std.slice now.

[mikedanese]

I've implemented python behavior for all positive value (index, end, step) inputs. I think it's reasonable to support negative inputs later on.

[mikedanese]

@sparkprime This is breaking with:

FAIL (exit code): slice.sugar.jsonnet
This run's output:
STATIC ERROR: slice.sugar.jsonnet:32:21-22: Not a unary operator: ::

I'll investigate what is going on but it's not immediately obvious to me.

[sparkprime]

Oh this is bad unfortunately -- { x:: 3 } is lexed as the single token ::

[sparkprime]

I guess you can change the parser to check for that, a bit like C++ does with

C<vector<int>>

[sparkprime]

A workaround is also to do [ : : ]

[mikedanese]

Ok, fixed with https://github.com/mikedanese/jsonnet/blob/slice/core/parser.cpp#L1201-L1208 and enabled the test.

[mikedanese]

I can do it without push() but it requires a whole nother branch which feels gross

[sparkprime]

If you desugared them to null, you could test for null in std.slice and implement the full semantics that way

[sparkprime]

>>> [1,2,3][None:None:None]
[1, 2, 3]

In fact this is mandatory for python compatibility :)

[mikedanese]

@sparkprime addressed all review comments.

[mikedanese]

O(n^2) :( but would rather optimize this in the vm for array concatinations where one side is not used again.

$ jsonnet git:(slice): cat exp.jsonnet 
std.range(0, std.extVar("vars").count)[2:100000000000:2]
$ jsonnet git:(slice): time ./jsonnet --code-var vars='{"count": 1000}' exp.jsonnet > /dev/null 
./jsonnet --code-var vars='{"count": 1000}' exp.jsonnet > /dev/null  0.25s user 0.00s system 99% cpu 0.258 total
$ jsonnet git:(slice): time ./jsonnet --code-var vars='{"count": 10000}' exp.jsonnet > /dev/null                                                                                             
./jsonnet --code-var vars='{"count": 10000}' exp.jsonnet > /dev/null  1.98s user 0.01s system 99% cpu 1.990 total
$ jsonnet git:(slice): time ./jsonnet --code-var vars='{"count": 100000}' exp.jsonnet > /dev/null
./jsonnet --code-var vars='{"count": 100000}' exp.jsonnet > /dev/null  46.23s user 0.30s system 99% cpu 46.594 total

[sparkprime]

Yes I am not sure how to solve this. There are various instances of it throughout the code.

The Python approach is to make std.join O(n) and then do everything in terms of that.

Currently std.join is O(n^2) and making it O(n) would mean making it native.

There may be a better way though.

[mikedanese]

I'm curious about implementing the internal representation with a persistent data structure. Arrays could still be copy on write minus the deep copy.