SortCompare (Array#sort)

[Yaffle]

http://tc39.github.io/ecma262/#sec-sortcompare

According to spec comparefn will be never called for undefined values, but v8 and WebKit implementations do not implement this behavior:

var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      array[i] = undefined;
    }
  }
  if (counter === 1) {
    if (a == undefined || b == undefined) {
      console.log("comparator was called for an undefined value");
    }
  }
  counter += 1;
  return a - b;
});

var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      delete array[i];
    }
  }
  if (counter === 1) {
    if (a == undefined || b == undefined) {
      console.log("comparator was called for a hole");
    }
  }
  counter += 1;
  return a - b;
});

var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      delete array[i];
      Object.prototype[i.toString()] = 42;
    }
  }
  if (counter === 1) {
    if (a === 42 || b === 42) {
      console.log("comparator was called for a value from prototype");
    }
  }
  counter += 1;
  return a - b;
});

The spec only tells that the sort order is implementation defined...
v8 and WebKit do this for performance reasons, probably, because they can move undefined values and holes to the end of an array before the sorting and call comparefn directly:
https://github.com/v8/v8/blob/master/src/js/array.js#L1194
https://github.com/WebKit/webkit/blob/cc80ecf5f0902e724a6e398d152323120e4e3b82/Source/JavaScriptCore/builtins/ArrayPrototype.js#L473

[bterlson]

Not trying to be obtuse: what's the ECMA262 bug here?

[bterlson]

Not trying to be obtuse: what's the ECMA262 bug here?

[ljharb]

If SpiderMonkey and Chakra follow the spec, but v8 and JSC don't, then this is a v8 and a JSC bug. @Yaffle can you confirm that SpiderMonkey and Chakra follow the spec here?

[ljharb]

If SpiderMonkey and Chakra follow the spec, but v8 and JSC don't, then this is a v8 and a JSC bug. @Yaffle can you confirm that SpiderMonkey and Chakra follow the spec here?

[bterlson]

Also how does the first and second case differ? Maybe the first means to assign undefined rather than delete the property?

[bterlson]

Also how does the first and second case differ? Maybe the first means to assign undefined rather than delete the property?

[bterlson]

Here's the results I get, with the update I suggest above:

spidermonkey

Undefined elements

Holes

Holes with prototype property

chakra

Undefined elements

Holes

Holes with prototype property

d8

Undefined elements

comparator was called for an undefined value

Holes

comparator was called for a hole

Holes with prototype property

comparator was called for a value from prototype

Anyone using eshost can use the following script:

print('##### Undefined elements');
var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      array[i] = undefined;
    }
  }
  if (counter === 1) {
    if (a == undefined || b == undefined) {
      print("comparator was called for an undefined value");
    }
  }
  counter += 1;
  return a - b;
});

print('##### Holes');
var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      delete array[i];
    }
  }
  if (counter === 1) {
    if (a == undefined || b == undefined) {
      print("comparator was called for a hole");
    }
  }
  counter += 1;
  return a - b;
});

print('##### Holes with prototype property');
var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      delete array[i];
      Object.prototype[i.toString()] = 42;
    }
  }
  if (counter === 1) {
    if (a === 42 || b === 42) {
      print("comparator was called for a value from prototype");
    }
  }
  counter += 1;
  return a - b;
});

[bterlson]

Here's the results I get, with the update I suggest above:

spidermonkey

Undefined elements

Holes

Holes with prototype property

chakra

Undefined elements

Holes

Holes with prototype property

d8

Undefined elements

comparator was called for an undefined value

Holes

comparator was called for a hole

Holes with prototype property

comparator was called for a value from prototype

Anyone using eshost can use the following script:

print('##### Undefined elements');
var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      array[i] = undefined;
    }
  }
  if (counter === 1) {
    if (a == undefined || b == undefined) {
      print("comparator was called for an undefined value");
    }
  }
  counter += 1;
  return a - b;
});

print('##### Holes');
var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      delete array[i];
    }
  }
  if (counter === 1) {
    if (a == undefined || b == undefined) {
      print("comparator was called for a hole");
    }
  }
  counter += 1;
  return a - b;
});

print('##### Holes with prototype property');
var array = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
var counter = 0;
array.sort(function (a, b) {
  if (counter === 0) {
    for (var i = 0; i < array.length; i += 1) {
      delete array[i];
      Object.prototype[i.toString()] = 42;
    }
  }
  if (counter === 1) {
    if (a === 42 || b === 42) {
      print("comparator was called for a value from prototype");
    }
  }
  counter += 1;
  return a - b;
});

[Yaffle]

@bterlson , the spec tells when and how to call comparefn:

The arguments for calls to SortCompare are values returned by a previous call to the [[Get]] internal method, unless the properties accessed by those previous calls did not exist according to HasOwnProperty. If both perspective arguments to SortCompare correspond to non-existent properties, use +0 instead of calling SortCompare. If only the first perspective argument is non-existent use +1. If only the second perspective argument is non-existent use -1.
and steps for SortCompare has checks for undefined values.

Well... not a big bug...

[Yaffle]

@bterlson , the spec tells when and how to call comparefn:

The arguments for calls to SortCompare are values returned by a previous call to the [[Get]] internal method, unless the properties accessed by those previous calls did not exist according to HasOwnProperty. If both perspective arguments to SortCompare correspond to non-existent properties, use +0 instead of calling SortCompare. If only the first perspective argument is non-existent use +1. If only the second perspective argument is non-existent use -1.
and steps for SortCompare has checks for undefined values.

Well... not a big bug...

[Yaffle]

@ljharb , SpiderMonkey may switch to "self-hosted" Array#sort ( https://bugzilla.mozilla.org/show_bug.cgi?id=715181 ) and there is a suggestion to use same behaviour for undefined values and holes as in V8 and JSC

[Yaffle]

@ljharb , SpiderMonkey may switch to "self-hosted" Array#sort ( https://bugzilla.mozilla.org/show_bug.cgi?id=715181 ) and there is a suggestion to use same behaviour for undefined values and holes as in V8 and JSC

[bterlson]

@Yaffle in other words, you are requesting a spec change to the v8/jsc approach? If so, can you expand on why it is necessary and/or desirable behavior? Otherwise this seems like a bug in V8 and JSC that SM should not copy...

[bterlson]

@Yaffle in other words, you are requesting a spec change to the v8/jsc approach? If so, can you expand on why it is necessary and/or desirable behavior? Otherwise this seems like a bug in V8 and JSC that SM should not copy...

[allenwb]

The special treatment of non-existent properties dates back to the ES3 spec. ES5 changes the language used to describe it from prose to the use of [[HasProperty]]. ES6 moved the test from inside of SortCompare into a guard on calling SortCompare. None of them call a user provided comparefn for non-existent properties.

I don't recall the exact motivation for the ES6 refactoring but I'm pretty use that there are either es-discuss or bugs.ecmascript.org discussions that relate it it.

Note that ES3 spec. has an explicit change from ES1&2 which did not explicitly check for non-existent properties and which explicitly stated that the handling of non-existent properties was implementation-dependent.

Clearly, when ES3 was being developed a decision was made that undefined values and holes should not have the same treatment. In particular, that holes sort after undefined at the end of the array. I don't see why we would want to change that long standing design design.

[allenwb]

The special treatment of non-existent properties dates back to the ES3 spec. ES5 changes the language used to describe it from prose to the use of [[HasProperty]]. ES6 moved the test from inside of SortCompare into a guard on calling SortCompare. None of them call a user provided comparefn for non-existent properties.

I don't recall the exact motivation for the ES6 refactoring but I'm pretty use that there are either es-discuss or bugs.ecmascript.org discussions that relate it it.

Note that ES3 spec. has an explicit change from ES1&2 which did not explicitly check for non-existent properties and which explicitly stated that the handling of non-existent properties was implementation-dependent.

Clearly, when ES3 was being developed a decision was made that undefined values and holes should not have the same treatment. In particular, that holes sort after undefined at the end of the array. I don't see why we would want to change that long standing design design.

[rossberg]

@allenwb, motivation might be that this semantics induces a non-zero runtime cost while having a zero real-world benefit.

[rossberg]

@allenwb, motivation might be that this semantics induces a non-zero runtime cost while having a zero real-world benefit.

[anba]

I don't recall the exact motivation for the ES6 refactoring but I'm pretty use that there are either es-discuss or bugs.ecmascript.org discussions that relate it it.

https://bugs.ecmascript.org/show_bug.cgi?id=3089

[anba]

I don't recall the exact motivation for the ES6 refactoring but I'm pretty use that there are either es-discuss or bugs.ecmascript.org discussions that relate it it.

https://bugs.ecmascript.org/show_bug.cgi?id=3089

[Yaffle]

https://hg.mozilla.org/mozilla-central/rev/1c4b0a89fd5b - Firefox 47 does not use HasOwnProperty check and patches holes (note: the body of comparefn should not be equal to return a-b):

var array = [1, 2, 3, 4, 5, 6, 7, 8, , 10];
array.sort(function (a, b) {
  return a < b ? -1 : +1;
});
console.log(array, "9" in array);

[Yaffle]

https://hg.mozilla.org/mozilla-central/rev/1c4b0a89fd5b - Firefox 47 does not use HasOwnProperty check and patches holes (note: the body of comparefn should not be equal to return a-b):

var array = [1, 2, 3, 4, 5, 6, 7, 8, , 10];
array.sort(function (a, b) {
  return a < b ? -1 : +1;
});
console.log(array, "9" in array);

[ljharb]

@Yaffle that doesn't patch holes - I get Array [ 1, 2, 3, 4, 5, 6, 7, 8, 10, <1 empty slot> ] in the FF 46 console. It just seems to sort holes at the end (which is what #302 (comment) says)

[ljharb]

@Yaffle that doesn't patch holes - I get Array [ 1, 2, 3, 4, 5, 6, 7, 8, 10, <1 empty slot> ] in the FF 46 console. It just seems to sort holes at the end (which is what #302 (comment) says)

[Yaffle]

@ljharb , the change was applied recently, do you use the latest nightly build?

[Yaffle]

@ljharb , the change was applied recently, do you use the latest nightly build?

[ljharb]

I used v46. On the latest, v47, I see the same behavior.

[ljharb]

I used v46. On the latest, v47, I see the same behavior.

[Yaffle]

somehow I see Array [ 1, 2, 3, 4, 5, 6, 7, 8, 10, undefined ] true

[Yaffle]

somehow I see Array [ 1, 2, 3, 4, 5, 6, 7, 8, 10, undefined ] true

[ljharb]

Ah, oops, yes I see the change you mean. 46 said "empty slot" but 47 says "undefined". Thanks.

[ljharb]

Ah, oops, yes I see the change you mean. 46 said "empty slot" but 47 says "undefined". Thanks.