Fix SORT_REGULAR with new transitive comparison functions #20517

jmarble · 2025-11-18T03:43:57Z

Summary

Fixes #20262 by making SORT_REGULAR fall back to a fully transitive comparator whenever loose comparison semantics would otherwise be non-transitive (numeric strings vs ints/floats, enums, nested arrays/objects). This keeps duplicates grouped so array_unique() and the sort family behave consistently.

Highlights

Introduce shared Zend helpers (zend_compare_{long,double}_to_string_ex(), zendi_smart_strcmp_ex()) so transitive scalar comparisons are implemented once and reused everywhere.
Add php_array_compare_transitive()/php_array_compare_transitive_objects() plus php_array_sort_regular() so all SORT_REGULAR entry points (and array_unique()) automatically use the transitive comparator.
Add regression tests: array_unique with enums/objects/nested arrays and SORT_REGULAR consistency tests for sort()/ksort() on numeric-string edge cases.
Performance is effectively neutral (regressions/wins within a few percent in microbenchmarks, no notable outliers).

Girgias

Various comments and questions and this needs a rebase as I refactored the sorting code to remove a bunch of duplication.

ext/standard/array.c

Girgias · 2025-11-18T13:22:33Z

ext/standard/array.c

+static int php_array_hash_compare_transitive(zval *zv1, zval *zv2) /* {{{ */
+{
+	return php_array_compare_transitive(zv1, zv2);
+}
+/* }}} */


Kept this one so we can pass a compare_func_t to zend_hash_compare().
php_array_compare_transitive() doesn’t match that signature, so we still need this tiny adapter.

My previous comment is no longer valid, this can be removed, but I noticed a measurable regression in my benchmarks after removing it, so I decided to keep it in place. I should probably include a comment in the function regarding it.

How are you benchmarking? Because I don't really see why it would regress?

ext/standard/array.c

jmarble · 2025-11-18T18:58:35Z

@Girgias thank you for taking the time to provide the careful review! Looks like I was able to capture your sorting code refactor when I created this new branch. I'll push a fresh commit what I addressed in your code comments. Thanks again for the help!

Girgias

Please only do the fix for the transitivity.

Optimizations can be decided later, but currently it just pollutes the PR and makes it harder to review and merge.

- Add zend_compare_{long,double}_to_string_ex() plus zendi_smart_strcmp_ex() so SORT_REGULAR can invoke transitive-aware scalar comparisons without touching zend_compare() - Introduce php_array_compare_transitive() (pared-down zend_compare()) and php_array_compare_transitive_objects() (mirrors zend_std_compare_objects()) so arrays, objects, and enums recurse with transitive ordering - Route the public sort APIs and array_unique() through php_array_sort_regular() so PHP_SORT_REGULAR always uses the new comparator - Add regression tests: phpGH-20262 (array_unique with enums/objects/nested arrays) plus SORT_REGULAR consistency tests for sort()/ksort() on numeric-string edge cases Fixes: phpGH-20262

jmarble · 2025-11-21T22:48:16Z

@Girgias yes, I clearly got a bit carried away haha. I decided to reimplement and force push a clean commit. Sorry for the mess I made of this PR.

I have a bag full of optimizations we can save for a follow-up PR. One worth calling out would be to split zendi_smart_strcmp() so the transitive comparator doesn’t need to re-run the non-transitive fast paths. I also found an opportunity to add a single-bucket fast path in zend_compare_symbol_tables() which showed close to 1.25x speedup on array comparison.

Girgias · 2025-11-22T21:07:12Z

Zend/zend_operators.h

+static zend_always_inline int zend_compare_long_to_string_ex(zend_long lval, zend_string *str, bool transitive)
+{
+	zend_long str_lval;
+	double str_dval;
+	uint8_t type = is_numeric_string(ZSTR_VAL(str), ZSTR_LEN(str), &str_lval, &str_dval, 0);
+
+	if (type == IS_LONG) {
+		return ZEND_THREEWAY_COMPARE(lval, str_lval);
+	}
+
+	if (type == IS_DOUBLE) {
+		return ZEND_THREEWAY_COMPARE((double) lval, str_dval);
+	}
+
+	if (transitive) {
+		if (ZSTR_LEN(str) == 0) {
+			return 1;
+		}
+		return -1;
+	}
+
+	zend_string *lval_as_str = zend_long_to_str(lval);
+	int cmp_result = zend_binary_strcmp(
+		ZSTR_VAL(lval_as_str), ZSTR_LEN(lval_as_str), ZSTR_VAL(str), ZSTR_LEN(str));
+	zend_string_release(lval_as_str);
+	return ZEND_NORMALIZE_BOOL(cmp_result);
+}
+
+static zend_always_inline int zend_compare_double_to_string_ex(double dval, zend_string *str, bool transitive)
+{
+	zend_long str_lval;
+	double str_dval;
+	uint8_t type = is_numeric_string(ZSTR_VAL(str), ZSTR_LEN(str), &str_lval, &str_dval, 0);
+
+	ZEND_ASSERT(!zend_isnan(dval));
+
+	if (type == IS_LONG) {
+		str_dval = (double) str_lval;
+		return ZEND_THREEWAY_COMPARE(dval, str_dval);
+	}
+
+	if (type == IS_DOUBLE) {
+		return ZEND_THREEWAY_COMPARE(dval, str_dval);
+	}
+
+	if (transitive) {
+		if (ZSTR_LEN(str) == 0) {
+			return 1;
+		}
+		return -1;
+	}
+
+	zend_string *dval_as_str = zend_double_to_str(dval);
+	int cmp_result = zend_binary_strcmp(
+		ZSTR_VAL(dval_as_str), ZSTR_LEN(dval_as_str), ZSTR_VAL(str), ZSTR_LEN(str));
+	zend_string_release(dval_as_str);
+	return ZEND_NORMALIZE_BOOL(cmp_result);
+}
+
+static zend_always_inline int zendi_smart_strcmp_ex(zend_string *s1, zend_string *s2, bool transitive)
+{
+	uint8_t ret1, ret2;
+	int oflow1, oflow2;
+	zend_long lval1 = 0, lval2 = 0;
+	double dval1 = 0.0, dval2 = 0.0;
+
+	if (UNEXPECTED(ZSTR_LEN(s1) == 0 || ZSTR_LEN(s2) == 0)) {
+		if (transitive) {
+			if (ZSTR_LEN(s1) == 0 && ZSTR_LEN(s2) == 0) {
+				return 0;
+			}
+			return ZSTR_LEN(s1) == 0 ? -1 : 1;
+		}
+	}
+
+	ret1 = is_numeric_string_ex(ZSTR_VAL(s1), ZSTR_LEN(s1), &lval1, &dval1, false, &oflow1, NULL);
+	ret2 = is_numeric_string_ex(ZSTR_VAL(s2), ZSTR_LEN(s2), &lval2, &dval2, false, &oflow2, NULL);
+
+	if (ret1 && ret2) {
+#if ZEND_ULONG_MAX == 0xFFFFFFFF
+		if (oflow1 != 0 && oflow1 == oflow2 && dval1 - dval2 == 0. &&
+			((oflow1 == 1 && dval1 > 9007199254740991. /*0x1FFFFFFFFFFFFF*/)
+			|| (oflow1 == -1 && dval1 < -9007199254740991.))) {
+#else
+		if (oflow1 != 0 && oflow1 == oflow2 && dval1 - dval2 == 0.) {
+#endif
+			/* both values are integers overflowed to the same side, and the
+			 * double comparison may have resulted in crucial accuracy lost */
+			goto string_cmp;
+		}
+		if ((ret1 == IS_DOUBLE) || (ret2 == IS_DOUBLE)) {
+			if (ret1 != IS_DOUBLE) {
+				if (oflow2) {
+					/* 2nd operand is integer > LONG_MAX (oflow2==1) or < LONG_MIN (-1) */
+					return -1 * oflow2;
+				}
+				dval1 = (double) lval1;
+			} else if (ret2 != IS_DOUBLE) {
+				if (oflow1) {
+					return oflow1;
+				}
+				dval2 = (double) lval2;
+			} else if (dval1 == dval2 && !zend_finite(dval1)) {
+				/* Both values overflowed and have the same sign,
+				 * so a numeric comparison would be inaccurate */
+				goto string_cmp;
+			}
+			dval1 = dval1 - dval2;
+			return ZEND_NORMALIZE_BOOL(dval1);
+		} else { /* they both have to be long's */
+			return lval1 > lval2 ? 1 : (lval1 < lval2 ? -1 : 0);
+		}
+	} else if (ret1) {
+		if (transitive) {
+			return -1;
+		}
+		goto string_cmp;
+	} else if (ret2) {
+		if (transitive) {
+			return 1;
+		}
+		goto string_cmp;
+	}
+
+	int strcmp_ret;
+string_cmp:
+	if (transitive) {
+		return zend_compare_non_numeric_strings(s1, s2);
+	}
+
+	strcmp_ret = zend_binary_strcmp(ZSTR_VAL(s1), ZSTR_LEN(s1), ZSTR_VAL(s2), ZSTR_LEN(s2));
+	return ZEND_NORMALIZE_BOOL(strcmp_ret);
+}


Why are these in the header?

Girgias · 2025-11-22T21:09:08Z

ext/standard/array.c

+		uintptr_t lhs_ptr = (uintptr_t) Z_OBJ_P(lhs);
+		uintptr_t rhs_ptr = (uintptr_t) Z_OBJ_P(rhs);


I'm not sure if this will result in stable results, @iluuu1994 or @arnaud-lb might have a better idea on how to implement an enum comparison.

Girgias · 2025-11-22T21:10:59Z

ext/standard/array.c

+static int php_array_hash_compare_transitive(zval *zv1, zval *zv2) /* {{{ */
+{
+	return php_array_compare_transitive(zv1, zv2);
+}
+/* }}} */


How are you benchmarking? Because I don't really see why it would regress?

Girgias · 2025-11-22T21:20:14Z

ext/standard/array.c

+/* Mirrors zend_std_compare_objects(), but recurses via php_array_compare_transitive()
+ * so nested properties obey SORT_REGULAR's transitive ordering. */
+static int php_array_compare_transitive_objects(zval *o1, zval *o2) /* {{{ */


What I think might make more sense is to create a zend_std_compare_objects_ex() function that takes a function pointer for the prop table comparison if this is identical.

As hopefully the compiler will inline the behaviour properly in zend_std_compare_objects() so that it should be equivalent. As for quite a bit I was trying to understand what the point of this is.

ext/standard/array.c

Girgias · 2025-11-22T21:24:49Z

ext/standard/array.c

+	if (UNEXPECTED(php_array_is_enum_zval(op1) || php_array_is_enum_zval(op2))) {
+		return php_array_compare_enum_zvals(op1, op2);
+	}


Why are you handling enums early here?

Girgias · 2025-11-22T21:26:23Z

ext/standard/array.c

+	ZVAL_DEREF(op1);
+	ZVAL_DEREF(op2);


Surely if you deref you want to deref before the int/str comparisons?

Girgias · 2025-11-22T21:29:33Z

ext/standard/array.c

 PHP_FUNCTION(arsort)
 {
-	php_sort(INTERNAL_FUNCTION_PARAM_PASSTHRU, php_get_data_reverse_compare_func, false);
+	HashTable *array;
+	zend_long sort_type = PHP_SORT_REGULAR;
+
+	ZEND_PARSE_PARAMETERS_START(1, 2)
+		Z_PARAM_ARRAY_HT_EX(array, 0, 1)
+		Z_PARAM_OPTIONAL
+		Z_PARAM_LONG(sort_type)
+	ZEND_PARSE_PARAMETERS_END();
+
+	if ((sort_type & ~PHP_SORT_FLAG_CASE) == PHP_SORT_REGULAR) {
+		php_array_sort_regular(array, false, true, false);
+		RETURN_TRUE;
+	}
+
+	php_array_apply_sort(array, sort_type, php_get_data_reverse_compare_func, false);
+	RETURN_TRUE;
 }


Please revert these changes, the point of this was to simplify the implementation by having a unique one.
So please change the various php_get_*_compare_func().

github-actions bot added Category: Engine Extension: standard ABI break labels Nov 18, 2025

This was referenced Nov 18, 2025

Fix SORT_REGULAR transitivity for mixed types #20315

Open

[PoC] support sorting packed arrays without converting to hashed arrays first #20408

Draft

Girgias requested changes Nov 18, 2025

View reviewed changes

jmarble marked this pull request as ready for review November 18, 2025 19:49

jmarble requested a review from bukka as a code owner November 18, 2025 19:49

jmarble requested a review from Girgias November 18, 2025 19:49

jmarble requested review from TimWolla, alexdowad, devnexen, kocsismate, ndossche and youkidearitai as code owners November 21, 2025 05:23

github-actions bot added Category: Build System Extension: bz2 Extension: gd Extension: libxml Extension: mbstring Extension: phar Extension: soap Extension: xml Extension: zip Extension: zlib Extension: uri labels Nov 21, 2025

jmarble force-pushed the fix-php-array-sort-regular branch 2 times, most recently from f6e9f05 to 374a660 Compare November 21, 2025 06:37

Girgias requested changes Nov 21, 2025

View reviewed changes

jmarble marked this pull request as draft November 21, 2025 16:15

jmarble force-pushed the fix-php-array-sort-regular branch from 374a660 to 2ff1700 Compare November 21, 2025 22:03

jmarble marked this pull request as ready for review November 21, 2025 22:48

Girgias requested changes Nov 22, 2025

View reviewed changes

		uintptr_t lhs_ptr = (uintptr_t) Z_OBJ_P(lhs);
		uintptr_t rhs_ptr = (uintptr_t) Z_OBJ_P(rhs);

Fix SORT_REGULAR with new transitive comparison functions #20517

Are you sure you want to change the base?

Fix SORT_REGULAR with new transitive comparison functions #20517

Conversation

jmarble commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Highlights

Uh oh!

Girgias left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jmarble commented Nov 18, 2025

Uh oh!

Girgias left a comment

Choose a reason for hiding this comment

Uh oh!

jmarble commented Nov 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jmarble commented Nov 18, 2025 •

edited

Loading