MLX compatibility: Constants, Datatypes and Array Attributes

[aaishwarymishra]

Constant

Array API	MLX analog	status	test node	result
e	mx.e	exact	array_api_tests/test_constants.py::test_e	pass
inf	mx.inf	exact	array_api_tests/test_constants.py::test_inf	pass
nan	mx.nan	exact	array_api_tests/test_constants.py::test_nan	pass
newaxis	mx.newaxis	exact	array_api_tests/test_constants.py::test_newaxis	pass
pi	mx.pi	exact	array_api_tests/test_constants.py::test_pi	pass

[aaishwarymishra]

DataTypes

Array API	MLX analog	status	test node	result
bool	mx.bool	exact	test_has_names (dtype via _array_module)	PASS / defined
complex128		missing	test_has_names	missing (stub)
complex64	mx.complex64	exact	test_has_names	PASS / defined
float32	mx.float32	exact	test_has_names	PASS / defined
float64	mx.float64	exact	test_has_names	defined (but unsupported on GPU)
int16	mx.int16	exact	test_has_names	PASS / defined
int32	mx.int32	exact	test_has_names	PASS / defined
int64	mx.int64	exact	test_has_names	PASS / defined
int8	mx.int8	exact	test_has_names	PASS / defined
uint16	mx.uint16	exact	test_has_names	PASS / defined
uint32	mx.uint32	exact	test_has_names	PASS / defined
uint64	mx.uint64	exact	test_has_names	PASS / defined
uint8	mx.uint8	exact	test_has_names	PASS / defined

Note: In reporting.py we have

def to_json_serializable(o):
    if o in dtype_to_name:
        return dtype_to_name[o]

While running tests all the test failed because this method was being called at each test and the main problem is if the o is of the type None

MlLX throws an exception for equality like

if None == mlx.core.float16:
    print("success)

We either have to modify the method to something like

def to_json_serializable(o):
    if o is not None and o in dtype_to_name:
        return dtype_to_name[o]

or try to reach out to mlx team for the fix

[aaishwarymishra]

Array Attributes

Array API	MLX Analog	Status	Test Node	Result	Blame / Notes
T	array.T	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
abs	array.abs	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
add	array.add	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
and	array.and	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
array_namespace	array.array_namespace	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
bool	array.bool	exact	array_api_tests/test_array_object.py	PASS	test_scalar_casting[bool] passes
complex	missing	manual	array_api_tests/test_array_object.py	FAIL	test_has_names only signal: mlx missing complex(); test_array_object doesn't parametrize it
dlpack	array.dlpack	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
dlpack_device	array.dlpack_device	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
eq	array.eq	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
float	array.float	exact	array_api_tests/test_array_object.py	FAIL	test_scalar_casting[float(float64)] fails — present but wrong behavior
floordiv	array.floordiv	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
ge	array.ge	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
getitem	array.getitem	exact	array_api_tests/test_array_object.py	FAIL	test_getitem, test_getitem_masking, test_getitem_arrays_and_ints_1 fail (real indexing bugs)
gt	array.gt	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
index	missing	manual	array_api_tests/test_array_object.py	FAIL	test_has_names + test_scalar_casting[index(*)] both fail: mlx missing index()
int	array.int	exact	array_api_tests/test_array_object.py	PASS	test_scalar_casting[int(*)] passes for all int dtypes
invert	array.invert	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
le	array.le	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
lshift	array.lshift	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
lt	array.lt	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
matmul	array.matmul	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
mod	array.mod	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
mul	array.mul	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
ne	array.ne	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
neg	array.neg	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
or	array.or	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
pos	missing	manual	array_api_tests/test_array_object.py	FAIL	test_has_names only signal: mlx missing pos(); test_array_object doesn't test it
pow	array.pow	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
rshift	array.rshift	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
setitem	array.setitem	exact	array_api_tests/test_array_object.py	FAIL	test_setitem, test_setitem_masking fail (ValueError / masking TypeError)
sub	array.sub	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
truediv	array.truediv	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
xor	array.xor	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
device	missing	manual	array_api_tests/test_array_object.py	FAIL	test_has_names only signal: mlx missing device attribute
dtype	array.dtype	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
mT	missing	manual	array_api_tests/test_array_object.py	FAIL	test_has_names only signal: mlx missing mT attribute
ndim	array.ndim	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
shape	array.shape	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
size	array.size	exact	array_api_tests/test_array_object.py	PASS*	presence only; not behaviorally run in this file
to_device	missing	manual	array_api_tests/test_array_object.py	FAIL	test_has_names only signal: mlx missing to_device()

Note: Even though test_has_name fails for complex but when I simply try

arr = mx.array([1.0, 2.0, 3.0])

complex_arr = complex(arr)

It works fine

[ev-br]

Great. what about functions from the spec?

[aaishwarymishra]

Great. what about functions from the spec?

Hi, its still work in progress, I am planning on making sub-issues for them and reference them on the tracking issue, I am still working on this issue investigating test failures etc, once done link this to main tracker issue and continue to do same with methods and functions in the future :)

[aaishwarymishra]

2 unique failures in array_api_tests/test_array_object.py :

1. mx.complex64 throws in TypeError: float() argument must be a string or a real number, not 'complex' when used complex on them.

import mlx.core as mx
arr = mx.array(1, dtype=mx.complex64)
arr2 = complex(arr)

2. No boolean indexing causes few tests to fail

arr2 = arr[[True,False]] # read operations trows error boolean assignment works

Other failures were because of missing array attributes like no __index__ and no float64 support on the gpu