Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NumPy 2.0 compatibility #2216

Closed
mgorny opened this issue Apr 30, 2024 · 14 comments · Fixed by #2220
Closed

NumPy 2.0 compatibility #2216

mgorny opened this issue Apr 30, 2024 · 14 comments · Fixed by #2220

Comments

@mgorny
Copy link
Contributor

mgorny commented Apr 30, 2024

When running trimesh's 4.3.2 test suite against NumPy 2.0.0rc1, I'm getting lots of test failures.

The vast majority of them are akin to:

FAILED tests/test_inertia.py::InertiaTest::test_inertia - AttributeError: `ptp` was removed from the ndarray class in NumPy 2.0. Use np.ptp(arr, ...) instead.

The remaining (non-ptp) failures are:

FAILED tests/test_texture.py::TextureTest::test_pbr_material_fusion - AttributeError: 'ColorVisuals' object has no attribute 'material'
FAILED tests/test_ply.py::PlyTest::test_fix_texture - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_trackball.py::TrackballTest::test_drag_roll - ValueError: Unable to avoid copy while creating an array as requested.
FAILED tests/test_trackball.py::TrackballTest::test_drag_rotate - ValueError: Unable to avoid copy while creating an array as requested
.
FAILED tests/test_trackball.py::TrackballTest::test_resize - ValueError: Unable to avoid copy while creating an array as requested.
FAILED tests/test_trackball.py::TrackballTest::test_rotate - ValueError: Unable to avoid copy while creating an array as requested.
FAILED tests/test_transformations.py::TransformTest::test_doctest - ValueError: TestResults(failed=31, attempted=389)
FAILED tests/test_transformations.py::TransformTest::test_rotation - ValueError: Unable to avoid copy while creating an array as reques
ted.
FAILED tests/test_transformations.py::TransformTest::test_symbolic_euler - ValueError: could not convert string to float: 'np.float64(0
.417022004702574)'
FAILED tests/test_transformations.py::TransformTest::test_symbolic_translate - ValueError: could not convert string to float: 'np.float
64(0.417022004702574)'
FAILED tests/test_ply.py::PlyTest::test_skip_texturefile - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_ply.py::PlyTest::test_texturefile - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_nsphere.py::NSphereTest::test_minball - TypeError: float() argument must be a string or a real number, not 'Trimesh'
FAILED tests/test_obj.py::OBJTest::test_chair - StopIteration
FAILED tests/test_ply.py::PlyTest::test_uv_export - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_scene.py::SceneTests::test_scaling_3D_mixed - AttributeError: 'NoneType' object has no attribute 'copy'
FAILED tests/test_points.py::PointsTest::test_obb - ValueError: Oriented bounds must be passed a mesh or a set of points!
FAILED tests/test_creation.py::CreationTest::test_path_sweep - ValueError: zero-size array to reduction operation maximum which has no 
identity
FAILED tests/test_scene.py::SceneTests::test_zipped - assert 0 >= 6 
FAILED tests/test_ply.py::PlyTest::test_blender_uv - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_stl.py::STLTests::test_ascii_keyword - assert False
FAILED tests/test_inertia.py::InertiaTest::test_frame_inertia_box - ValueError: Unable to avoid copy while creating an array as request
ed.
FAILED tests/test_segments.py::SegmentsTest::test_extrude - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_segments.py::SegmentsTest::test_svg - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_section.py::SliceTest::test_cap_coplanar - assert False
FAILED tests/test_gltf.py::GLTFTest::test_skip_materials - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_gltf.py::GLTFTest::test_spec_gloss_factors_only - KeyError: 'Cube.005'
FAILED tests/test_gltf.py::GLTFTest::test_specular_glossiness - assert 0 > 0
FAILED tests/test_gltf.py::GLTFTest::test_webp - OverflowError: Python integer 2147483648 out of bounds for int32
FAILED tests/test_gltf.py::GLTFTest::test_tex_export - OverflowError: Python integer 2147483648 out of bounds for int32

The log with tracebacks has over 50M, so I can't really attach it. However, I suppose you will be able to reproduce most of it, and I can re-report when the most common issues are resolved.

@mikedh
Copy link
Owner

mikedh commented Apr 30, 2024

Thanks for the list!! I have been running the ruff NPY migration rule but I guess they didn't add ptp (i.e astral-sh/ruff#10646) to that.

Seems like a lot of the other ones are "shapely didn't import" maybe it needs a prerelease or something, I'll take a look.

@mgorny
Copy link
Contributor Author

mgorny commented Apr 30, 2024

Seems like a lot of the other ones are "shapely didn't import" maybe it needs a prerelease or something, I'll take a look.

Oh, sorry, that's probably on my end. Lemme rebuild shapely and see if that helps.

@mgorny
Copy link
Contributor Author

mgorny commented Apr 30, 2024

Yeah, it was my fault. I've fixed the Gentoo Shapely package and updated the list. Thanks!

@mgorny
Copy link
Contributor Author

mgorny commented May 1, 2024

Do you want me to try making a pull request, at least for the problems with "obvious" solutions?

@mikedh
Copy link
Owner

mikedh commented May 6, 2024

Yeah a PR would be awesome! I got distracted trying to write a ruff rule in addition to NPY201 for the ndarray.ptp removal but since it's not a simple replacement it does not look easy to do automatically like that.

I wonder why they nuked ptp, and why the deprecation messages didn't show up in pytest.

@mikedh
Copy link
Owner

mikedh commented May 7, 2024

I got the ptp instances in the referenced PR and a precision issue with trimesh.grouping. I think the remaining issues are:

  • sympy is failing to convert a float64 scalar because it goes through a string for precision reasons. Not sure why this wasn't fixed by Fix parsing and printing of numpy types for numpy 2.0 sympy/sympy#26112
  • trimesh.transformations uses doctests as this was pulled from elsewhere. The string formatting changes break all the doctests which I should probably just suck it up and convert to normal test functions.

@mgorny
Copy link
Contributor Author

mgorny commented May 7, 2024

Thanks. With the linked PR applied on top of 4.3.2, I'm down to:

FAILED tests/test_transformations.py::TransformTest::test_doctest - ValueError: TestResults(failed=7, attempted=389)
FAILED tests/test_transformations.py::TransformTest::test_symbolic_euler - ValueError: could not convert string to float: 'np.float64(0.417022004702574)'
FAILED tests/test_transformations.py::TransformTest::test_symbolic_translate - ValueError: could not convert string to float: 'np.float64(0.417022004702574)'
FAILED tests/test_voxel.py::VoxelGridTest::test_roundtrip - AssertionError: assert False
 +  where False = <function allclose at 0x7f4bdf7353b0>(array([[-0.55, -0.55, -0.55],\n       [-0.25,  0.55,  0.55]]), array([[-0.55, -0.55, -0.55],\n       [ 0.55,  0.55,  0.55]]))
 +    where <function allclose at 0x7f4bdf7353b0> = <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'>.allclose
 +      where <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'> = g.np
 +    and   array([[-0.55, -0.55, -0.55],\n       [-0.25,  0.55,  0.55]]) = <trimesh.VoxelGrid(11, 11, 11)>.bounds
 +    and   array([[-0.55, -0.55, -0.55],\n       [ 0.55,  0.55,  0.55]]) = <trimesh.VoxelGrid(np.int64(11), np.int64(11), np.int64(11))>.bounds
FAILED tests/test_section.py::SliceTest::test_cap - assert False
 +  where False = <trimesh.Trimesh(vertices.shape=(6, 3), faces.shape=(8, 3))>.is_watertight
FAILED tests/test_section.py::SliceTest::test_cap_coplanar - assert False
 +  where False = <trimesh.Trimesh(vertices.shape=(3505, 3), faces.shape=(7014, 3))>.is_watertight
FAILED tests/test_section.py::SliceTest::test_slice_onplane - assert False
 +  where False = <trimesh.Trimesh(vertices.shape=(1550, 3), faces.shape=(3132, 3))>.is_watertight
FAILED tests/test_creation.py::CreationTest::test_path_sweep - ValueError: zero-size array to reduction operation maximum which has no identity
FAILED tests/test_creation.py::CreationTest::test_triangulate - AssertionError: assert np.False_
 +  where np.False_ = <function isclose at 0x7f4bdf735530>(np.float64(0.0), 3.1365484905459398)
 +    where <function isclose at 0x7f4bdf735530> = <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'>.isclose
 +      where <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'> = g.np
FAILED tests/test_creation.py::CreationTest::test_triangulate_plumbing - AssertionError: assert np.False_
 +  where np.False_ = <function isclose at 0x7f4bdf735530>(np.float64(0.0), 30768.414982316746)
 +    where <function isclose at 0x7f4bdf735530> = <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'>.isclose
 +      where <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'> = g.np

@mikedh
Copy link
Owner

mikedh commented May 7, 2024

Nice! I think some/most of these are fixed by installing the sympy prerelease?

@mgorny
Copy link
Contributor Author

mgorny commented May 8, 2024

I'm afraid upgrading to sympy 1.12.1rc1 doesn't improve things (though sympy itself stops failing tests).

@mikedh
Copy link
Owner

mikedh commented May 8, 2024

Yeah actually same re: sympy it was just my development machine had silently downgraded numpy.

At least a few of these are mapbox_earcut. It's a pybind thing using numpy which is probably gonna need a PR unfortunately haha:

In [10]: mapbox_earcut.triangulate_float64(vertices, rings)
Out[10]: 
array([99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99, 99,
       99, 99, 99, 99, 99], dtype=uint32)


@mikedh
Copy link
Owner

mikedh commented May 8, 2024

If you install mapbox-earcut from source test_creation passes. I think it just has to be rebuilt with a newer version of numpy than what it was currently built with.

@mgorny
Copy link
Contributor Author

mgorny commented May 9, 2024

Thanks! That gets me down to two failures:

FAILED tests/test_transformations.py::TransformTest::test_doctest - ValueError: TestResults(failed=7, attempted=389)
FAILED tests/test_voxel.py::VoxelGridTest::test_roundtrip - AssertionError: assert False
 +  where False = <function allclose at 0x7f5ccf730b70>(array([[-0.55, -0.55, -0.55],\n       [-0.25,  0.55,  0.55]]), array([[-0.55, -0.55, -0.55],\n       [ 0.55,  0.55,  0.55]]))
 +    where <function allclose at 0x7f5ccf730b70> = <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'>.allclose
 +      where <module 'numpy' from '/usr/lib/python3.11/site-packages/numpy/__init__.py'> = g.np
 +    and   array([[-0.55, -0.55, -0.55],\n       [-0.25,  0.55,  0.55]]) = <trimesh.VoxelGrid(11, 11, 11)>.bounds
 +    and   array([[-0.55, -0.55, -0.55],\n       [ 0.55,  0.55,  0.55]]) = <trimesh.VoxelGrid(np.int64(11), np.int64(11), np.int64(11))>.bounds

The first one seems to be the doctest case, the second one I don't know.

@mikedh
Copy link
Owner

mikedh commented May 10, 2024

I think I got those last 2, and am putting together a PR for mapbox-earcut.

@mgorny
Copy link
Contributor Author

mgorny commented May 18, 2024

Thanks. It seems that the tests pass for me on main right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants