Loosen ml-dtypes dependency

[kvablack]

tensorstore recently added ml-dtypes>=0.3.1 as a dependency, making it technically incompatible with any version of TensorFlow, which has ml-dtypes~=0.2.0 at most.

[laramiel]

You can try running third_party/pypa/generate_workspace.sh , adding -r ml-dtypes=0.2.0, and see if the constraints are met.

See tools/pypi_solver/main.py, which has a number of options to attempt to get compatible versions.

[kvablack]

Sorry, I'm not sure I understand. If I want to install tensorstore and TensorFlow via pip (i.e., I have tensorflow == 2.15.0 and tensorstore == 0.1.51 in my requirements.txt), how can I do that without causing a version conflict? I'm not familiar with how generate_workspace.sh would become part of that workflow.

The reality of what's happening is actually more sinister: I have a chain of several dependencies that ends with tensorstore, but I also need TensorFlow, so the version of several other packages is silently "capped" by this ml-dtypes conflict. It seems like ml-types >= 0.3.1 was only added for float8 and int4 integration, which I definitely don't need. I would like to be able to get the most up-to-date version of tensorstore, as well as those other packages, without resorting to some complex pip install --no-deps shenanigans.

[jbms]

Currently we do specifically depend on 0.3.1 because certain data types were added in that version, and tensorstore will not work with an earlier version.

We could potentially change tensorstore to just skip support for missing data types, but it would instead be better to just change the version requirement for tensorflow, since I'm pretty sure it will in fact work with a newer version of ml-dtypes.

[kvablack]

Ok thanks, I guess I'm off to try and convince the TensorFlow maintainers then