Add support for python 3.12

[h-mayorquin]

As in the title. Fingers cross is this easy.

[CodyCBakerPhD]

@h-mayorquin I heard that GitHub added Mac silicon runners, too, would you be able to add that (if it's true?)

[h-mayorquin]

@CodyCBakerPhD Did a quick google but it seems that Mac is a paid service. Anyway, I think we should do outside of this future.

Back to the PR, aiohttp is failing. It is related to the long comming deprecation of disutils. It seems that it will be fixed soon in aiohttp side though.

But unless we want to pin to the pre-release:
aio-libs/aiohttp#7639 (comment)

We will need to wait a bit for their latest wheels:
aio-libs/aiohttp#7675

I am not sure wher this dependency is coming from.

[CodyCBakerPhD]

I'm OK to wait a bit on this then, thanks for digging into it

I am not sure wher this dependency is coming from.

It's from the NWB Inspector: https://github.com/NeurodataWithoutBorders/nwbinspector/blob/dev/requirements.txt#L5

But it would be needed and used by any environment capable of streaming data from DANDI using fsspec

[h-mayorquin]

OK, this will take a while. python 3.12 is supported in the latest version of numpy but some of the packages that we install have put an upper bound on numpy. Probably should put to rest until we consolidate dependencies so this can be figured out without parsing the dependency tree.

[bendichter]

It looks like numba, which is required by SpikeInterface, is causing problems, as they still have not pypi released a version that supports py3.12,

[h-mayorquin]

Seems that numba intends to get this sorted soon.
numba/numba#9197

Python 3.12 was a special jump for a lot of scientific software because distutils was droped. Here is an interesting article from a couple of months that describes some of the problems that I guess Numba is still facing:
https://labs.quansight.org/blog/building-scipy-with-flang

[h-mayorquin]

Numba is done, next blocker is hdmf-zarr:

hdmf-dev/hdmf-zarr#162

[bendichter]

edit: nevermind, numcodecs is still giving us trouble

With hdmf-dev/hdmf-zarr#162 closed we are unblocked!

[h-mayorquin]

edit: nevermind, numcodecs is still giving us trouble

With hdmf-dev/hdmf-zarr#162 closed we are unblocked!

I thought that it was solved with:
hdmf-dev/hdmf-zarr#162

I did not see that causing problems after, where are you seeing the problems? The problem that I saw after is that there is some problem with the edf depedency:

neuroconv/src/neuroconv/datainterfaces/ecephys/edf/requirements.txt

Lines 1 to 2 in 258b536

	pyedflib>=0.1.30
	numpy<1.25.0;python_version>="3.11"

Because we have an upper bound for numpy but python 3.12 requires numpy to be larger than that. I tried some quick fixes but it did not work out and did not have time to dig further.

[CodyCBakerPhD]

I did not see that causing problems after, where are you seeing the problems?

Different problems per platform

Ubuntu & Windows, minor test case that needs updating due to a slight error type change: https://github.com/catalystneuro/neuroconv/actions/runs/7987728897/job/21810807661
Mac, numcodecs build issue: https://github.com/catalystneuro/neuroconv/actions/runs/7987728897/job/21810808203

[h-mayorquin]

@CodyCBakerPhD Thanks. I did not checked all the CI actions. So this is a Mac problem. I don't know why the Mac runner is trying to build a wheel for numcodecs, there are wheels available:
https://pypi.org/project/numcodecs/#files

[h-mayorquin]

Also, to add to our discussion about python EOL, zarr already dropped python 3.8 and numpy smaller than 1.20 already:
zarr-developers/zarr-python#1557

[CodyCBakerPhD]

Ubuntu and Windows passing on 3.12!

Now just the Mac numcodecs thing https://github.com/catalystneuro/neuroconv/actions/runs/7994819629/job/21833777077?pr=626

[CodyCBakerPhD]

Heh, well this is an interesting one

I'm not quite sure what to do with the MaxOne interface or MaxTwo #221 because various platforms on the CI have always had trouble getting the custom HDF5 plugin to work properly to decompress data contained in the files

I had gotten the tests and functionality to work on local devices, so it is/was somewhat functional in practice

[h-mayorquin]

What do you mean, what are the options here?

[CodyCBakerPhD]

(a) Keep Maxwell interfaces around despite no CI tests

(b) Deprecate and remove

[h-mayorquin]

Can we add?
h5py<=3.9.0; python_version<='3.12'

Aren't they still tested on ubuntu for python <3.12?

[CodyCBakerPhD]

Aren't they still tested on ubuntu for python <3.12?

I even had to exclude ubuntu a while back

https://github.com/catalystneuro/neuroconv/blob/main/tests/test_on_data/test_recording_interfaces.py#L187

[h-mayorquin]

I see.

My vote: I say let's keep it and we pin down the correct CI testing when and if we get a job / contract related to it.

Meanwhile, probably not worth figuring it out at this very moment.

[h-mayorquin]

This seems ready @CodyCBakerPhD and @bendichter

Thanks @CodyCBakerPhD for taking the last mile of this one : )

[CodyCBakerPhD]

Any idea what that issue was for mac and numcodecs?

[h-mayorquin]

It seems to me that some configuration of the full dependencies was making pip not pick the wheel for Mac.

[bendichter]

This looks great!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (22318d9) 92.60% compared to head (8e60912) 92.63%.
Report is 11 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #626      +/-   ##
==========================================
+ Coverage   92.60%   92.63%   +0.02%     
==========================================
  Files         115      115              
  Lines        6221     6216       -5     
==========================================
- Hits         5761     5758       -3     
+ Misses        460      458       -2     

Flag	Coverage Δ
unittests	92.63% <ø> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...tainterfaces/ecephys/spike2/spike2datainterface.py	55.17% <ø> (ø)