Less strict numpy and pyarrow dependencies

[Tom-Newton]

Currently this library has quite strict requirements for pyarrow and numpy versions which significantly limits the usefulness. As far as I can tell this library does not require any particular features of numpy or pyarrow added in recent versions.

Resolves: #55

I tested building the wheel locally with poetry build and using it in my project and it worked as expected.

I've had difficulty updating the poetry.lock file though. I've never used poetry before but I think I need to run poetry update. Unfortunately this seems to be taking an extremely long time (> 20 minutes) to the point that I think it must be broken.

Signed-off-by: Thomas Newton thomas.w.newton@gmail.com

[Tom-Newton]

It looks like numpy is barely used in this library so I think pretty much any version should work. I chose >=1.16.6 because that is what the latest version of pyarrow requires.

[Tom-Newton]

It looks like the special requirement for python 3.11 was added recently in #60 so I assume that is important but I changed it to accept >= to this version.

I also think we should only enforce this requirement on python 3.11

[Tom-Newton]

Probably we could allow even older versions but I think this should make most people happy.

[Tom-Newton]

Looks like there is one failure on the CI checks

Download action repository 'actions/checkout@v2' (SHA:dc323e67f16fb5f7663d20ff7941f27f5809e9b6)
Warning: Failed to download action 'https://api.github.com/repos/actions/checkout/tarball/dc[32](https://github.com/databricks/databricks-sql-python/actions/runs/4168062376/jobs/7214961091#step:1:36)3e67f16fb5f7663d20ff7941f27f5809e9b6'. Error: Resource temporarily unavailable (api.github.com:443)
Warning: Back off 22.447 seconds before retry.
Warning: Failed to download action 'https://api.github.com/repos/actions/checkout/tarball/dc323e67f16fb5f7663d20ff7941f27f5809e9b6'. Error: Resource temporarily unavailable (api.github.com:443)
Warning: Back off 24.765 seconds before retry.
Error: Resource temporarily unavailable (api.github.com:443)

I don't think this can be because of my changes. I guess its just a transient thing but I don't think I can re-trigger it.

[Tom-Newton]

Thanks for triggering the checks @susodapop. It looks like all the unittests passed for all the different python versions. One of the code quality checks is failing though. From the error (Error: Resource temporarily unavailable (api.github.com:443)) I would guess a retry would solve it.

If someone could retry that check and review the PR that would be much appreciated.

[Tom-Newton]

@susodapop sorry to be impatient but please could you review this? I think this is about to start causing us some significant problems if its not fixed.

[susodapop]

Not impatient at all! We just needed to freeze our dependencies to get v2.4.0 out the door. Else this would have merged already.

[susodapop]

@Tom-Newton can you grant me push on your fork? I need to re-generate the poetry.lock file before I can merge this but I can't push my version to your fork right now.

Also this reminds me that I need to add a segment to our CONTRIBUTING.md file about updating the project dependencies. Unless you regenerate the poetry.lock file then none of the changes are actually picked up during our unit/e2e tests 🤦

[Tom-Newton]

Superseded by #90