ENH: Drop numpy 1.16 in line with NEP 29

[jamesmyatt]

Is your feature request related to a problem?

In order to address #38100 , I'd like to update pandas.core.common.random_state and anything that uses it to use the new and improved numpy.random API introduced in NumPy 1.17. That change is not the subject of this issue/discussion.

However, the easiest/cheapest/cleanest way to implement this would be to assume numpy 1.17, since random_state is only used by the sample method, rather than implement adapters or defensive programming. The NEP 29 deprecation policy gives the deprecation date for numpy 1.16 as Jan 13, 2021, so it would be in-line with this policy to deprecate.

I'd like to discuss this proposed change and the approach for how to implement it. In #38100, @jreback said he was "not averse" to this.

Describe the solution you'd like

I think that the only change initially would be to update install config files (e.g. setup.cfg) and docs to update the minimum numpy version, but no other code changes.

I think this change should be part of pandas 1.3.0 release. In line with https://github.com/pandas-dev/pandas/blob/master/pyproject.toml#L9, I suggest to make 1.17.3 the minimum numpy version.

Later pull requests would incrementally remove defensive logic to handle numpy 1.16 (e.g. pandas.compat.numpy.np_version_under1p17). Enhancements that depend on numpy 1.17 (e.g. using numpy.random.Generator) could be implemented in parallel with the work to tidy the compatibility code.

I'm happy if someone else wants to take this on, but otherwise I should be able to contribute some PRs myself.

API breaking implications

Mandatory dependency version would be incremented, but this is in line with the published deprecation policy.

pandas functionality should otherwise be unaffected since it already works with numpy 1.17+.

Describe alternatives you've considered

Doing nothing would be against the published deprecation policy and delay opportunities to implement enhancements that exploit numpy 1.17.

Additional context

N/A

[phofl]

We have a ci using numpy 1.16, has to be updated too then

[rhshadrach]

+1

[jreback]

@jamesmyatt happy to take updates to use 1.17 features that are now available by default