Question: Solve the problem in new versions of np.linalg.solve

[quant12345]

Describe the issue:

Hi, all!
I made a pull-request in statsmodels and my test does not work, where numpy version is --- 2.1.0.dev0. The error occurs in: np.linalg.solve(cm, resid_take).

The dimensions are as follows:
cm.shape (200, 5, 5) resid_take.shape (200, 5)

In the error:
ValueError: solve: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (m,m),(m,n)->(m,n) (size 200 is different from 5)

I've read here and other places but can't figure out how I need to change the arrays to make the code work in newer versions?

Reproduce the code example:

https://github.com/statsmodels/statsmodels/pull/9264

Error message:

statsmodels/regression/tests/test_processreg.py:171: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
statsmodels/regression/tests/test_processreg.py:160: in run_formula
    f = preg.fit()
statsmodels/regression/process_regression.py:730: in fit
    f = minimize(
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/scipy/optimize/_minimize.py:722: in minimize
    res = _minimize_powell(fun, x0, args, callback, bounds, **options)
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/scipy/optimize/_optimize.py:3479: in _minimize_powell
    fval = squeeze(func(x))
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/scipy/optimize/_optimize.py:526: in function_wrapper
    fx = function(np.copy(x), *(wrapper_args + args))
statsmodels/regression/process_regression.py:731: in <lambda>
    lambda x: -self.loglike(x),
statsmodels/regression/process_regression.py:563: in loglike
    ls = np.linalg.solve(cm, resid_take)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
ValueError: solve: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (m,m),(m,n)->(m,n) (size 200 is different from 5)

Python and NumPy Versions:

Python 3.10.12
NumPy - 2.1.0

Runtime Environment:

No response

Context for the issue:

No response

[mattip]

Looking at (m,m),(m,n)->(m,n) with inputs (200,5,5) and (200,5). The first input is understood as m=5 (the assignment is right-to-left). The second one has m=200, n=5. Since in the first m=5, and in the second m=200, the detection fails.

[quant12345]

@mattip how to solve this. It is perhaps to swap the axes a?

[rkern]

No, just reshape it to be (200, 5, 1); that is, make it a stack of explicit 1-column matrices. There is a helpful change note in the docstring explaining what changed in numpy 2.0.

[rkern]

This will work in both 1.x and 2.x numpy builds.

As (I think) we've answered the question and there is no coding change that needs to be made, I am going to close this. Do feel free to continue asking more questions in this closed issue if you need any more clarification, though.

[quant12345]

@rkern @mattip Thank you!

Did this:

ls = np.linalg.solve(cm, resid_take.reshape(200, 5, 1)).reshape(200, 5)