PERF Support converting 32-bit matrices directly to liblinear format …

[alexhenrie]

This decreases the memory required for regression by about 33% on 32-bit dense matrices and by about 42% on 32-bit CSR matrices while not noticeably affecting the running time in any 32-bit or 64-bit case. Direct support for 32-bit matrices is the ideal solution for my group because we only need 32-bit precision and cutting the memory requirement by a third will get us inside our servers' memory limits.

[jnothman]

Nice! Could you please double check that this code path has test coverage?

[jnothman]

Is the optimiser likely to compile this out of the loop?

[alexhenrie]

Yes, but even if it doesn't, the CPU's branch predictor will reduce the cost of the if statement to zero.

[jnothman]

Otherwise lgtm

[rth]

Thanks @alexhenrie I have not reviewed it in detail but the results look nice!

Could you please double check that this code path has test coverage?

Yes, for such refactorings I think we need to be sure that this doesn't change the obtained coefficients and predictions. There are existing tests that check coef_ equality between solvers but we need to ensure,

• that the used absolute tolerance is sufficient to detect possible issues
• that those are indeed running on sparse and also 32 bit. Maybe adding the test_dtype_match could be useful. or adding/changing the pytest parametrization of existing tests.

[jnothman]

liblinear already treats the features as 32 bit. Our current code was casting to 64 bit in order to put the data into our helper containers, then casting it to 32 bit for liblinear's structs. So unless the C cast and the numpy cast are different, this should give identical results.

[alexhenrie]

struct feature_node in linear.h represents the value in double precision, so I would expect a slight loss in precision if the inputs are limited to 32-bit originally.

What file do you want the new tests in?

[alexhenrie]

Ah, I see now. test_dtype_match is an existing function that should check that liblinear's output is the same with 32-bit dense input, 32-bit sparse input, 64-bit dense input, or 64-bit sparse input. I just pushed a commit that does exactly that.

test_dtype_match was also checking that the output precision was the same as the input precision. However, it is statistically correct for the logistic regression output to have a higher precision than any single one of its inputs: For example, if the input variables are all binary (0 or 1), it doesn't matter whether the inputs are floats or doubles, the output can still be perfectly calculated to double precision. Therefore I have removed the tests that limit the output precision to the input precision.

The new tests exposed a bug in the 'saga' solver where using sparse input instead of dense input changes the output significantly. I have not figured out how to fix this bug.

[jnothman]

Sorry, my mistake, liblinear processes in float64 (double), but we are still doing the same cast as before, so I don't think we've changed the fit here, have we?

[jnothman]

The new tests exposed a bug in the 'saga' solver where using sparse input instead of dense input changes the output significantly. I have not figured out how to fix this bug.

Sometimes the fit will be different, but not erroneous. I assume you mean that the sparse handling is wrong?

[alexhenrie]

@jnothman Oh, I see what you mean now. If you have 32-bit inputs, the output before and after this patch is necessarily the same because the only difference is when the conversion to 64-bit happens.

I think there's a bug because according to the new tests, the output of the 'saga' solver varies much more than I would expect. I worked around the problem by increasing the tolerance when using 'saga'. Can you think of a better solution?

[alexhenrie]

Strangely, when I set fit_intercept=False, the 'saga' solver gives consistent output. But I don't know if there is a bug in the intercept calculation or if calculating an intercept just magnifies a problem that comes from somewhere else.

[alexhenrie]

@jnothman Could you merge these commits now please? :-)

[rth]

Why this change? The whole point of adding support of 32bit in other solvers was to ensure that coef_ is of the same dtype as X. We can't remove that constraint for other solvers at least.

[alexhenrie]

As I explained in #14296 (comment), there is no reason to prevent any solver from using 32-bit input to produce 64-bit output.

[rth]

Same as above

[rth]

As I explained in #14296 (comment), there is no reason to prevent any solver from using 32-bit input to produce 64-bit output.

That's what we are trying to prevent explicitly as part of #8769, and this would remove the associated tests. The goal is to reduce memory usage and allow faster BLAS operations for those that support AVX* etc when float32 input is provided. So I think tests affecting other solvers should not be modified in this aspect, if those tests don't work for liblinear, you can also create a separate test.

[alexhenrie]

Fine, I just changed the tests back for all solvers except liblinear. Whatever it takes to get this pull request accepted.

[jeremiedbb]

liblinear uses double precision. You want to avoid a copy by directly converting float to double during the conversion to liblinear format. Did I understand correctly ?

[jeremiedbb]

Please use pytest.skip(some informative message) instead of return

[alexhenrie]

Done.

[jeremiedbb]

maybe explain the reason of the FIXME by adding a comment saying that saga does not correctly fit the intercept on sparse data with the default tol and max_iter parameters.

[alexhenrie]

Done.

[jeremiedbb]

maybe use a char* for consistency. Also, since (np.array).data gives a char* we can be specific

[alexhenrie]

This is exactly the kind of thing void* is for: In no case do we want to read the array data as chars because that would result in gibberish, and it would be a mistake to do so. void* prevents data access until it has been determined whether the array is an array of floats or an array of doubles.

Nonetheless, the machine code generated is the same whether x is defined as void* or char*. So if you want to give up the protection against accidentally accessing the array data as chars, it won't hurt anything to use char*. Let me know and I'll make the change.

[jeremiedbb]

Why did you move Y to last position ? it makes the change log harder to read :)

[alexhenrie]

liblinear uses double precision. You want to avoid a copy by directly converting float to double during the conversion to liblinear format. Did I understand correctly ?

Yes, that is correct. The space wasted by having to represent the input datapoints as doubles instead of floats puts us over our server's memory limit.

[alexhenrie]

Why did you move Y to last position ? it makes the change log harder to read :)

I wanted all of the variables describing X to be next to each other. I'll change it back if you prefer it the other way, although it won't make the diff much more readable.

[jnothman]

Suggested change

	pytest.skip('liblinear does not support multinomial classes')
	pytest.skip('liblinear does not support multinomial logistic')

[alexhenrie]

Done.

[alexhenrie]

@jeremiedbb Have I answered your concerns to your satisfaction?

[jeremiedbb]

Sorry for the delay. You're right that void* is a better practice. But numpy decided to use char*. Maybe there's a good reason, I've no idea. So I still think we should stick to char*. Otherwise it makes the code harder to follow: char* converted to void* converted to double*... :)

[alexhenrie]

@jeremiedbb Okay, I just changed the two void * to char *. Anything else?

[jeremiedbb]

Thanks @alexhenrie !

[alexhenrie]

Thank you so much! This is an enormous help to me and my colleagues :)