[Inference] The Default Threshold for Cholesky Decomposition

[kayween]

The default threshold for Cholesky decomposition (as opposed to CG) is 800, which seems to be very small.

linear_operator/linear_operator/settings.py

Lines 392 to 400 in dbed373

	class max_cholesky_size(_value_context):
	"""
	If the size of of a LinearOperator is less than `max_cholesky_size`,
	then `root_decomposition` and `solve` of LinearOperator will use Cholesky rather than Lanczos/CG.
	(Default: 800)
	"""
	_global_value = 800

I was a bit surprised that an exact GP trained on 1000 data points was using CG for inference. Because I somehow vaguely remember this threshold was increased to 6000 a long time ago. (I tried to find relevant issues/PRs for reference but I couldn't. So maybe it's my hallucination.)

In any case, this threshold was set 6 years ago. Hardware are much better nowadays. Any thoughts on increasing this threshold? Probably need to benchmark inference speed again.