Update GP ARD docs to use built-in cov function

[amas0]

Submission Checklist

• Builds locally
• New functions marked with <<{ since VERSION }>>
• Declare copyright holder and open-source license: see below

Summary

Slight rewrite of the Automatic Relevance Determination section the Stan User's guide on Gaussian processes.

Proposed to fix #856

[bob-carpenter]

Thanks for opening a PR.

I'm sorry I wasn't clear before.

We try very hard in the User's Guide to present fairly optimal code. If we use less efficient code, our users will copy it and be unhappy.

It is OK to provide a less efficient alternative for pedagogical purposes before introducing the more efficient variant. Here, you can leave the Cholesky-based version, then note that various pieces add up to the same thing as doing it directly with a dense covariance matrix rather than its Cholesky factor.

[bob-carpenter]

My last comment was in the wrong place. Sorry for being misleading before, but we do not want to remove the efficient version of this with Cholesky factors.

What I would recommend is adding a note that shows the equivalent dense covariance matrix parameterization and mentions it is less efficient than working with Cholesky factors.

We do not want to provide code that users copy that isn't efficient.

[amas0]

I apologize, but I seem to have gotten confused along the way here. I raised this issue because for my problem the current code in the guide is significantly *less efficient.* The suggested change still uses Cholesky factors but replaces a manual construction of `K` with the built-in `gp_exp_quad_cov`. Let me try to clarify a bit: I had been using the Cholesky factor version of the ARD covariance matrix that was provided in the User's Guide, i.e. I just copy-pasted the user-defined `L_gp_exp_quad_cov_ARD`. The wording in the User's guide: "...though it currently requires the user to directly code the covariance matrix" implied to me that this code was provided because the approach used in other sections of the guide (such as under "Latent variable GP") -- where `K` is constructed with `gp_exp_quad_cov` and then the Cholesky factor `L_K` is constructed with `cholesky_decompose(K)` -- did not work with varying length scales across the input dimensions. When I was looking through the Function's reference, I found that `gp_exp_quad_cov` *did* support varying length scales. So I tried passing an array of length scales to `gp_exp_quad_cov` and computing the Cholesky factor from that, instead of using `L_gp_exp_quad_cov_ARD`. When I did this, the wall time of sampling from my model was significantly faster and produced similar results. I don't know what the differences are under the hood, but for my problem using `L_gp_exp_quad_cov_ARD` seemed to be noticeably less efficient -- my best guess is that the provided code in the guide manually constructs the dense matrix `K` instead of doing it with the built-in?

…

On Tue, Jan 28, 2025 at 11:54 AM Bob Carpenter ***@***.***> wrote: ***@***.**** requested changes on this pull request. My last comment was in the wrong place. Sorry for being misleading before, but we do not want to remove the efficient version of this with Cholesky factors. What I would recommend is adding a note that shows the equivalent dense covariance matrix parameterization and mentions it is less efficient than working with Cholesky factors. We do not want to provide code that users copy that isn't efficient. — Reply to this email directly, view it on GitHub <#858 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3SOHZWZZPDUPSEXLKCAJL2M6Y4HAVCNFSM6AAAAABV7LZGUKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDKNZYHA4DENJYGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bob-carpenter]

Thanks, @amas0. My bad. I didn't see that the L_... form wasn't a built-in but was something the user built from scratch with their own squared exponential kernel.

I'll go back and approve now that I see what's going on!

[bob-carpenter]

Looks good---I missed the first time that the L_gp_exp_quad_cov_ARD was user-defined!