Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds docs page on denominator degrees of freedom #535

Open
palday opened this issue Jun 24, 2021 · 4 comments
Open

Adds docs page on denominator degrees of freedom #535

palday opened this issue Jun 24, 2021 · 4 comments
Labels
docs Documentation Issues

Comments

@palday
Copy link
Member

palday commented Jun 24, 2021

This is discussed in lots of places elsewhere, but maybe we can consolidate our take and our priorities into a page like the one for rank deficiency.

@palday palday added the docs Documentation Issues label Jun 24, 2021
@dmbates
Copy link
Collaborator

dmbates commented Aug 15, 2021

On a related issue, I have been looking at the method for leverage in src/mixedmodels.jl. As stated in the comments, this is a very inefficient way of determining these values. It uses existing code but at the expense of doing the same calculation over and over again. It calls updateL to re-evaluate the entire L blocked matrix even though the only part that changes for each observation is the last row of L. I have some time now and will work on obtaining these values without all the redundant computation so that we can use "trace of the hat matrix" in degrees of freedom calculations. As it stands it takes 160 seconds on my laptop even with MKL to evaluate the leverage for the "full" more fit to the :mrk17_exp1 data.

@dmbates
Copy link
Collaborator

dmbates commented Aug 16, 2021

The good news is that there is low-hanging fruit for speeding up evaluation of the leverage values. I'll start on that today but it may take a few days.

@palday
Copy link
Member Author

palday commented Aug 16, 2021

Will #545 help with this?

@dmbates
Copy link
Collaborator

dmbates commented Aug 16, 2021

I think the formulation in terms of elementary operations using the blocks themselves is sufficiently straightforward to implement. Each leverage value uses only one diagonal block from m.L[1] in the vector-valued case and that is the big savings. There is no need to deal with the entire block in the first column of blocks. That, by itself, is enough to give substantial time savings.

dmbates added a commit that referenced this issue Aug 22, 2021
* Faster leverage, only works for 1 g.f.

* Use 3block form of L.  Still only applies to 1 gf

* Handle multiple g.f.  Add tests

* Apply suggestions from code review

Co-authored-by: Phillip Alday <palday@users.noreply.github.com>

* Update pls.jl

Loosen the tolerance on a comparison of effective degrees of freedom.  This calculation seems unstable.

* Update pls.jl

Be more careful with my fractions.

* Update src/linearmixedmodel.jl

* NEWS.md entry for #553, first referenced  in #535

* tweak news crossref

Co-authored-by: Phillip Alday <palday@users.noreply.github.com>
Co-authored-by: Phillip Alday <me@phillipalday.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation Issues
Projects
None yet
Development

No branches or pull requests

2 participants