Update to use g_mlm() #33

manchen07 · 2020-05-20T11:06:28Z

Create a new branch as this is an extensive change.
I used a previous branch: updating-to-g_mlm.
Import g_mlm() from lmeInfo.
I put lmeInfo in the Depends field in DESCRIPTION and used importFrom g_mlm(), extract_varcomp(), varcomp_vcov(), and Fisher_info(). I tried moving lmeInfo to the Imports field but the Laski_g_mlm example in confidence-interval-functions.R returns error that g_mlm can not be found.
Import CI_g() from lmeInfo and remove scdhlm::CI_g(). Note that we'll first need to make sure that lmeInfo::CI_g() works with g_REML objects.
I first tried to Defunct scdhlm::CI_g() and import lmeInfo::CI_g(), but CI_g() still calls the function scdhlm::CI_g(). And I am not sure how to make lmeInfo::CI_g() works with g_REML objects. If we do not update CI_g() in lmeInfo, I think the easiest way to update scdhlm::CI_g() is to rewrite the function (this is what I did).
Update the scdhlm vignette to use g_mlm().
Done. But g_mlm() and the inline code are not evaluated now because scdhlm is not updated yet. After this branch gets merged to master branch, I will reinstall scdhlm from Github and then re-knit the Rmd. Should I add confidence intervals for all the models in the vignette?
Update the README to use g_mlm().
Done. Same with the vignette that g_mlm(), CI_g() and the inline code are not evaluated.
Edit documentation to deprecate g_REML().
Added documentation for the deprecated and removed functions.
Update the shiny app to use g_mlm().
Need more time on this.

I am not sure about importing g_mlm() and CI_g(). Please let me know if you would like to do this differently. Thanks!

…from lmeInfo. Deprecate extract_varcomp(), Info_Expected_lmeAR1(), and g_REML() in scdhlm. Defunct summary.g_REML() in scdhlm.

codecov-commenter · 2020-05-20T11:09:33Z

Codecov Report

Merging #33 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #33   +/-   ##
=======================================
  Coverage   52.81%   52.81%           
=======================================
  Files           7        7           
  Lines         640      640           
=======================================
  Hits          338      338           
  Misses        302      302

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56218b5...56218b5. Read the comment docs.

manchen07 · 2020-05-20T17:31:07Z

Update the shiny app to use g_mlm().

In lme-fit.R, I updated lme_fit_MB() and effect_size_RML() with g_mlm().

The functions can handle 3 level or 4 level data now, which includes case/series/within, cluster/case/within, and cluster/case/series/within. I do not know how to update the functions so that they can handle data with arbitrary levels.
I set infotype = "expected in g_mlm() for now. Might need to update the UI if we allow specifying infotype.

In helper-functions.R, I updated summarize_ES().

The rho is not calculated correctly now. I am not sure how to calculate and display rho for models with more than 2 levels.
Should we allow the users to specify symmetric or asymmetric for CI_g()? I chose asymmetric so that the result is consistent with the published shiny app.
Should we add cor_params and var_params estimates to the summary table? For example, if a user specifies a multivariate model with corSymm correlation structure, should we report this information in the summary table?

Sorry for so many questions...Please consider this whole pull request as a draft :) Just wanted to show you what I have done so far. Thank you!

…into updating-to-g_mlm

jepusto

Great work! I've left several comments on some pieces. We can talk through the comments tomorrow. The only major comment I have is on the changes to the shiny app lme-fit functions. Let's talk about your approach here to make sure we're on the same page.

I am getting errors when I use the shiny app to calculate effect sizes for certain models right now. For instance, try using the Schutte data in a model with random intercept & slope in baseline and a fixed slope in treatment (or pretty much any other model).

R/REML-ES-functions.R

README.Rmd

R/confidence-interval-functions.R

inst/shiny-examples/scdhlm/helper-functions.R

jepusto · 2020-05-21T01:38:21Z

I made some minor changes to the README and vignette, so be sure to re-pull.

jepusto · 2020-05-21T02:22:24Z

Hi Man, Responses to your questions below (in addition to the comments on the pull request). James

On Wed, May 20, 2020 at 12:31 PM Man Chen ***@***.***> wrote: Update the shiny app to use g_mlm(). - In lme-fit.R, I updated lme_fit_MB() and effect_size_RML() with g_mlm(). 1. The functions can handle 3 level or 4 level data now, which includes case/series/within, cluster/case/within, and cluster/case/series/within. I do not know how to update the functions so that they can handle data with arbitrary levels. This is definitely above and beyond what we need to worry about for now.

Since you've already made progress on it, let's stick with the 3/4 level structure. We'll eventually need to update the UI to provide options to add these levels, but that will take some work and I want to save it for a later milestone.

1. I set infotype = "expected in g_mlm() for now. Might need to update the UI if we allow specifying infotype. I think it's fine to impose this choice and not provide an option. Anyone

who cares about using infotype = "average" can always move to writing code (especially if we provide template code in the shiny app).

- In helper-functions.R, I updated summarize_ES(). 1. The rho is not calculated correctly now. I am not sure how to calculate and display rho for models with more than 2 levels. Let's discuss. 1. Should we allow the users to specify symmetric or asymmetric for CI_g()? I chose asymmetric so that the result is consistent with the published shiny app. I thought in the Pusto Hedges Shadish paper we ended up recommending

symmetric confidence intervals. So it's odd that the app uses asymmetric. I think I'd prefer to change it to symmetric, but let's discuss tomorrow.

1. Should we add cor_params and var_params estimates to the summary table? For example, if a user specifies a multivariate model with corSymm correlation structure, should we report this information in the summary table? Let's discuss. My initial inclination is to say no, it would be Too Much

Information.

Sorry for so many questions...

Don't be! All very good questions. I remain VERY impressed with your work!

jepusto · 2020-05-21T20:30:18Z

Hi Man, I think I figured out the namespace issues. The trick is that we have to import the functions from lmeInfo, but then also re-export them from scdhlm so that they can be called by a user who has loaded the package (but hasn't loaded lmeInfo). But then if we export the functions, they also have to be documented, so I added a very simple set of Roxygen documentation. See the file R/lmeInfo-imports.R.

manchen07 · 2020-05-21T20:38:58Z

Aah! That makes sense! Thank you!

…

On Thu, May 21, 2020 at 3:30 PM James E. Pustejovsky < ***@***.***> wrote: Hi Man, I think I figured out the namespace issues. The trick is that we have to import the functions from lmeInfo, but then also re-export them from scdhlm so that they can be called by a user who has loaded the package (but hasn't loaded lmeInfo). But then if we export the functions, they also have to be documented, so I added a very simple set of Roxygen documentation. See the file R/lmeInfo-imports.R. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#33 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN4DSOVLAGFCS7XABKVLQNTRSWFORANCNFSM4NF2UUFQ> .

-- Man Chen, M.Ed. Doctoral student Quantitative Methods, Department of Educational Psychology University of Texas at Austin

…hod for g_REML. Added print method for g_REML.

manchen07 · 2020-05-21T23:06:32Z

I am getting errors when I use the shiny app to calculate effect sizes for certain models right now. For instance, try using the Schutte data in a model with random intercept & slope in baseline and a fixed slope in treatment (or pretty much any other model).

Was the error "object fixed not found"? I got this error. Then I added X_design and Z_design back and it seems that shiny app is working. I do not understand why we need them for g_mlm(). Please let me know if you get a different error. I can debug more.

manchen07 · 2020-05-21T23:13:35Z

I am not sure why Travis CI build failed...

jepusto · 2020-05-22T02:08:35Z

Do we need to export extract_varcomp_ex()? I don't see why anyone would need the function, and it is not exported in the master branch.

…ction is no longer exported.

jepusto · 2020-05-22T02:45:51Z

I am getting errors when I use the shiny app to calculate effect sizes for certain models right now. For instance, try using the Schutte data in a model with random intercept & slope in baseline and a fixed slope in treatment (or pretty much any other model).

Was the error "object fixed not found"? I got this error. Then I added X_design and Z_design back and it seems that shiny app is working. I do not understand why we need them for g_mlm(). Please let me know if you get a different error. I can debug more.

Yes, the error was "object fixed not found". Even with X_design and Z_design back in, the app is still throwing the error. It's also erroring if you choose moment estimation in the Modeling tab.

jepusto · 2020-05-22T03:00:22Z

The bug in the shiny app seems to be a problem with the call to model_matrix(mod, data = nlme::getData(mod)), which happens in lmeInfo::Fisher_info(). There seems to be a problem with how model_matrix() evaluates the formula from mod. Not quite sure what to do about it at the moment so I'm gonna go to bed.

manchen07 · 2020-05-22T03:07:01Z

Perfect! Sleep always helps. I will work on the CI_g and shiny app tomorrow. Thanks for the commits and updates!

On Thu, May 21, 2020 at 10:00 PM James E. Pustejovsky < ***@***.***> wrote: The bug in the shiny app seems to be a problem with the call to model_matrix(mod, data = nlme::getData(mod)), which happens in lmeInfo::Fisher_info(). There seems to be a problem with how model_matrix() evaluates the formula from mod. Not quite sure what to do about it at the moment so I'm gonna go to bed. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#33 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN4DSOX6NMD6TLBZSPEF6RTRSXTFFANCNFSM4NF2UUFQ> .

-- Man Chen, M.Ed. Doctoral student Quantitative Methods, Department of Educational Psychology University of Texas at Austin

manchen07 · 2020-05-22T10:51:20Z

I am getting errors when I use the shiny app to calculate effect sizes for certain models right now. For instance, try using the Schutte data in a model with random intercept & slope in baseline and a fixed slope in treatment (or pretty much any other model).

Was the error "object fixed not found"? I got this error. Then I added X_design and Z_design back and it seems that shiny app is working. I do not understand why we need them for g_mlm(). Please let me know if you get a different error. I can debug more.

Yes, the error was "object fixed not found". Even with X_design and Z_design back in, the app is still throwing the error. It's also erroring if you choose moment estimation in the Modeling tab.

I tried to fix the bugs.

Regarding the error "object fixed not found", in lme-fit.R, mod$call$fixed was the word "fixed" and mod$call$random was "random". I think this is why model_matrix() could not evaluate the formula from mod. So I add the fixed and random formula to mod now.
Regarding the error with moment estimation, that is because effect_size_ABK() and effect_size_MB() save the autocorrelation as "phi" not "cor_params".

I am not sure if the errors are fixed although it seems work on my end. Please let me know how it goes.

…_g to g_REML and g_HPS objects. Not sure about g_mlm objects.

jepusto · 2020-05-22T16:17:25Z

Looking this over now. I think I know how to resolve.

…

On Fri, May 22, 2020 at 10:59 AM Man Chen ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In R/confidence-interval-functions.R <#33 (comment)>: > #' -CI_g <- function(g, cover = 0.95, bound = 35) UseMethod("CI_g") + +CI_g <- function(g, cover = 0.95, bound = 35, symmetric = TRUE) UseMethod("CI_g") I imported CI_g() from lmeInfo. Looks like it is working although I did not import CI_g.g_mlm(). I am super confused with expanding S3 method. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#33 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCVGIUADB5M5SU3T5BNCITRS2OO5ANCNFSM4NF2UUFQ> .

Merge branch 'updating-to-g_mlm' of http://github.com/jepusto/scdhlm into updating-to-g_mlm # Conflicts: # R/confidence-interval-functions.R # man/CI_g.Rd

jepusto · 2020-05-22T16:29:24Z

Okay I think we've got the CI_g issues resolved. Can you think of anything else outstanding? If not, I'll go ahead and merge with the master branch.

manchen07 · 2020-05-22T16:32:33Z

Okay I think we've got the CI_g issues resolved. Can you think of anything else outstanding? If not, I'll go ahead and merge with the master branch.

Yes, please merge! We do need to reinstall scdhlm from Github and update README (I can do this). Not sure if the pkgdown will work but we will see. Thanks!

manchen07 added 7 commits May 20, 2020 05:23

Update Version number, the URL, and the Depends field in DESCRIPTION.

42fce7a

Updated README.

fe1556e

Updated the vignette.

0d465aa

Import g_mlm(), extract_varcomp(), varcomp_vcov(), and Fisher_info() …

ac84463

…from lmeInfo. Deprecate extract_varcomp(), Info_Expected_lmeAR1(), and g_REML() in scdhlm. Defunct summary.g_REML() in scdhlm.

Update CI_g() in scdhlm.

437c2e4

Add unit test to check whether g_mlm() is imported appropriately.

48c284a

Update NAMESPACE.

d5f1a21

manchen07 requested a review from jepusto May 20, 2020 11:06

manchen07 linked an issue May 20, 2020 that may be closed by this pull request

Update to use g_mlm() #14

Closed

Update the shiny app using g_mlm().

6ac9e75

jepusto added 3 commits May 20, 2020 14:26

Edited README.

d2db7d1

Merge branch 'updating-to-g_mlm' of http://github.com/jepusto/scdhlm …

d07e391

…into updating-to-g_mlm

Edited vignette.

2e6074f

jepusto requested changes May 21, 2020

View reviewed changes

Fixed namespace issues with lmeInfo.

b5bea5d

manchen07 added 7 commits May 21, 2020 17:34

Deleted varcomp_vcov and added extract_varcomp.

55eb1f8

Updated documentation.

49645dd

Renamed extract_varcomp to extract_varcom_ex. Updated the summary met…

935b185

…hod for g_REML. Added print method for g_REML.

Updated the unit test.

a0ff9b3

Allow exporting extract_varcomp_ex.

a8484b4

Removed the 3/4 levels code. Debugged shiny app.

88d9655

Updated NEWS (needs to be edited).

74fce93

Edited NEWS. Fixed typo in scdhlm-deprecated Roxygen.

b0fcbeb

jepusto added 2 commits May 21, 2020 21:21

Renamed extract_varcomp_ex to extract_varcomp_lmeAR1. The renamed fun…

c7e1a11

…ction is no longer exported.

Tweaked summary.g_REML() for style.

c38ef41

Fiddling with shiny app.

e2d5441

Possibly fixed the bugs in shiny app.

14f9bd9

jepusto and others added 7 commits May 22, 2020 10:01

Reviewed changes to shiny app functions.

822823a

Imported CI_g from lmeInfo.

9d1def0

Updated documentation.

bab33df

Removed the definition of CI_g method in scdhlm. Extended lmeInfo::CI…

db95579

…_g to g_REML and g_HPS objects. Not sure about g_mlm objects.

Removed CI_g.mlm from scdhlm.

d73131f

Fixed the duplicated CI_g alias.

f67fb08

CI_g imported from lmeInfo.

0902236

Cleaned up CI_g import/export/documentation.

56218b5

Merge branch 'updating-to-g_mlm' of http://github.com/jepusto/scdhlm into updating-to-g_mlm # Conflicts: # R/confidence-interval-functions.R # man/CI_g.Rd

jepusto approved these changes May 22, 2020

View reviewed changes

jepusto merged commit eb961e1 into master May 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to use g_mlm() #33

Update to use g_mlm() #33

manchen07 commented May 20, 2020

codecov-commenter commented May 20, 2020 •

edited

Loading

manchen07 commented May 20, 2020

jepusto left a comment

jepusto commented May 21, 2020

jepusto commented May 21, 2020 via email

jepusto commented May 21, 2020

manchen07 commented May 21, 2020 via email

manchen07 commented May 21, 2020

manchen07 commented May 21, 2020

jepusto commented May 22, 2020

jepusto commented May 22, 2020

jepusto commented May 22, 2020

manchen07 commented May 22, 2020 via email

manchen07 commented May 22, 2020

jepusto commented May 22, 2020 via email

jepusto commented May 22, 2020

manchen07 commented May 22, 2020

Update to use g_mlm() #33

Update to use g_mlm() #33

Conversation

manchen07 commented May 20, 2020

codecov-commenter commented May 20, 2020 • edited Loading

Codecov Report

manchen07 commented May 20, 2020

jepusto left a comment

Choose a reason for hiding this comment

jepusto commented May 21, 2020

jepusto commented May 21, 2020 via email

jepusto commented May 21, 2020

manchen07 commented May 21, 2020 via email

manchen07 commented May 21, 2020

manchen07 commented May 21, 2020

jepusto commented May 22, 2020

jepusto commented May 22, 2020

jepusto commented May 22, 2020

manchen07 commented May 22, 2020 via email

manchen07 commented May 22, 2020

jepusto commented May 22, 2020 via email

jepusto commented May 22, 2020

manchen07 commented May 22, 2020

codecov-commenter commented May 20, 2020 •

edited

Loading