Publication in JOSS #36

strengejacke · 2019-03-12T14:23:24Z

What do you think about preparing (and submitting) a paper for JOSS?

https://joss.theoj.org/

It would be a very short paper, mostly similar to a readme-file, but would have the advantage of a "nicer" citation for the package. The journal has an ISSN, so it's being considered as "citable".

DominiqueMakowski · 2019-03-12T14:38:50Z

Yes definitely! However, I would like the JOSS paper to be a bit longer than usual (maybe by including some examples or things from the vignettes). Maybe around 1-2 pages? The shameful reason for that is that my two previous JOSS papers have been weirdly look at on researchgate by people not used to "technical publications"... 😅

"Ah you have published a new paper! Let's see the full text..." *clicks on "full-text"*

*nothing changes*

*tries to scroll past the first half-page*

*nothing changes*

strengejacke · 2019-03-12T14:48:21Z

:-D

I think there are no limitations to number of pages. My ggeffects-paper has 4 pages, I think. I just wanted to say that it should not be too much effort...

DominiqueMakowski · 2019-04-21T12:23:26Z

We should do this independently of the other paper. Maybe we could (re)use the content of the README for this one?

DominiqueMakowski · 2019-06-10T08:48:58Z

Requirements:

good paper title
statement of need
conceptual presentation
features presentation (in the README)

strengejacke · 2019-06-21T20:43:17Z

I can give it a try next week and work on the statement of need. I think the introduction as it currently stands is too long and too generic. We might shorten a bit here and then focus a bit more on what is currently missing (and thus needed), and how bayestestR can help here.

DominiqueMakowski · 2019-06-22T01:42:22Z

okay. however if possible I would rather add what's missing rather than replace the current generic intro (altho it can definitely be streamlined); since bayestestR might be a point of entry for beginners to Bayesian stats, and that the package is not only the functions but the documentation, it would not hurt having such few paragraphs presenting a bit the idea.

Also, although fairly decent, I feel like the title could be improved to be more impactful/catchy :)

mattansb · 2019-06-22T05:45:00Z

since bayestestR might be a point of entry for beginners to Bayesian stats, and that the package is not only the functions but the documentation

😎

DominiqueMakowski · 2019-06-22T05:51:34Z

since bayestestR might be a point of entry for beginners to Bayesian stats, and that the package is not only the functions but the documentation [as well]

Well at least I have a strong prior that it is the case ¯_(ツ)_/¯

strengejacke · 2019-06-25T07:59:51Z

Ok, I started working on the paper. I streamlined the intro a bit, and will now add a statement of need.

To summarize my idea behind the changes:

I removed the frq-paragraph with repro-crisis, not 100% sure if this paragraph captures the story correctly
I removed the examples with "correlation" and ".42", because I think the message is clear w/o those examples.
I think we need a transition to the statement of need and the content of the paper. I like the idea, and I think this is most accessible / readable, of the four characteristics of a posterior: centrality, uncertainty, existance and significance. Thus, the intro ends with

In essence, the Bayesian sampling algorithms (such as MCMC sampling) return a probability distribution (the posterior) of an effect that is compatible with the observed data. The effect can be described by characterizing the posterior distribution of the related effect in relation to its centrality (point-estimates), uncertainty, existance and significane.

and I will now continue with the statement of need (that bayestestR boldly goes where no package has gone before) and then we almost have the same structure "centrality (point-estimates), uncertainty, existance and significane" already in the paper, which makes it easy to follow.

strengejacke · 2019-06-25T09:05:52Z

I added the statement of need to the paper, and a shorter version to the readme.

We need to think about where to put which function. If we really follow the outline "centrality (point-estimates), uncertainty, existance and significane", which functions goes where?

Centrality / Point-Estimate

map_estimate()

Uncertainty

hdi()
ci()

Existance (Evidence?)
Significance (Evidence?)

rope(), equivqalence_test(), p_direction(), p_map(), bayesfactor(), ... Suggestions where to place these functions? I would suggest:

Existance (Evidence?)

rope()
equivqalence_test()
bayesfactor()

Significance (Evidence?)

p_direction()
p_map()
p_rope()

Or we could combine 3. and 4. under a section "Existence and Evidence" or so.

strengejacke · 2019-06-25T09:09:40Z

I'll wait for feedback before continuing here.

mattansb · 2019-06-25T09:16:47Z

I think:

p_map and pd should be moved to Existance (together with they BF they answer the Q: is the parameter different than 0?)
equivqalence_test and rope/p_rope should be moved to Significance (they BF they answer the Q: is the parameter meaningfully different than 0?)

Or we could combine 3. and 4. under a section "Existence and Evidence" or so.

Also a possibility...

mattansb · 2019-06-25T09:17:39Z

This is a short paper, right? I might not get to it before Thursday... Sorry..

strengejacke · 2019-06-25T09:19:55Z

This is a short paper, right?

Yes. JOSS-papers typically are about 2 to 5 pages, I think. The current insight-paper is an exception with some more pages... The focus for JOSS also lies on well-documented packages, including the website. So we need to take care of readme, website, package-docs and papers. In short, the repository typicall reqires a certain outline and content.

DominiqueMakowski · 2019-06-25T09:29:28Z

let me read

DominiqueMakowski · 2019-06-25T09:59:44Z

I've rephrased the statement a bit. I am also wondering about changing the title from:

"Understanding and Describing Bayesian Models with bayestestR"

to something like:

"Estimating Effects and their Uncertainty, Existence and Significance in the Bayesian Framework using BayestestR"

A bit longer but more accurate and punchy?

strengejacke · 2019-06-25T10:04:50Z

Revisions look good! According to the title: "Estimating Effects" is probably misleading, else I think a longer title is ok!

strengejacke · 2019-06-25T10:06:32Z

We may use your revisions from the last intro-paragraph for the readme as well.

DominiqueMakowski · 2019-06-25T10:08:19Z

p_map and pd should be moved to Existance (together with they BF they answer the Q: is the parameter different than 0?)
equivqalence_test and rope/p_rope should be moved to Significance (they BF they answer the Q: is the parameter meaningfully different than 0?)

pd is 100% an index of effect existence (about the certainty associated with a directionality). Rope percentage and equivalence test are 100% indices of effect significance (in the original meaning, "important" as in non-negligible).

For p_rope and p_map, I am not so sure, I would tend to say respectively significance and existence (but it's less clear to me). For BF, it seems like it could be both depending on the hypothesis being tested (0, ]-Inf, 0[, or ROPE)

DominiqueMakowski · 2019-06-25T10:10:59Z

title -> Understanding and Describing Effects and their Uncertainty, Existence and Significance in the Bayesian Framework using BayestestR?

DominiqueMakowski · 2019-06-25T10:21:32Z

bayestestR: Understanding and Describing Effects and their Uncertainty, Existence and Significance in the Bayesian Framework

mattansb · 2019-06-25T10:25:19Z

For BF, it seems like it could be both depending on the hypothesis being tested

Good point! In that case it might be better to merge 3 and 4... I can add some explanation about the usage that is possible with out code regarding BFs (focusing on models and sdBF I think is most concise for this short ms).

DominiqueMakowski · 2019-06-25T10:53:11Z

In that case, maybe let's merge it under "NHST" here and we will discuss this category (and the distinction between existence and significance) in details in the other paper?

strengejacke · 2019-06-25T11:24:32Z

Sounds good! Shall I continue to work on the paper, or do you want to do the next step? I'm fine with both.

DominiqueMakowski · 2019-07-02T14:41:36Z

Link to review: openjournals/joss-reviews#1541

#36

strengejacke · 2019-07-30T14:07:15Z

Shall we add a new release now, to register a zenodo-DOI for the latest version?

DominiqueMakowski · 2019-07-30T14:10:32Z

I would wait for the editor's decision

strengejacke · 2019-07-30T14:12:10Z

probably makes sense...

DominiqueMakowski · 2019-07-30T14:13:14Z

😁

strengejacke · 2019-07-30T14:15:51Z

Looks like we might manage it to get the JOSS citation in time for the requested CRAN update, right?

mattansb · 2019-07-30T14:16:23Z

Amen to that!

DominiqueMakowski · 2019-07-30T14:19:25Z

https://www.youtube.com/watch?v=NgqtL8NqyOM

mattansb · 2019-07-30T14:21:52Z

@DominiqueMakowski The most advance SW gif yet...

strengejacke · 2019-08-01T09:37:11Z

Shall we ping the editor?

DominiqueMakowski · 2019-08-01T10:54:06Z

tbh I would find it a bit impolite haha, it's summer after all some people do have a life (from what I've heard) :)

the cran deadline require us to submit the 8th at max. so I say we wait and worst case scenario we submit as is the 0.2.5 and the updated ref will go into the future update no big deal to me...

strengejacke · 2019-08-01T11:18:35Z

True, didn't want to push or be impatient. Usually reviewers explicitly ping the handling editor saying they are done, not sure if was clear 😉

Anyway, we can submit w/o joss citation in "worst case" (that case wouldn't be too bad anyway)

DominiqueMakowski · 2019-08-06T04:49:11Z

since CRAN submissions are closing from 9 to 18, I suggest submitting this as-is now, and parameters down the road if it gets accepted fast. What do you say?

mattansb · 2019-08-06T05:31:12Z

Fine with me.

strengejacke · 2019-08-06T06:00:38Z

Yes, let's submit bayestestR (after win builder checks?).

DominiqueMakowski · 2019-08-06T06:17:25Z

@strengejacke if you have access to a computer can submit to winbuilder, otherwise I'll do it in 1 hour

strengejacke · 2019-08-06T07:10:45Z

I just arrived in the office and can do it if not too late

strengejacke · 2019-08-06T07:28:06Z

ok, done,

DominiqueMakowski · 2019-08-06T07:56:10Z

https://win-builder.r-project.org/HkM0svA1p2sV
The files will be removed after roughly 72 hours.
Installation time in seconds: 14
Check time in seconds: 715
Status: OK
R Under development (unstable) (2019-07-05 r76784)

https://win-builder.r-project.org/zS4mvi9NvWjS
The files will be removed after roughly 72 hours.
Installation time in seconds: 14
Check time in seconds: 711
Status: 2 ERRORs, 1 NOTE
R version 3.5.3 (2019-03-11)

https://win-builder.r-project.org/59uYjJZgGdft
The files will be removed after roughly 72 hours.
Installation time in seconds: 16
Check time in seconds: 689
Status: OK
R version 3.6.1 (2019-07-05)

DominiqueMakowski · 2019-08-06T07:57:31Z

tests that fail

  -- 1. Failure: emmGrid ci (@test-emmGrid.R#16)  --------------------------------
  xci$CI_low not equal to c(-0.236749774206338, 1.23103419307697, -2.99025704276072).
  3/3 mismatches (average diff: 0.0574)
  [1] -0.266 - -0.237 == -0.0289
  [2]  1.338 -  1.231 ==  0.1070
  [3] -2.954 - -2.990 ==  0.0362
  
  -- 2. Failure: emmGrid p_rope (@test-emmGrid.R#58)  ----------------------------
  xprope$p_ROPE not equal to c(0.695, 1, 0.87).
  2/3 mismatches (average diff: 0.025)
  [1] 0.655 - 0.695 == -0.04
  [3] 0.880 - 0.870 ==  0.01

-> increasing the tol should resolve

Examples too long

Examples with CPU or elapsed time > 10s
   user system elapsed
ci 4.29   0.64   10.07`

DominiqueMakowski · 2019-08-06T08:03:15Z

I'll address and submit

strengejacke · 2019-08-06T08:58:59Z

increasing the tol should resolve

I don't think so, we would end up in having a tolerance for literally any value. Maybe we should skip for R < 3.6? We have had these problems with emmeans before, and I'm not sure why.

mattansb · 2019-08-06T09:06:27Z

Question is - why have the values changed at all - it ran fine last time around, no?

strengejacke · 2019-08-06T13:49:03Z

Yes, strange indeed. I recall i had something similar with one of my other packages, also related to emmeans. Not sure how I fixed it, though.

strengejacke · 2019-08-08T13:24:39Z

@DominiqueMakowski I just saw that the handling editor has changed his last answer in the review-thread into a check-list, not sure if you can check those items? Maybe he waits until these are marked as "done"?

DominiqueMakowski · 2019-08-08T13:29:35Z

I can't check them... but I start to think that Chris has a cyclical schedule of checking the state of its assigned papers like once every x days 😁, thus he'll probably accept on his next round 🤞

strengejacke · 2019-08-13T06:36:55Z

great work, guys, nice publication! @mattansb I think you must request authorship in researchgate, not sure if your account was correctly linked to the entry in RG.

Close this now?

DominiqueMakowski · 2019-08-13T06:54:17Z

Let's close, awesome synergic teamwork, that can now fully focus ON OUR NEXT CHALLENGE 👀 ☺️

DominiqueMakowski added this to To do in 0.2.0 Apr 2, 2019

DominiqueMakowski added this to the 0.2.0 milestone Apr 2, 2019

DominiqueMakowski modified the milestones: 0.2.0, 0.3.0 May 19, 2019

DominiqueMakowski mentioned this issue May 19, 2019

CRAN v2 #132

Closed

7 tasks

DominiqueMakowski removed this from To do in 0.2.0 May 23, 2019

DominiqueMakowski added a commit that referenced this issue Jul 15, 2019

added DOIs

d5dc5ae

#36

DominiqueMakowski closed this as completed Aug 13, 2019

Publication in JOSS #36

Publication in JOSS #36

Comments

strengejacke commented Mar 12, 2019

DominiqueMakowski commented Mar 12, 2019

strengejacke commented Mar 12, 2019

DominiqueMakowski commented Apr 21, 2019

DominiqueMakowski commented Jun 10, 2019 • edited by strengejacke

strengejacke commented Jun 21, 2019

DominiqueMakowski commented Jun 22, 2019

mattansb commented Jun 22, 2019

DominiqueMakowski commented Jun 22, 2019 • edited

strengejacke commented Jun 25, 2019

strengejacke commented Jun 25, 2019 • edited

strengejacke commented Jun 25, 2019

mattansb commented Jun 25, 2019 • edited

mattansb commented Jun 25, 2019

strengejacke commented Jun 25, 2019

DominiqueMakowski commented Jun 25, 2019

DominiqueMakowski commented Jun 25, 2019 • edited

strengejacke commented Jun 25, 2019

strengejacke commented Jun 25, 2019

DominiqueMakowski commented Jun 25, 2019 • edited

DominiqueMakowski commented Jun 25, 2019

DominiqueMakowski commented Jun 25, 2019

mattansb commented Jun 25, 2019

DominiqueMakowski commented Jun 25, 2019

strengejacke commented Jun 25, 2019

DominiqueMakowski commented Jul 2, 2019 • edited

strengejacke commented Jul 30, 2019

DominiqueMakowski commented Jul 30, 2019

strengejacke commented Jul 30, 2019

DominiqueMakowski commented Jul 30, 2019

strengejacke commented Jul 30, 2019

mattansb commented Jul 30, 2019

DominiqueMakowski commented Jul 30, 2019

mattansb commented Jul 30, 2019

strengejacke commented Aug 1, 2019

DominiqueMakowski commented Aug 1, 2019

strengejacke commented Aug 1, 2019

DominiqueMakowski commented Aug 6, 2019 • edited

mattansb commented Aug 6, 2019

strengejacke commented Aug 6, 2019

DominiqueMakowski commented Aug 6, 2019

strengejacke commented Aug 6, 2019

strengejacke commented Aug 6, 2019

DominiqueMakowski commented Aug 6, 2019 • edited

DominiqueMakowski commented Aug 6, 2019 • edited

tests that fail

Examples too long

DominiqueMakowski commented Aug 6, 2019

strengejacke commented Aug 6, 2019

mattansb commented Aug 6, 2019

strengejacke commented Aug 6, 2019

strengejacke commented Aug 8, 2019 • edited

DominiqueMakowski commented Aug 8, 2019

strengejacke commented Aug 13, 2019

DominiqueMakowski commented Aug 13, 2019

DominiqueMakowski commented Jun 10, 2019 •

edited by strengejacke

DominiqueMakowski commented Jun 22, 2019 •

edited

strengejacke commented Jun 25, 2019 •

edited

mattansb commented Jun 25, 2019 •

edited

DominiqueMakowski commented Jun 25, 2019 •

edited

DominiqueMakowski commented Jun 25, 2019 •

edited

DominiqueMakowski commented Jul 2, 2019 •

edited

DominiqueMakowski commented Aug 6, 2019 •

edited

DominiqueMakowski commented Aug 6, 2019 •

edited

DominiqueMakowski commented Aug 6, 2019 •

edited

strengejacke commented Aug 8, 2019 •

edited