cbc_power - "NA" generated for 'estimated coefficient' and 'standard errors' in Output #6
Comments
|
You sometimes can get In the example that you referenced here, I cannot replicate the set.seed(1234)
library(cbcTools)
# A simple conjoint experiment about apples
# Generate all possible profiles
profiles <- cbc_profiles(
price = c(1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5),
type = c("Fuji", "Gala", "Honeycrisp"),
freshness = c('Poor', 'Average', 'Excellent')
)
# Make a randomized survey design
design <- cbc_design(
profiles = profiles,
n_resp = 300, # Number of respondents
n_alts = 3, # Number of alternatives per question
n_q = 6 # Number of questions per respondent
)
# Simulate random choices
data <- cbc_choices(
design = design,
obsID = "obsID"
)
# Conduct a power analysis
power <- cbc_power(
data = data,
pars = c("price", "type", "freshness"),
outcome = "choice",
obsID = "obsID",
nbreaks = 10,
n_q = 6
)Now preview the results: head(power)
#> sampleSize coef est se
#> 1 30 price -0.07194700 0.07609126
#> 2 30 typeGala 0.34172607 0.22791822
#> 3 30 typeHoneycrisp 0.50663546 0.22807907
#> 4 30 freshnessAverage 0.01794571 0.22327851
#> 5 30 freshnessExcellent 0.13248210 0.22751630
#> 6 60 price 0.02152378 0.05237815tail(power)
#> sampleSize coef est se
#> 45 270 freshnessExcellent -0.17115823 0.07369265
#> 46 300 price 0.03658552 0.02240770
#> 47 300 typeGala 0.05830221 0.07098261
#> 48 300 typeHoneycrisp 0.04437270 0.07072744
#> 49 300 freshnessAverage -0.20545772 0.06966451
#> 50 300 freshnessExcellent -0.17473701 0.07007619 |
|
John -- I copied and ran your code based on your initial reply. Unfortunately, I am still getting "NA" in the Estimates and Standard Errors (See below):
Consider submitting praise at Please cite the package in your publications, see:
Any other ideas to work around or through this? This is a useful function that would like to utilize Look forward to your help. Regards -- |
|
Steve, I've replicated the code in a Google colab notebook here You can see that with a fresh installation of cbcTools (v0.2.0), the example runs without producing Can you copy-paste that exact code from the colab notebook and post the results? There must be an issue somewhere, perhaps just a simple typo? Can you confirm that you have installed v0.2.0 from CRAN? |
|
Hi John --- Yes I did a "fresh installation" of cbcTools (v0.2.0). I have uninstalled and installed about 3 times. Below is the 'output' from just the 'installed' process. Maybe you can see something there that is causing this 'NA' issue - as I am still getting that Result (see Results after the 'install' step below, as well).
trying URL 'https://cran.rstudio.com/src/contrib/cbcTools_0.2.0.tar.gz'
The downloaded source packages are in
Consider submitting praise at Please cite the package in your publications, see: Below are the colab notebook code and Results:
Any other thoughts | ideas? Regards -- Steve P. |
|
John --
I just submitted a post about this via GitHub.
Regards --
Steve P.
…On Mon, Mar 6, 2023 at 5:27 PM John Helveston ***@***.***> wrote:
Steve, I've replicated the code in a Google colab notebook here
<https://colab.research.google.com/drive/13LIuoyRexTzqsqcsNyriD03bB3VgFVsj?usp=sharing>
You can see that with a fresh installation of cbcTools (v0.2.0), the
example runs without producing NA values.
Can you copy-paste that exact code from the colab notebook and post the
results? There must be an issue somewhere, perhaps just a simple typo? Can
you confirm that you have installed v0.2.0 from CRAN?
—
Reply to this email directly, view it on GitHub
<#6 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6JFGDRC7ARK5DY35DS2KADW2ZQG5ANCNFSM6AAAAAAVQI2M5E>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
I'm at a bit of a loss here. I've ran the same code on multiple machines, never getting Can you run This shouldn't be a problem as |
|
John --
Below is the sessionInfo() Output. Maybe something there (I hope).
sessionInfo()R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United
States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] cbcTools_0.2.0
loaded via a namespace (and not attached):
[1] dplyr_1.0.1 crayon_1.3.4 grid_4.0.2 R6_2.4.1
lifecycle_1.0.0
[6] gtable_0.3.0 magrittr_1.5 scales_1.1.1 ggplot2_3.3.5
pillar_1.4.6
[11] rlang_0.4.10 rstudioapi_0.13 generics_0.0.2 vctrs_0.3.2
ellipsis_0.3.1
[16] logitr_1.0.1 tools_4.0.2 glue_1.4.1 purrr_0.3.4
munsell_0.5.0
[21] parallel_4.0.2 compiler_4.0.2 pkgconfig_2.0.3
colorspace_1.4-1 tidyselect_1.1.0
[26] tibble_3.0.3
…
On Tue, Mar 7, 2023 at 11:26 AM John Helveston ***@***.***> wrote:
I'm at a bit of a loss here. I've ran the same code on multiple machines,
never getting NA values.
Can you run sessionInfo() on your machine and copy over what is printed
out? My only guess is that you have a version issue somewhere, perhaps some
dependency that isn't updated?
This shouldn't be a problem as cbcTools should update all needed packages
when installing, but perhaps I have an error in there somewhere.
—
Reply to this email directly, view it on GitHub
<#6 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6JFGDRUF7ZZZ7KALYK6NFTW25OSFANCNFSM6AAAAAAVQI2M5E>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Okay this all looks good. The only thing slightly out of date is the R version, but it shouldn't be an issue. Maybe try updating to the latest R? I can't imagine that would change anything though. My only other suggestion at this point is to perhaps screen record yourself walking through this example and post it on youtube. Maybe I'll catch something watching it run on your machine. But really I'm stuck if I can't get the error to occur. The fact that it runs without |
|
John --
I think I may have figured it out with one of your suggestions -- an *Outdated
Package*.
I looked at the package *DEPENDENCIES *and examined to see which ones
needed to be *Updated *and then did so accordingly.
- I then copy-paste and re-ran your 'colab notebook ' code in my RStudio
-- again.
- *Voila*! *Success*!!
However -- The only '*difference*' I see is that I *do not get the same*
"Estimates" and "SE" figures as generated via the 'colab notebook ' despite
setting the same seed to '1234'.
- Anyway, I am assuming that *cbc_power* is working OK despite this '
*difference*'.
- Perhaps your R expertise has a *possible explanation* for this
'difference' despite using the *same *'*seed*'. .
- Would appreciate it if you would let me know what you think.
Thanks for your assistance on your *cbc_Tools* package.
Regards --
Steve P.
…On Tue, Mar 7, 2023 at 11:54 AM John Helveston ***@***.***> wrote:
Okay this all looks good. The only thing slightly out of date is the R
version, but it shouldn't be an issue. Maybe try updating to the latest R?
I can't imagine that would change anything though.
My only other suggestion at this point is to perhaps screen record
yourself walking through this example and post it on youtube. Maybe I'll
catch something watching it run on your machine.
But really I'm stuck if I can't get the error to occur. The fact that it
runs without NAs on the colab notebook suggests that the issue is
probably something on your machine specifically (e.g. an outdated package).
But even that looks okay based on the list I'm seeing here.
—
Reply to this email directly, view it on GitHub
<#6 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6JFGDRWOSGC7TSPKSZCTFLW25R5NANCNFSM6AAAAAAVQI2M5E>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Fantastic! Really glad we figured out the dependency issue. That said, this is still concerning as the packages should be updated upon installing the package with As for the differences, I'll take a look as well to see if I can figure out the source of the issue. Can you post your SE results here? |
|
John --
Yes -- good news!!
Below is the Output for the entire run of the code in question that
you wanted to see.
Regards --
Steve P.
> set.seed(1234)> library(cbcTools)Registered S3 methods overwritten by 'tibble':
method from
format.tbl pillar
print.tbl pillarVersion: 0.2.0
Author: John Paul Helveston (George Washington University)
Consider submitting praise athttps://github.com/jhelvy/cbcTools/issues/3.
Please cite the package in your publications, see:
citation("cbcTools")Warning messages:
1: replacing previous import ‘ellipsis::check_dots_unnamed’ by
‘rlang::check_dots_unnamed’ when loading ‘tibble’
2: replacing previous import ‘ellipsis::check_dots_used’ by
‘rlang::check_dots_used’ when loading ‘tibble’
3: replacing previous import ‘ellipsis::check_dots_empty’ by
‘rlang::check_dots_empty’ when loading ‘tibble’ > > # A simple
conjoint experiment about apples> > # Generate all possible profiles>
profiles <- cbc_profiles(+ price = c(1, 1.5, 2, 2.5, 3, 3.5, 4,
4.5, 5),+ type = c("Fuji", "Gala", "Honeycrisp"),+ freshness
= c('Poor', 'Average', 'Excellent')+ )> > # Make a randomized survey
design> design <- cbc_design(+ profiles = profiles,+ n_resp =
300, # Number of respondents+ n_alts = 3, # Number of alternatives
per question+ n_q = 6 # Number of questions per respondent+ )>
> # Simulate random choices> data <- cbc_choices(+ design =
design,+ obsID = "obsID"+ )> > > # Conduct a power analysis> power
<- cbc_power(+ data = data,+ pars = c("price", "type",
"freshness"),+ outcome = "choice",+ obsID = "obsID",+ nbreaks
= 10,+ n_q = 6+ )Estimating models using 3 cores...done!> >
head(power) sampleSize coef est se
1 30 price 0.05156809 0.06744069
2 30 typeGala 0.07141222 0.24363835
3 30 typeHoneycrisp 0.28785009 0.22567518
4 30 freshnessAverage -0.13833523 0.22695078
5 30 freshnessExcellent -0.43825846 0.23193178
6 60 price 0.06345388 0.04915863> tail(power)
sampleSize coef est se
45 270 freshnessExcellent -0.07822489 0.07474956
46 300 price 0.02910679 0.02198652
47 300 typeGala -0.04227355 0.07151511
48 300 typeHoneycrisp -0.01387631 0.07098512
49 300 freshnessAverage 0.03019946 0.07031702
50 300 freshnessExcellent -0.05680868 0.07079312>
…On Tue, Mar 7, 2023 at 2:56 PM John Helveston ***@***.***> wrote:
Fantastic! Really glad we figured out the dependency issue. That said,
this is still concerning as the packages should be updated upon installing
the package with install.packages('cbcTools'). So I'll have to take a
look at what's going on there.
As for the differences, I'll take a look as well to see if I can figure
out the source of the issue. Can you post your SE results here?
—
Reply to this email directly, view it on GitHub
<#6 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6JFGDQSNYRZFMTVMCKP773W26HIRANCNFSM6AAAAAAVQI2M5E>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
More mysteries to solve...once again tricky as I am getting the same values on my local machine as those in the colab notebook. So once again I suspect there is a very small difference between our system configurations leading to slight differences. Given how small the differences are though, I suspect this is the result of a difference in the random number generator. It may be that the same seed produces slight differences depending on the version of R you're using, or perhaps some other package. All that said, I'm not too concerned here as these results are quite similar. I do, however, need to update the package DESCRIPTION file to make sure the appropriate dependencies are in place. Can you tell / do you remember which specific packages you updated? If so, I can force a minimum version so that when users install {cbcTools} the appropriate dependencies are installed with it. |
|
John --
Yes -- perhaps the 'seed' may be generating different random numbers based
on the Version of R. Always new 'adventures' in issues when working open
source tools I guess.
Anyway -- the packages I updated are the ones I *bolded *below.
- I believe these were based on installing *cbTools *directly
within the *colab
notebook *you created.
- I then examined each one in my R environment to see which packages I
needed to update.
Regardless, it may be necessary that users ensure that *Some, if not All *of
these Dependencies are updated.
- NOTE: There may be more packages involved, as I did check for
Dependent packages with cbcTools within my own R Studio environment.
- However, I did *not *update any more packages beyond what I
highlighted below (when running the 'install' code via the *colab
notebook)*.
- I was happy that cbc_power worked after the updates I completed --
and didn't want to "push my luck" with doing anymore updates at
this time..
Please note -- I did install *logitr *the other day, as well.
- That is why I did not have to update at this time -- and possibly why
some or all of the *un*-*bolded *packages I show below did not have to
be updated, as well.
Thanks again. Let me know if you need anything else.
Regards --
Steve P.
…--------------------------------------------------------------------------------------
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
also installing the dependencies ‘*zoo*’, ‘sandwich’, ‘*httpuv*’, ‘xtable’,
‘fontawesome’, ‘*sourcetools*’, ‘*later*’, ‘*promises*’, ‘rbibutils’, ‘
*mvtnorm*’, ‘*gmm*’, ‘*Formula*’, ‘*shiny*’, ‘*Rcpp*’, ‘*Rdpack*’, ‘
*tmvtnorm*’, ‘*dfidx*’, ‘*RcppArmadillo*’, ‘*nloptr*’, ‘rngWELL’,
‘fastDummies’, ‘*idefix*’, ‘logitr’, ‘randtoolbox’
On Tue, Mar 7, 2023 at 3:51 PM John Helveston ***@***.***> wrote:
More mysteries to solve...once again tricky as I am getting the same
values on my local machine as those in the colab notebook
<https://colab.research.google.com/drive/13LIuoyRexTzqsqcsNyriD03bB3VgFVsj?usp=sharing>
.
So once again I suspect there is a very small difference between our
system configurations leading to slight differences. Given how small the
differences are though, I suspect this is the result of a difference in the
random number generator. It may be that the same seed produces slight
differences depending on the version of R you're using, or perhaps some
other package. All that said, I'm not too concerned here as these results
are quite similar.
I do, however, need to update the package DESCRIPTION
<https://github.com/jhelvy/cbcTools/blob/main/DESCRIPTION> file to make
sure the appropriate dependencies are in place. Can you tell / do you
remember which specific packages you updated? If so, I can force a minimum
version so that when users install {cbcTools} the appropriate dependencies
are installed with it.
—
Reply to this email directly, view it on GitHub
<#6 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6JFGDWBADZSD4PZYFFVECDW26NUBANCNFSM6AAAAAAVQI2M5E>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
John --
When running cbc_tools with the Example from (https://rdrr.io/github/jhelvy/cbcTools/man/cbc_power.html)
I get the following Output with "NA"s for Estimated Coefficients and Standard Errors:
Estimating models using 3 cores...
done!
I re-ran this several times -- but still get this "NA" Result. Am I missing something?
The text was updated successfully, but these errors were encountered: