tidy.STM with content variable in stm

[jonathanvoelkle]

I realized that when then document-level content covariate is set in stm, the beta-list contains also the interaction coefficients besides the topic-term probabilities and provides a beta-matrix for each covariate separately, in which case tidy.STM returns a tibble where some topics/terms are NAs, and multiple values for topic-term combinations (exactly the number of levels in the covariate) - reprex below.

So I restricted it to only return the beta-values and added a column to the beta tibble which indicates the covariate.

library(stm)
#> stm v1.3.6 successfully loaded. See ?stm for help. 
#>  Papers, resources, and other materials at structuraltopicmodel.com
library(tidytext)
topic_model_content <- stm(poliblog5k.docs, poliblog5k.voc, K = 20,
            prevalence = ~ rating + s(day), content = ~ rating,
            max.em.its = 3, data = poliblog5k.meta, init.type = "Spectral",
            verbose = F)
td_beta_content <- tidy(topic_model_content)
td_beta_content
#> # A tibble: 271,097 × 3
#>    topic term      beta
#>    <int> <chr>    <dbl>
#>  1    NA <NA>  0.000133
#>  2    NA <NA>  0.000292
#>  3    NA <NA>  0.000269
#>  4    NA <NA>  0.000536
#>  5    NA <NA>  0.000376
#>  6    NA <NA>  0.000109
#>  7    NA <NA>  0.000294
#>  8    NA <NA>  0.000101
#>  9    NA <NA>  0.000130
#> 10    NA <NA>  0.000246
#> # … with 271,087 more rows

^{Created on 2022-04-20 by the reprex package (v2.0.1)}

[juliasilge]

Nice! I have not used that content argument before. We should try to use a column name in the output that aligns with some existing tidy() output. Maybe y.level?

Would you be up for adding a test in this file, probably using gadarian since we already use that in tests?

[juliasilge]

Thank you so much @jonathanvoelkle! 🙌

[github-actions]

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.