Dear Quanteda Developers,
Thank you for your great package.
I found a strange behavior of remove_numbers option in CRAN version 2.0.0
from:
tokens(c("bernard hinault\n\nprofessionista su strada dal 1975 al 1986, 13"), remove_numbers = TRUE)I'm expecting
> tokens from 1 document.
> text1 :
> [1] "bernard" "hinault" "professionista" "su" "strada"
> [6] "dal" "al" ","
That's true in previous version 1.5.2, but in version 2.0.0, I'm surprisingly obtaining:
> Tokens consisting of 1 document.
> text1 :
> [1] "bernard" "hinault" "professionista" "su" "strada"
> [6] "dal" "1975" "al" "1986" ","
It seems to me that the command is not removing numbers that can be years.
I navigated through the documentation, but I don't find useful information.
Am I losing something important?
Thank you for your time,
Mario
System information
Please run sessionInfo() and paste the output.
R version 3.6.0 (2019-04-26)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] text2vec_0.6 jsonlite_1.6.1 quanteda_2.0.0 forcats_0.5.0 stringr_1.4.0
[6] dplyr_0.8.4 purrr_0.3.3 readr_1.3.1 tidyr_1.0.2 tibble_2.1.3
[11] ggplot2_3.3.0 tidyverse_1.3.0
loaded via a namespace (and not attached):
[1] Rcpp_1.0.3 lubridate_1.7.4 lattice_0.20-40 assertthat_0.2.1
[5] digest_0.6.25 packrat_0.5.0 RhpcBLASctl_0.20-17 utf8_1.1.4
[9] R6_2.4.1 cellranger_1.1.0 backports_1.1.5 reprex_0.3.0
[13] httr_1.4.1 pillar_1.4.3 rlang_0.4.5 readxl_1.3.1
[17] rstudioapi_0.11 data.table_1.12.8 Matrix_1.2-18 munsell_0.5.0
[21] rcorpora_2.0.0 broom_0.5.5 compiler_3.6.0 modelr_0.1.6
[25] pkgconfig_2.0.3 tidyselect_1.0.0 lgr_0.3.3 fansi_0.4.1
[29] crayon_1.3.4 dbplyr_1.4.2 withr_2.1.2 grid_3.6.0
[33] nlme_3.1-145 gtable_0.3.0 lifecycle_0.1.0 DBI_1.1.0
[37] magrittr_1.5 scales_1.1.0 RcppParallel_4.4.4 cli_2.0.2
[41] stringi_1.4.6 fs_1.3.2 xml2_1.2.2 stopwords_1.0
[45] generics_0.0.2 vctrs_0.2.3 fastmatch_1.1-0 tools_3.6.0
[49] float_0.2-3 glue_1.3.1 mlapi_0.1.0 hms_0.5.3
[53] parallel_3.6.0 colorspace_1.4-1 rvest_0.3.5 rsparse_0.3.3.4
[57] haven_2.2.0
Dear Quanteda Developers,
Thank you for your great package.
I found a strange behavior of remove_numbers option in CRAN version 2.0.0
from:
I'm expecting
That's true in previous version 1.5.2, but in version 2.0.0, I'm surprisingly obtaining:
It seems to me that the command is not removing numbers that can be years.
I navigated through the documentation, but I don't find useful information.
Am I losing something important?
Thank you for your time,
Mario
System information
Please run
sessionInfo()and paste the output.