Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with subassignment operator in version 3.1.0 #868

Closed
agila5 opened this issue Mar 2, 2021 · 7 comments · Fixed by #903
Closed

Problem with subassignment operator in version 3.1.0 #868

agila5 opened this issue Mar 2, 2021 · 7 comments · Fixed by #903
Labels
bug an unexpected problem or unintended behavior
Milestone

Comments

@agila5
Copy link

agila5 commented Mar 2, 2021

Dear package developers, I want to report a problematic interaction between tibble and tmap that was reported by @Robinlovelace a few hours ago in the tmap repository. I think that the problem is not really related to tmap, but it can be summarised as follows:

# CRAN version (i.e. 3.1.0)
remove.packages("tibble")
#> Removing package from 'C:/Users/Utente/Documents/R/win-library/3.6'
#> (as 'lib' is unspecified)
remotes::install_version("tibble", "3.1.0", quiet = TRUE)

# test
library(tibble)
x <- tibble(a = "1")
x[FALSE, ] <- NA
#> Error: Assigned data `NA` must be compatible with existing data.
#> i Error occurred for column `a`.
#> x Can't convert <logical> to <character>.
x[FALSE, ] <- NA_character_
x
#> # A tibble: 1 x 1
#>   a    
#>   <chr>
#> 1 1

Created on 2021-03-02 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.3 (2020-02-29)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  Italian_Italy.1252          
#>  ctype    Italian_Italy.1252          
#>  tz       Europe/Berlin               
#>  date     2021-03-02                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
#>  bit           4.0.4   2020-08-04 [1] CRAN (R 3.6.3)
#>  bit64         4.0.5   2020-08-30 [1] CRAN (R 3.6.3)
#>  blob          1.2.1   2020-01-20 [1] CRAN (R 3.6.2)
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 3.6.3)
#>  cli           2.3.1   2021-02-23 [1] CRAN (R 3.6.3)
#>  cranlike      1.0.2   2018-11-26 [1] CRAN (R 3.6.3)
#>  crayon        1.4.1   2021-02-08 [1] CRAN (R 3.6.3)
#>  curl          4.3     2019-12-02 [1] CRAN (R 3.6.1)
#>  DBI           1.1.1   2021-01-15 [1] CRAN (R 3.6.3)
#>  debugme       1.1.0   2017-10-22 [1] CRAN (R 3.6.3)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 3.6.3)
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 3.6.3)
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 3.6.3)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 3.6.0)
#>  fansi         0.4.2   2021-01-15 [1] CRAN (R 3.6.3)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 3.6.3)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 3.6.3)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.6.0)
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 3.6.3)
#>  knitr         1.30    2020-09-22 [1] CRAN (R 3.6.3)
#>  lifecycle     1.0.0   2021-02-15 [1] CRAN (R 3.6.3)
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 3.6.3)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.6.0)
#>  parsedate     1.2.0   2019-05-08 [1] CRAN (R 3.6.3)
#>  pillar        1.5.0   2021-02-22 [1] CRAN (R 3.6.3)
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 3.6.3)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 3.6.1)
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 3.6.3)
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 3.6.2)
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 3.6.3)
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 3.6.3)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 3.6.3)
#>  R6            2.5.0   2020-10-28 [1] CRAN (R 3.6.3)
#>  rappdirs      0.3.1   2016-03-28 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.6   2021-01-15 [1] CRAN (R 3.6.3)
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 3.6.3)
#>  rlang         0.4.10  2020-12-30 [1] CRAN (R 3.6.3)
#>  rmarkdown     2.7     2021-02-19 [1] CRAN (R 3.6.3)
#>  rprojroot     2.0.2   2020-11-15 [1] CRAN (R 3.6.3)
#>  RSQLite       2.2.1   2020-09-30 [1] CRAN (R 3.6.3)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 3.6.3)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.6.0)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 3.6.3)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      3.0.0   2020-10-31 [1] CRAN (R 3.6.3)
#>  tibble      * 3.1.0   2021-02-25 [1] CRAN (R 3.6.3)
#>  usethis       2.0.0   2020-12-10 [1] CRAN (R 3.6.3)
#>  utf8          1.1.4   2018-05-24 [1] CRAN (R 3.6.0)
#>  vctrs         0.3.6   2020-12-17 [1] CRAN (R 3.6.3)
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 3.6.3)
#>  xfun          0.19    2020-10-30 [1] CRAN (R 3.6.3)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 3.6.2)
#> 
#> [1] C:/Users/Utente/Documents/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.3/library

The same code should work with version 3.0.6:

# version 3.0.6
remove.packages("tibble")
#> Removing package from 'C:/Users/Utente/Documents/R/win-library/3.6'
#> (as 'lib' is unspecified)
remotes::install_version("tibble", "3.0.6", quiet = TRUE)

# test
library(tibble)
x <- tibble(a = "1")
x[FALSE, ] <- NA
x
#> # A tibble: 1 x 1
#>   a    
#>   <chr>
#> 1 1

Created on 2021-03-02 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.3 (2020-02-29)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  Italian_Italy.1252          
#>  ctype    Italian_Italy.1252          
#>  tz       Europe/Berlin               
#>  date     2021-03-02                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
#>  bit           4.0.4   2020-08-04 [1] CRAN (R 3.6.3)
#>  bit64         4.0.5   2020-08-30 [1] CRAN (R 3.6.3)
#>  blob          1.2.1   2020-01-20 [1] CRAN (R 3.6.2)
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 3.6.3)
#>  cli           2.3.1   2021-02-23 [1] CRAN (R 3.6.3)
#>  cranlike      1.0.2   2018-11-26 [1] CRAN (R 3.6.3)
#>  crayon        1.4.1   2021-02-08 [1] CRAN (R 3.6.3)
#>  curl          4.3     2019-12-02 [1] CRAN (R 3.6.1)
#>  DBI           1.1.1   2021-01-15 [1] CRAN (R 3.6.3)
#>  debugme       1.1.0   2017-10-22 [1] CRAN (R 3.6.3)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 3.6.3)
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 3.6.3)
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 3.6.3)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 3.6.0)
#>  fansi         0.4.2   2021-01-15 [1] CRAN (R 3.6.3)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 3.6.3)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 3.6.3)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.6.0)
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 3.6.3)
#>  knitr         1.30    2020-09-22 [1] CRAN (R 3.6.3)
#>  lifecycle     1.0.0   2021-02-15 [1] CRAN (R 3.6.3)
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 3.6.3)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.6.0)
#>  parsedate     1.2.0   2019-05-08 [1] CRAN (R 3.6.3)
#>  pillar        1.5.0   2021-02-22 [1] CRAN (R 3.6.3)
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 3.6.3)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 3.6.1)
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 3.6.3)
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 3.6.2)
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 3.6.3)
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 3.6.3)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 3.6.3)
#>  R6            2.5.0   2020-10-28 [1] CRAN (R 3.6.3)
#>  rappdirs      0.3.1   2016-03-28 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.6   2021-01-15 [1] CRAN (R 3.6.3)
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 3.6.3)
#>  rlang         0.4.10  2020-12-30 [1] CRAN (R 3.6.3)
#>  rmarkdown     2.7     2021-02-19 [1] CRAN (R 3.6.3)
#>  rprojroot     2.0.2   2020-11-15 [1] CRAN (R 3.6.3)
#>  RSQLite       2.2.1   2020-09-30 [1] CRAN (R 3.6.3)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 3.6.3)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.6.0)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 3.6.3)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      3.0.0   2020-10-31 [1] CRAN (R 3.6.3)
#>  tibble      * 3.0.6   2021-01-29 [1] CRAN (R 3.6.3)
#>  usethis       2.0.0   2020-12-10 [1] CRAN (R 3.6.3)
#>  utf8          1.1.4   2018-05-24 [1] CRAN (R 3.6.0)
#>  vctrs         0.3.6   2020-12-17 [1] CRAN (R 3.6.3)
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 3.6.3)
#>  xfun          0.19    2020-10-30 [1] CRAN (R 3.6.3)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 3.6.2)
#> 
#> [1] C:/Users/Utente/Documents/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.3/library

More details about the original issue: https://github.com/mtennekes/tmap/issues/551

EDIT:

The same code works with regular data.frame:

x <- data.frame(a = "1")
x[FALSE, ] <- NA
x
#>   a
#> 1 1

Created on 2021-03-02 by the reprex package (v0.3.0)

@Robinlovelace
Copy link

I have just responded to an email from CRAN saying my pct package will be removed on 18th March (in 3 days) unless fallout from this issue was resolved. I replied as follows:

I have checked the tests: https://cran.r-project.org/web/checks/check_results_pct.html
This is the result of a known issue in the latest release of tibble and its interaction the tmap:

Is it OK to wait for these issues to be fixed upstream?
All the best,
Robin

In case anyone else is seeing CRAN checks fail because of this issue, it may be helpful to get an update on this, e.g. 'wip' or 'wontfix'. Heads up @krlmlr and other tibble developers. Thanks.

@krlmlr
Copy link
Member

krlmlr commented Mar 18, 2021

Thanks. Is this a duplicate of #773?

Please work around for now.

@agila5
Copy link
Author

agila5 commented Mar 18, 2021

Thanks. Is this a duplicate of #773?

I think it is more or less the same problem. Moreover, I don't know exactly the rules behind subset-assignment for tibble(s), but I think that the behaviour is not always consistent. For example:

# packages
library(tibble)

# data
x <- tibble(
  a = TRUE, 
  b = 1L, 
  c = sqrt(2), 
  d = 3i + 1, 
  e = "e", 
  f = raw(1), 
)
x
#> # A tibble: 1 x 6
#>   a         b     c d     e     f    
#>   <lgl> <int> <dbl> <cpl> <chr> <raw>
#> 1 TRUE      1  1.41 1+3i  e     00

# subset-assign 0 row
x[FALSE, "a"] <- NA
x[FALSE, "b"] <- NA
x[FALSE, "c"] <- NA
x[FALSE, "d"] <- NA
x[FALSE, "e"] <- NA
#> Error: Assigned data `NA` must be compatible with existing data.
#> i Error occurred for column `e`.
#> x Can't convert <logical> to <character>.
x[FALSE, "f"] <- NA
#> Error: Assigned data `NA` must be compatible with existing data.
#> i Error occurred for column `f`.
#> x Can't convert <logical> to <raw>.
x
#> # A tibble: 1 x 6
#>   a         b     c d     e     f    
#>   <lgl> <int> <dbl> <cpl> <chr> <raw>
#> 1 TRUE      1  1.41 1+3i  e     00

while, when I consider subset-assignment with 1 row,

x[1, "a"] <- NA
x[1, "b"] <- NA
x[1, "c"] <- NA
x[1, "d"] <- NA
x[1, "e"] <- NA
x[1, "f"] <- NA
x 
#> # A tibble: 1 x 6
#>   a         b     c d     e     f    
#>   <lgl> <int> <dbl> <cpl> <chr> <raw>
#> 1 NA       NA    NA NA    <NA>  00

Created on 2021-03-18 by the reprex package (v1.0.0)

@krlmlr krlmlr added the bug an unexpected problem or unintended behavior label Apr 16, 2021
@krlmlr krlmlr modified the milestones: 3.1.1, 3.1.2 Apr 16, 2021
@krlmlr krlmlr modified the milestones: 3.1.2, 3.1.3 Jul 17, 2021
@krlmlr
Copy link
Member

krlmlr commented Jul 17, 2021

It's a different issue, #773 is about assigning a value to an all-NA vector.

@krlmlr
Copy link
Member

krlmlr commented Jul 21, 2021

I have a PR ready that fixes this and the sister issue #773. Before actually merging, I'd like to rethink if we really want this, or if we'd rather prefer consistent failure in all cases.

Requiring typed NA adds a bit of safety for programming. I imagine this will be used mostly when populating/tweaking individual cells in an existing data frame. If we fail consistently, this means that the onus is on the programmer to double-check and ensure that the NA value they stuffed into this column is actually intended, and not just a programming error.

What do you think?

CC @lionel-.

@krlmlr
Copy link
Member

krlmlr commented Jul 21, 2021

On the other hand, in #773 (comment), @hadley suggests:

In general, this is a deliberate change, so that you don't accidentally change the type of your columns. However, this specific case seems to be a bug, since generally we treat vectors consisting only of missing values as compatible with any other vector.

Following that, this means that we might not need this extra safety, but also raises the following questions that should be resolved in vctrs:

  • Are NA_integer_ and "x" compatible?
  • Is tibble(a = integer(), b = logical()) compatible to tibble(a = 3, b = "x") ?
  • Is tibble(a = 1, b = NA) compatible to tibble(a = 3, b = "x") ?

The answer might be "No", "Yes" and "Yes", with severe implications, but again it should be decided elsewhere.

krlmlr added a commit that referenced this issue Jul 21, 2021
- `tbl[row, col] <- rhs` treats an all-`NA` logical vector as a missing value both for existing data (#773) and for the right-hand side value (#868). This means that a column initialized with `NA` (of type `logical`) will change its type when a row is updated to a value of a different type.
@github-actions
Copy link
Contributor

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue and link to this old issue if necessary.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug an unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants