Skip to content

test_that() fails in interactive mode when there are several failures within the same expectation. #279

Closed

Description

@antoine-lizee
> test_that("everything goes normal", {
+     expect_equal(as.character(5), 5)
+ })
Error in vapply(failures, as.character, character(1)) : 
  values must be length 1,
 but FUN(X[[1]]) result is length 2

> get_reporter()$failures[[1]]
Not expected: as.character(5) not equal to 5
Modes: numeric, character. Not expected: as.character(5) not equal to 5
target is numeric, current is character. 

From what I tracked down, there are several problems:

  1. the "worst" one is that failures don't get cleaned up in the reporter if the function end_test() throws an error, which prevents further use of the package. A quick fix would be to flush it first.
    Then, either:
  2. format.expectation (called by as.character) should collapse the several failure_msg of the expectation.
  3. Only one failure_msg should be allowed.

Activity

  1. changed the title [-]test_that() fails in interactive mode when there is several failures within the same expectation.[/-] [+]test_that() fails in interactive mode when there are several failures within the same expectation.[/+] on Jul 15, 2015
  2. antoine-lizee commented on Jul 15, 2015

    @antoine-lizee
    Author

    This first PR solves 1. and 2. in a pretty stupid way. I still don't know about 3. Are failures supposed to have several messages?

  3. added a commit that references this issue on Jul 16, 2015
    187fb22
  4. antoine-lizee commented on Jul 16, 2015

    @antoine-lizee
    Author

    This PR is the right fix. The problem was coming from compare.numeric that was behaving differently than compare.default (the fallback of compare.character).

    My diagnostic repex:

    str(testthat:::compare.default("35", 4)) # compare.character() -> NextMethod() -> compare.default()
    str(testthat:::compare.numeric(35, "4")) # 2 MESSAGES !
    

    Should I add a better failover for the end_test() method of the reporter? This still could be a problem later on. (point (1.) above) I could keep something along the lines of my first PR or have something like a

    tryCatch({...}, 
             finally = {failures <<- list()}
    )
    

    instead. Or just let it be :-).

  5. antoine-lizee commented on Sep 24, 2015

    @antoine-lizee
    Author

    Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions