`forecast` and `scores` objects print output every time a column is changed

[nikosbosse]

Consider this example:

a <- data.table(x = 1:30, y = 2:31)
a[, model := "a"]

ex <- data.table::copy(example_quantile)
ex[, model := paste(model, "a")]

Upon changing the model column, the first data.table prints nothing. For ex, however, the whole data.table is printed again.

---

EDIT: This is no longer true
Same happens with scores objects

ex <- score(example_quantile)
ex[, test := 3]

---

@Bisaloo do you have any thoughts on this?

[nikosbosse]

@Bisaloo do you think you might have a minute to look at this, or are you swamped at the moment? In that case I can try and take over. Thank you!

For context: we have to update scoringutils on CRAN until October 31, 2024. The reason for that is that the package author fields are currently malformed in the CRAN version (they are fixed on gh) and we got a few messages asking us to fix this by October 31 latest. Ideally, it would be nice to have this fixed before we release anything to CRAN.

see #920

[nikosbosse]

@seabbs tagging you for context on the deadline

[seabbs]

For context: we have to update scoringutils on CRAN until October 31, 2024. The reason for that is that the package author fields are currently malformed in the CRAN version (they are fixed on gh) and we got a few messages asking us to fix this by October 31 latest. Ideally, it would be nice to have this fixed before we release anything to CRAN.

Ah I see. So we could just update the CRAN version from the release that we made it with (i.e update that branch and release from there). That takes the pressure off the very breaking dev version but also deadlines are nice

[nikosbosse]

Exactly. But I think we could actually make the October 31 deadline. Only 3 more issues to got!
https://github.com/epiforecasts/scoringutils/issues?q=is:issue+is:open+-milestone:scoringutils-2.x

Edit: SHOULD!

[Bisaloo]

I can have a look after Wednesday. I don't have any intuition of what may be causing this yet 🤔

[nikosbosse]

Merci!

[Bisaloo]

From what I understand, data.table has a complex internal state to control whether or not an object should be printed. The problem is that this state is stored in the .global internal variable (Rdatatable/data.table@342d6ba) which we cannot touch or even inspect from scoringutils 🤔

scoringutils could potentially add its own internal variable to determine if printing should proceed (i.e., forwarded to print.data.table()) or cancelled but the logic in data.table at least is quite convoluted.

[nikosbosse]

I see. Hm...
@MichaelChirico might you have an idea for this?

[MichaelChirico]

Sorry, I'm just glancing & not able to glean the full context. Do you have something that's printing, that shouldn't be, or something that's not printing, that should be?

[nikosbosse]

Things are printing that shouldn't be printing.
This works fine:

a <- data.table(x = 1:30, y = 2:31)
a[, model := "a"]

This prints although it shouldn't:

ex <- data.table::copy(example_quantile)
ex[, model := paste(model, "a")]

Same happens with scores objects

ex <- score(example_quantile)
ex[, test := 3]

This is probably related to the fact that we have specific [.forecast and [.scores methods that revalidate objects when the user manipulates them.

[nikosbosse]

In the data.table code you linked @Bisaloo I see this comment in the code

This lets me think that the issue may be related to the fact that we're hitting [.forecast twice. If I run this code:

library(scoringutils)
debug(scoringutils:::`[.forecast`)
ex <- data.table::copy(example_quantile)
ex[, model := paste(model, "a")]

I'm landing at [.forecast twice. This does not seem to be due to the content of assert_forecast(). Replacing

validation <- try(
      assert_forecast(forecast = out, verbose = FALSE),,
      silent = TRUE
    )

with

validation <- try(
      TRUE,
      silent = TRUE
    )

in [.forecast doesn't change the fact that we're hitting [.forecast twice...