Wrong inputs column

[FabrizioSandri]

Description

I noticed that the analysis's result table contains for each test case of a specific function, the same input values.

How to reproduce

This problem can be reproduced by simply running RcppDeepState on the testSAN package and, by looking at the result's inputs column, you can notice that the inputs are the same. In the following snippet, you can observe the inputs saved in the resulting table for the rcpp_read_out_of_bound function of the testSAN package.

> result$inputs
[[1]]
[[1]]$rbound
[1] -717690435


[[2]]
[[2]]$rbound
[1] -717690435


[[3]]
[[3]]$rbound
[1] -717690435
...

Possible solution

By looking at the rcpp_read_out_of_bound_log_text log file, it is possible to confirm that the inputs are, in fact, successfully given to the test harness. Thus the inputs saved by the qs library appear to be the cause of this issue.

INFO: Starting fuzzing
input starts
rbound values: -1235514978
input ends
input starts
rbound values: 1545031884
input ends
input starts
rbound values: -635447766
input ends
...

The issue is that each time a new input is generated, the existing ./inputs/rbound.qs file is overwritten since the inputs are saved using the qs::c qsave function when the generator is invoked.
The section that saves the inputs has to be placed inside the runner in order to fix this issue. This causes the .qs file to be overwritten each time a new test case is run with valgrind, but this is not a problem because RcppDeepState reads the input file before starting the subsequent test.