Offer a better overview

[maelle]

I would like to ask if there is a detailed explanation of the types of diagnostics provided by autotest

in ropensci/software-review#596 (comment)

Maybe https://docs.ropensci.org/autotest/reference/autotest_types.html could be run in a vignette, at the very least.

[gilbertocamara]

Dear @maelle, I am afraid the output of autotest_types() is not clear enough. What is provided is a one-sentence explanation. From a software developer's point of view, this is not enough. It would be good if the decisions taken by the autotest designer would be justified. Let's take some examples:

(a) single_char_case: autotest_types() states that changing case in a character parameter should not alter the result of a function. Why? One can see many cases where uppercase and lowercase parameters should be treated locally.

(b) return_successful: autotest_types() states Check that function successfully returns an object. What does this mean? What if there are errors in the parameters and they are caught by the software?

(c) rect_as_other, rect_compare_col_names, rect_compare_col_structure: autotest_types() states: Convert one rectangular class to another. For example, it tests whether inputs of classes tibble, data.frame, data.table and list produce the same result. That is a reasonable assumption in most R packages, where input parameters are not specialized. However, in the sits package all of the most important function parameters are typed. These key parameters are tested for adhrence to the specific types required by each function.

My general feeling is that autotest is a great package, which is most relevant to R programmers who are not cautious about types and input checking. We have designed the sits package trying to use R as a strongly-typed language. We check every input parameters. Thus, we have a feeling there is a mismatch between our programming style and what is the expectations for autotest.

[mpadge]

@gilbertocamara Thanks for these concerns, which will definitely be taken on board in the upcoming major iteration of autotest. You are definitely right that the primary aim of autotest is to highlight implicit assumptions in parameter types for package authors who do not implement strict controls, or who rely on generic assumptions. Your sits package will provide an important test case that strongly contrasts this primary assumption. I'll keep this issue open, and document testing of sits once the next iteration commences (likely sometime after October 2023, depenedent on funding).

In regard to you specific comments, note that one issue which will be addressed in the upcoming development is easier user control over tests, to enable for example you as the author to easily "switch off" tests for changing case in particular character parameters. The various rect_ conversion tests have already been updated in an auxilliary package which will provide the main autotest engine in the updated version.

Thanks, Mark