Counterfactual Visibility

Matan Mazor 🤓, Rani Moran 🥸 & Clare Press 😎

A series of experiments looking at the effects of factual and counterfactual visibility on detection. The experiments in this repository are a subset of the detection in context experiments.

Data

Raw data from all four experiments is available on the project's OSF repository: osf.io/7v2d6/

Analysis Scripts

A fully reproducible data-to-paper code (in R and Rmarkdown), as well as reports of all pre-registered analyses, are available in the 'docs' subdirectory. Scripts currently use local paths for data files. To read data directly from OSF, replace mentions of loadAndPreprocessData with loadAndPreprocessDataFromOSF.

Experiment demos

You can try Experiment 1 (occluded pixels) by clicking here

You can try Experiment 2 (occluded rows) by clicking here

You can try Experiment 3 (occluded rows + reference) by clicking here

Pre-registration and pre-registration time-locking 🕝🔒

OSF pre-registrations are available for Exp. 1 (Exp. 3 in the Detection in Context OSF project), Exp. 2 (Exp. 4 in the Detection in Context OSF project) and Exp. 3 (Exp. 6 in the Detection in Context OSF project).

To ensure preregistration time-locking (in other words, that preregistration preceded data collection), we employed randomization-based preregistration. We used the SHA256 cryptographic hash function to translate our preregistered protocol folder (including the pre-registration document) to a string of 256 bits. These bits were then combined with the unique identifiers of single subjects, and the resulting string was used as seed for initializing the Mersenne Twister pseudorandom number generator prior to determining all random aspects of the experiment, including the order of trials, occluder positions, and random noise in the stimulus itself. This way, experimental randomization was causally dependent on, and therefore could not have been determined prior to, the specific contents of our preregistration document (Mazor, Mazor & Mukamel, 2019).

Exp. 1

protocol folder

protocol sum: e420455976659d9a46582ea0f7a64ba9e33810d90786c5157e2a188e8dcdd7c0

relevant pre-registration lines of code

Exp. 2

protocol folder

protocol sum: bf72004d226b7a89a2085b0d6238a8d9b9c638513127a47fd44c6a7d00112b2f

relevant pre-registration lines of code

⚠️ Notice that the long version of Exp. 2 was not pre-registered.

Exp. 3

protocol folder

protocol sum: 2be4e2548db0a221a06c936fbba47cecd28894e0400477ac4f580222b77a4a44

relevant pre-registration lines of code

⚠️ Notice that the long version of Exp. 3 was not pre-registered.

Since randomization in these experiments depends on participants' behaviour (the longer they take to respond, the more random luminance values are sampled, affecting all later trials), setting the seed to the same value is not sufficient to reproduce the same randomization: it is also necessary to reproduce participants' response times. For pre-registration verification purposes, we include "passive" demos (Exp. 1; Exp. 2; Exp. 3), where specifying a subject identifier initializes the PRNG and reproduces their responses and response times. The resulting randomization should correspond to the randomization observed in the data files. Because of this dependency on RT data, these demos only work for subject identifiers that exist in the data files.