Accessing stimulus provenance for P3 sessions

[rcpeene]

We've had some questions about how to find the stimulus code used on a given session. I will focus on our ecephys sessions to start but may add more about SLAP2 and MESO in the future.

Ecephys

The overall (simplified) data flow of stimulus information through our ecephys processing pipeline is as follows.
bonsai script -> running the session on the rig -> bonsai log -> stimulus pkl -> stim_epochs.csv -> NWB.intervals (final stim table on DANDI)

The stimulus is designed and executed using Bonsai. The bonsai script produces a log of the time and content of all the stimulus frames, as well as the timing and value of each sample of the running wheel encoder. However, we have different setups to run this on different rigs depending on the data modality. For ecephys, to be compatible with our historical stimulus setup that uses Camstim and related synchronization software, we converted the output from Bonsai into a stimulus pickle file. The stimulus pickle is designed to be the canonical output product of the stimulus after running a session. It includes a record of what stimulus instructions were provided, as well as what actual stimulus frames were produced and when. It also includes the raw running wheel encoder frames and additional metadata about the session.

Recall that the unpackaged session data is public on S3. I tend to use quilt as my viewer for S3 files but you can use many services to access S3 data. (This is explained in documentation elsewhere). For an example, we can see the stimulus information for one session here https://app.quilt.bio/b/aind-open-data/tree/ecephys_830794_2026-01-28_11-01-44/behavior/

Within the behavior folder, we can see the 1488757805_830794_20260128.behavior.pkl, the immediate product, hot off of the rig, describing the stimulus. Importantly, the pickle file is not synchronized to the ecephys recording. It stands alone as a record of the stimulus and running wheel encoder. We must do a complicated process to synchronize the information in the pkl the produce a stimulus table with timestamps that are referenced to the same clock as the ecephys recording. Synchronization is its own vast topic unto itself, so I will not discuss it much here, but know that its output is the file 1488757805_830794_20260128_stim_epochs.csv in our example session. This is the human-readable table that gets directly packaged into the final NWB product for each session. (You may notice that there is also an opto.pkl and opto_table.csv. The optotagging stimulation has different hardware associated with it, and thus produces different outputs. The opto table gets combined with the regular stimulus table during NWB packaging)

Now, opening and investigating .pkl files can be a bit of a harrowing task (A good use case for our AI agents), but I will just tell you that the provenance information can be found at these paths

root["script"]
root["params"]["session_type"]
root["params"]["bonsai_path"]
root["params"]["repository_commit_hash"]
root["params"]["repository_url"]

Which in this case, yield

script:
C:/BonsaiDataPredictiveProcessingRepo\openscope-community-predictive-processing\code/stimulus-control/src/Mindscope/generic_oddball.bonsai

session_type:
sequence_mismatch_long_zebra

bonsai_path:
code/stimulus-control/src/Mindscope/generic_oddball.bonsai

repository_commit_hash:
313d5d0622c3c82d025a84a395af6fde9f5aa81f

repository_url:
https://github.com/AllenNeuralDynamics/openscope-community-predictive-processing.git

commit_url:
https://github.com/AllenNeuralDynamics/openscope-community-predictive-processing/commit/313d5d0622c3c82d025a84a395af6fde9f5aa81f

From this we can see the exact repository, commit hash, and filepath for our session, and that it was a sequence mismatch session. Plugging this information into github will get you the link to this bonsai script
Of course, to read, reproduce, or tinker with this bonsai script you will have to have Bonsai installed yourself.