Autonomous research on physical instruments: what should the experiment be allowed to change?

[Cherish0925]

Hi everyone, I'm Shuxing. Autoresearch's editable experiment and explicit evaluation make me wonder what a similarly inspectable research process should look like when the experiment uses physical apparatus.

I've been developing AI4Instruments, a proposal based on my work with AI systems in real optical experiments. The idea is to let AI participate in instrument design and development as well as experimental use: changing acquisition software or firmware, exploring available controls, and helping develop new apparatus when a research question requires it. The repository currently has a paper, technical notes, and a small optical simulation.

A physical experiment makes some choices particularly interesting. Restoring an earlier program may leave the apparatus in a different physical state. Changing an acquisition method can change the meaning of the score. A useful research record therefore needs to connect the code revision, apparatus configuration, actions, observations, and reason for the next experiment.

I'd like to hear from anyone adapting autonomous research to hardware. How do you decide whether an apparent improvement comes from the proposed change, a changed measurement procedure, or drift in the apparatus? And what would be a good small experiment in which an agent can improve the experimental method itself, rather than only tune a fixed list of parameters?

This is an early design proposal with its own simulation; an autoresearch integration would be future work. Concrete examples, existing projects, and objections would help me choose a useful next implementation. The paper explains the wider direction.

Thanks,
Shuxing