trappyscopes is a python framework for building and controlling laboratory instruments. It facillitates the creation of highly heterogenous instrument assemblies by integration any existing python package. The aim of trappyscopes is to enable fast deployment of paralel measurement systems that are software defined.
The base model of this library imagines instruments as an arbitrary tree of python enabled computers and "pythonic" microcontrollers (micropython or circuitpython). With this base model, the control layer enables plug and play interfacing with minimal configuration. The key idea is to have the following workflow while building instruments.
- Connect all components with their respective interfaces (serial, ethernet, over wifi etc)
- Install this library and all required 3rd party packages for the components.
- Define the structure of the instrument in a configuraion file.
- You are done! Launch the cli to acquire data.
- Pythonic interfacing
- Software defined infrastructure is at the core of lab management.
- Made to support parallelization and open-source instrumentation.
- Seemlessly interfaces with any existing python pacakge.
Given the following configuration of instruments, where M1 and M2 are separate machines connected by a network:
graph LR
subgraph M1
M1_("Machine 1 (M1)")
M1_ -.- S("Sensor (MS1)")
end
subgraph M2
M2_("Machine 2 (M2)")
M2_ -.- S_("Sensor (MS2)")
end
M1 --mounts--> M2
# These commands work on M1
scope.MS1.read()
scope.M2.MS2.read()Given the configuration:
ScopeAssembly:
...
MS1:
description: Sensor on Machine 1.
kind: sensor1.library.Sensor1Constructor
args: []
kwargs: {}
M2:
description: Machine 2 over netwwork.
kind: hive.processorgroups.remote.RemoteGroup
args: []
kwargs:
- automount: True-
Install through
pippip install trappyscopes
-
You can install directly from the source and install the environment using the inbuilt command to build the environment. This is the recomended method because it also installs some required binary packages.
git clone -r <repo_link> cd <trappyscopes> python main.py --install
- To simply skip the binaries:
pip install . - Please check trappyscopes environment to learn more about building an environment for this utility.
-
Basic information about application startup
python main.py -h ## Or python main.py --help -
Once the installation is complete,
python main.pycan be replaced bytrappyscopes. If the package was not installed on the system (exists locally in a subdirectory), the bash script.\trappyscopescan be used. You might have to runchmod +x trappyscopesorsudo +x trappyscopesto provide executable priviledges to the script.
-
trappyscopescli is configured through aYAMLconfiguration file. Let´s start by generating a configurartion file.python main.py --new_config
- This generate the following files in the home directory:
(~)
|-trappyverse
. |- trappyconfig.yaml
.
-
trappyverse/trappyconfig.yamlis the default configuration file name and should be unchanged. We can have more than one configuration file on a system. To start the software with a custom configuration file:python main.py --config ~/parallelverse/customconfig.yaml ## Override default configuration options with this file (useful when more than one scope ## configurations need to be defined) python main.py --add_config ~/parallelverse/customconfig.yaml
-
Now let's look at the configuration file!
-
Naming the scope The first two lines are these:
name: <hostname> # Name of the scope, which defaults to hostname. The is defined as the global variable `scopeid` with the defaul startup recipie. kind: mystery-device # A signle word descriptor for the device. description: The functionally has not been described yet # A short description of the functionality of the device.
-
These fields can be edited as such and are of little consequence in terms of programming. The
namemust be chosen with care, and it's recommened that it is also the hostname of the machine. This makes remote access easy and preventss conflicts. -
name: MDevis a special name, which defines any device as a "Development Scope" and has some special priveledges. For more information, check theMDeventry in the notes. -
Scope configuration options Now let's set check some configuration options and learn what they do:
config: trappydir: ~/trappyverse # Directory where the configuration of the scope is stored. ui_mode: interactive # User interaction mode venv: # Whether to use a virtual environment for active: true # Config block is active. The function is turned on. command: source ~/opt/miniconda3/bin/activate name: trappy # Name of the virtual environment that will be called after the command git_sync: # Automatically git-sync certain repositories active: true # Config block is active. The function is turned on. command: git pull # Exact command to use for git syncronisation repos: # List of local repositories, where the command will be called - ~/lab_protocols/ - ~/lab_scripts/ set_wallpaper: false # This option will set a "information" panel as the wallpaper. This helps to id the device, incase of multiplexing. log_level: 20 # Log level of the root logger. Use 10 for debug. 20 is info and higher. config_server: # Configuration files are synchronised with this server, when changed. active: true server: <ip>/<address> share: <name-of-server-share> destination: "{date}" # Sub folder inside the share. This will create a folder with the "current date" opon operation. username: <username> password: <password> config_redact_fields: # Fields that will be redacted, if the config file is copied - username - password startup_recipie: core.startup # Startup procedure that defines how the CLI environment is created. `core.startup.__init__.py` defines the default one. startup_scripts: # Scripts to run by default when the CLI is started. - ./scripts/script1.py - ./scripts/script2.py
- Note some key features here:
- Any mapping can be turned off by defining a field
active: falseinside it. If this argument is skipped, then it's assumed to betrue. - Custom addresses (like the
destinationinconfig_server) can be defined with an "effifible" string (inspired by the f-strings in python):config_server: destination: "{date}_{scopeid}_{user}" # -> 2025_05_01_microscope1_User1
- The following terms can be used:
scopeid,user,date, andtime.
- Any mapping can be turned off by defining a field
- Note some key features here:
-
Define devices Now let's look at the default
ScopeAssemblyblock below:ScopeAssembly: <hostname>: description: "Host processorgroup." kind: hive.processorgroups.linux.LinuxMachine args: [] kwargs: {}
- The device that we see here is the host computing machine that is detected and mounted. It is one of the devices under the
ScopeAssembly, which is identified as the global variablescope. Within the scope assembly, we can define an arbitrary number of devices with the follwing schema:
ScopeAssembly: device_name: active: true description: Provide a meaningful description of the device. kind: <path.to.object.Constructor> args: [] # Arguments that are passed to the object constructor. kwargs: {} # Keyword arguments passed to the object constructor. # Optional configuration metaclass: hive.detector.Detector # Define the object as a detector and extend its functionality read_method: capture # Method of the origianl object that is interpreted as the "read" method. args: [] # These will be wrapped in a `functools.partial` instance. kwargs: {} # These will also be wrapped in a `functools.partial` instance. write_method: set # Similar to the "read_method" option. args: [] # These will be wrapped in a `functools.partial` instance. kwargs: {} # These will also be wrapped in a `functools.partial` instance.
- For more information regarding the optional configuration options, refer to: notes/devices.md.
- The device that we see here is the host computing machine that is detected and mounted. It is one of the devices under the
-
Experiment settings For now, we can leave the previous block as it was and quickly gloss over the
Experimentconfiguration block: TODO: Git auth for protocols.Experiment: exp_dir: ~/experiments # Default directory where experiments are stored protocols_dir: ~/lab_protocols # Directory where protocols are stored, This can also be a git-address. calibration_dir: ~/calibration_dir # Directory where calibrations are stored. exp_dir_structure: # This is the directory structure, that will be created within every experiment. - scripts - postprocess - converted - analysis exp_report: false # Whether the pdf report functionality is turned on or not. eid_generator: core.uid.uid # This is the funtion that will be called to generate experiment IDs. By default is calls `nanoid.generate('1234567890abcdef', 10)` file_server: # File server for synchronisation of experiments active: true server: <ip>/<address> share: <name-of-server-share> destination: "{date}" username: <username> password: <password>
-
The configuration is defined in
core.permaconfig.config.pyasTrappyConfig. It uses the Confuse library as a base.
- Start-up and usage
- Open an interactive session:
./ts -su UserName
- Scripts are an important part of running experimental procedures:
-
The scripts are executed in sequence and can be used to load pre-defined experimental protocols:
ts <script1> <script2> <script3> ts <script1> <script2> <script3> ts --iterate 3 <script1> ## Run Script1 three times
-
Alternatively, to load a script/execute a script from the interactive session:
ScriptEngine.run(globals(), scripts=["scriptfile.py"])
- Start an experiment
-
All data-collection should be done within the context of an
Experiment. It manages the data and metadata collection for an experiment:exp = Experiment("test") # OR # Constructs a better defnined name: 'MDev__YB__2025_08_23__5hh_55mm__test_experiment__6a092b60dc' exp = Experiment.Construct(["test", "experiment"])
-
Creation of an instance immediately changes the working directory to the experiment one. You should get the following output:
>>> exp = Experiment.Construct(["test", "experiment"]) { 'name': 'MDev__YB__2025_08_23__5hh_55mm__test_experiment__6a092b60dc', 'eid': '6a092b60dc', 'created': datetime.datetime(2025, 8, 23, 5, 55, 12, 771720), 'syspermastate': { 'mac_address': <MAC:ADDRESS>, 'ip_address': <IP:ADDRESS>, 'hostname': <HOSTNAME>, 'os': ['Darwin', '21.6.0'] } } ──────────────────────────────────── <Session: 6a092b60dc> ──────────────────────────────────── [05:55:13] INFO Creating new experiment: experiment.py:296 MDev__YB__2025_08_23__5hh_55mm__test_experiment__6a092b60dc ─────────────────────────────────────── Experiment open ─────────────────────────────────────── INFO Loading Experiment: experiment.py:305 MDev__YB__2025_08_23__5hh_55mm__test_experiment__6a092b60dc WARNING Experiment state not found. Not a problem if this is a new experiment. experiment.py:338 Working directory changed to: /Users/byatharth/experiments/MDev__YB__2025_08_23__5hh_55mm__test_experiment__6a092b60dc . ├── .experiment ├── analysis ├── converted ├── experiment.yaml ├── postprocess ├── scripts └── sessions.yaml 4 directories, 3 files user:YB || ‹‹MDev›› Experiment: MDev__YB__2025_08_23__5hh_55mm__test_experiment__6a092b60dc
-
The
Experimentclass manages the saving of data in specific folders and logs experiement events. -
A folder qualifies as an Experiemnt if it contains the
.experimentfile with the unique identifier of the experiment. The fileexperiment.yamlcontains the event logs of the experiments, -
Each time an experiment is open, a new session is created. An
Experimentis a composed of an arbitrary number of sessions. Thesessions.yamlcontains the environment information (list of packages, version control number, etc) for each session. -
Additionally, you can use the following utility functions:
## Find and open an existing experiment exp = findexp() ## Press enter to open a prompt with available experiments ## Find an delete an existing experiment ## Warning: this is an irreversible deletion delexp()
-
The
Experimentphilosophy is defined in the expframework submodule.