Studio Reproducibility

Reproducibility: Code, Bundles and Citation

Everything you build in Studio is also a program. This page covers the generated Python you can download for the questionnaire and each flow, the research bundle — a zip of one Save that runs without Studio — what it contains, how to run it, what it reproduces exactly and what it does not, and how to cite a study.

Download .py

Every Save stores the Python the engine generated for each document at that Save. A download gives you exactly that file — the one Studio validated, published and executed — not a fresh re-generation.

Where	What you get
Builder → More ▾ → Export Python	`questionnaire.py` of the current Save (disabled until the project has a Save)
Distribute → Publish panel → questionnaire.py	the same file
Flow editor → More ▾ → Export Python	`<flow>.py` of the current Save (disabled for a new or edited flow: "Save first")
History list → download icon on a row	`questionnaire.py` of that Save (disabled for a Save with errors)
History → a Save → More ▾ → questionnaire.py only	`questionnaire.py` of that Save
History → a Save → Documents tab → .py	the generated file of any questionnaire or flow document of that Save (and .json for the document itself)

A document that did not pass the engine check at that Save has no generated code: the download fails with "No generated code for this document."

questionnaire.py

A self-contained program that imports only siamang, the engine. It passes siamang validate, and siamang preview questionnaire.py opens the questionnaire in your browser — see the library's CLI Reference. It needs a version of the engine that includes Studio's changes; see Installing the engine.

A flow script

A readable script, in the order the flow runs:

"""Satisfaction by region (flow: satisfaction_by_region).

Generated by Siamang Studio from the Flows document (scripts/satisfaction_by_region.py,
schema 1.0) — the exact code Studio ran to produce the report. Edit the flow in
Studio and download again. Run locally from a research bundle:

    python scripts/satisfaction_by_region.py --data data/responses.parquet
"""
import argparse
…
from survey.questionnaire import survey

# ── Data source ──────────────────────────────────────────────────────────
parser = argparse.ArgumentParser(…)
parser.add_argument("--data", help="Local snapshot (Parquet/CSV/SAV) instead of the project database")
args = parser.parse_args()

# ── Responses: responses · main ──────────────────────────────────────────
# studio: src
if args.data:  # research bundle: reproduce from a data snapshot
    …
# ── Crosstab: satisfaction × region ──────────────────────────────────────
# studio: xtab
n_xtab_table = n_apply.report.crosstab("satisfaction", "region", pct="col", test=True)
n_xtab_stat = n_xtab_table.stats
…
# ── Save report: outputs/satisfaction_by_region.md ───────────────────────
# studio: save
n_save = Report.combine([n_section], title="Satisfaction 2026-Q3", toc=False, theme=None)
n_save.provenance(os.environ.get("SIAMANG_PROVENANCE"))
n_save.save("outputs/satisfaction_by_region.md")
n_save.save(Path("outputs/satisfaction_by_region.md").with_suffix(".html"))

Each node is a commented block — its title and summary, then # studio: <node id> — and its results are named after the node id (n_xtab_table). The Save report block writes the look of its Look parameter (theme=None when the node has none: the engine's defaults) and adds the provenance footer from the SIAMANG_PROVENANCE environment variable. The platform sets that variable on every run, and a research bundle's run.sh sets it from PROVENANCE.md — both only while Settings → Reports → End every report with the provenance footer is on. Run by hand without it, the report has no footer.

A flow script is not standalone:

it imports the questionnaire from survey/questionnaire.py, so run it from the root of a research bundle (the bundle's run.sh sets PYTHONPATH=.);
outside Studio it must be given the data: --data <file> (Parquet, CSV or another text table — .tsv, .txt — Excel .xlsx, .xlsm or .xls, SPSS .sav or Stata .dta). The file is read with the questionnaire's codebook, as the platform reads the responses, whatever part of the survey it holds (a pilot's data has only the columns of the questions reached). Without --data it tries to read the project database, which exists only inside Studio. A flow with two or more database sources takes one option per source instead — --data-<node-id>, with _ in the id written as - (for example --data-src-wave1);
Write table nodes are skipped when the script is given a data file (--data, or any --data-<node-id>).

The research bundle

History → open a Save → More ▾ → under Download a bundle:

Option	Contents
Code & documents	"questionnaire.py, JSON documents, flows, README" — everything except the answers
With the responses so far	"plus data/: the responses (latest), the tables and uploads the flows read" — the same, plus the data every flow reads and its manifest
questionnaire.py only	just the generated questionnaire

The zip is built when you click and named <project-slug>-s<N>.zip, or <project-slug>-s<N>-data.zip with the responses ("Bundle of Save #17 downloaded with the responses so far"). The data option takes at most 100,000 rows from each table it reads; a larger table is refused with "This table exceeds the synchronous export limit of 100,000 rows. No file was generated; use a database export for the complete dataset." Downloads are recorded in the organization's audit log.

What is inside

The files sit at the root of the zip — there is no enclosing folder, so unzip into a folder of your own.

README.md                                  what is inside, how to run it, what each flow reads, notes
PROVENANCE.md                              what everything was made from
siamang.yaml                               the project manifest the engine's CLI reads (flows in Run all order)
survey/__init__.py
survey/questionnaire.py                    the generated questionnaire
survey/questionnaire.json                  the questionnaire document
survey/codebook.md                         one row per variable: label, scale, values, missing, range, used by
analysis/<name>.codeframe.json             any codeframes (see Coding Open Answers)
flows/<name>.flow.json                     the flow documents
scripts/<name>.py                          their generated scripts
studio/settings.json                       environments, runtime, connectors, report and study settings
data/responses.csv                         (with responses) the answers, as a flow sees them
data/responses.<env>[.completed].csv       (with responses) the same, filtered as a Responses node filters
data/tables/<table>.csv                    (with responses) the project tables the flows read
data/tables/<table>.dictionary.json        (with responses) the variables stored with such a table
data/responses.manifest.json               (with responses) when taken, rows, sha256 of each file, environments, Save
assets/<name>                              (with responses) the uploads under Files that the flows read
environment/requirements.txt               the engine pin plus your runtime packages
environment/python-version                 the Python version Settings → Runtime asks for
environment/run.sh                         install, validate, run every flow on its data
environment/engine/<wheel>.whl             (when provided) Studio's build of the engine
environment/engine/SHA256SUMS              (when provided) its checksum
report/theme.json                          the project's report house style ({} when none)
METHODS.md                                 a draft Methods section
CITATION.cff                               how to cite this Save

Details:

data/responses.csv is the project's whole responses table — every environment, partial interviews included — as a flow sees it: one column per variable, with answers collected by an earlier survey runtime read into today's layout (an "Other (please specify)" answer as the question's Other code plus a <variable>_other text column, matrix answers as their column codes — see Responses). It carries the fieldwork columns a flow may read: duration_s, started_at, captcha, tab_switches, hidden_seconds and pastes. The survey link's parameters (url_*: panel ids and the like) are left out, except those a flow of the Save names; the invitation token (url_inv) never travels. The README says which were kept.
One file per filter. For each Responses node whose Environment or Only completed responses leaves out some rows, the bundle adds the rows it keeps, named after the filter: data/responses.main.csv, data/responses.main.completed.csv. When the filter keeps every row, the node reads data/responses.csv.
Tables. Each project table a flow reads (a Project table node, or a Responses node naming another table) is data/tables/<table>.csv, as it was in the project database when the bundle was built, with the variables its Write table stored beside it in data/tables/<table>.dictionary.json. A table no flow has written yet is not included. The same rule for link parameters applies to these tables: a url_* column a table carries (a cleaning flow writes the columns it read) travels only when a flow names it, and url_inv never does — in the table's CSV and in its dictionary alike.
Uploads. The files uploaded under Files that a flow names (assets/<name>, the File of a Data file node) are included at that path.
The manifest records taken_at, rows, sha256, file, label (latest), environments, snapshot and project, plus files (each extra data file with its table, rows, sha256 and filter) and uploads (file, bytes, sha256).
A document that did not pass the engine check at that Save has no generated file; the README's Notes say which (for example "scripts/tables.py is missing: the flow did not pass the engine check at this Save."). A flow saved earlier with a data source that names no table or environment a project can have is not run by run.sh either: "scripts/.py is not run by environment/run.sh: its data source names no table or environment a project can have (…)." Today such a flow fails the engine check at Save (see Responses), so it has no script. Likewise a flow saved before step texts had to be one line, with a line break in one: "scripts/.py is left out: a step's text holds a line break, which its generated script would run as code — fix it in the flow and save again." The same holds for the questionnaire: when a variable name, question Id or page name holds a line break (the Builder's own fields never produce one), the bundle leaves out the generated questionnaire with "survey/questionnaire.py is left out: a variable, question or page name holds a line break, which the code generated at this Save may run as code — rename it in the Builder and save again." Without that file run.sh stops at its siamang validate step. The README also flags a flow with several sources and a Write table whose script was generated by an older engine ("… read several sources and also write a table; the engine's script for such a flow stops at the write step (args.data). A Save in Studio regenerates the script with the current engine, which runs it.") — save it again before you download the bundle.
environment/python-version is the version Settings → Runtime → Python version asks for (3.11 unless you chose another; the field's hint reads "for the research bundle — Studio runs flows on Python 3.11"). Studio itself runs every flow on Python 3.11 — the README says both ("Python: Studio ran the flows on 3.11; environment/python-version asks for 3.12 (Settings → Runtime)"), and choosing another version gives the Save warning "Studio runs every flow on Python 3.11. Python 3.12 (Settings → Runtime) is the version a research bundle asks for (environment/python-version), not the one the flows run on here."

Installing the engine

requirements.txt pins the engine in one of three ways, and the README says which:

Shipped with the bundle. When Studio provides its build, the bundle contains it under environment/engine/ (checksum in SHA256SUMS) and requirements.txt installs it from there: "The engine is shipped in the bundle … and requirements.txt installs it from there."
From GitHub, at Studio's commit. requirements.txt reads siamang[charts,parquet] @ git+https://github.com/Siamang-Labs/siamang@<commit> at the commit that matches the engine Studio runs. It needs git and network access.
From GitHub, at the last merged commit. The same line at 4050de7, the Siamang-Labs/siamang commit that has Studio's engine changes as far as upstream has merged them. Studio runs a newer engine than that, and the README says what differs at that commit: "a Simulated data source, a Share of preference, t-test, Correlation matrix, Paired tests, Factor analysis, Descriptive statistics, Data check, MaxDiff scores, Bands, Key drivers, Perceptual map, Price sensitivity, Likert chart or Result chart node, every Export file, Code open answers and TURF node, a Bar chart with Show percent, Split by or Sort value, a Heatmap with Pearson or Kendall, Regression's ordinal model, Save report's tables in Excel, and a test chosen by hand (Correlation with Pearson or Kendall, Group means with a Test other than auto, Compare groups with Dunn's test, Crosstab with Fisher's exact test) stop with an error, and after Apply weight the MaxDiff and conjoint estimates, bar charts, heatmaps of means, principal components and scale reliability are computed unweighted; siamang preview shows that commit's survey runtime, not Studio's." If a script stops or its numbers differ from Studio's, install a later siamang revision that has those changes (the commit after @ in environment/requirements.txt).

The PyPI release siamang 0.6.0 (and anything older) is not enough: it predates Studio's engine changes and cannot run these scripts, although the engine Studio ran may report the same version number. A later release can still lag behind that engine, so install the revision environment/requirements.txt names. See Installation for installing the engine from its source.

Running a bundle

mkdir study-s17 && unzip acme-brand-s17-data.zip -d study-s17 && cd study-s17
bash environment/run.sh

run.sh does, in order: pip install -r environment/requirements.txt; siamang validate survey/questionnaire.py; exports SIAMANG_PROVENANCE (the text of PROVENANCE.md, which becomes every report's footer — only while End every report with the provenance footer was on at that Save) and PYTHONPATH=.; then runs every flow script in the order Studio's Run all runs them — a flow after the flows whose tables it reads — each on the data it reads:

python scripts/cleaning.py --data data/responses.main.csv
python scripts/tables.py --data data/tables/clean_responses.csv
python scripts/waves.py --data-src-w1 data/responses.csv --data-src-w2 data/tables/wave2.csv

A flow with one data source gets --data and that source's file; a flow with several gets one --data-<node-id> per source. A flow with no source but a Write table gets --data data/responses.csv, which only switches its platform-only steps off.
A flow without a script at that Save is not run.
Every name that comes from a flow document is quoted for the shell, and a flow whose data source names no table or environment a project can have is not run (the README's Notes say so), so a flow document cannot run commands on your machine.

It stops at the first error — unlike Run all, which carries on past a failed flow. Reports and files land in outputs/. run.sh no longer sets a report look: each flow renders in the look written into its script (see The reports).

The README's What each flow reads section has a table with the columns Flow, Source, On the platform and Here — one row per data source — and a list of what a file cannot stand in for, for example "Write table steps do not run here: they write to the project database, and the scripts skip them when given a data file." and "The data files are the snapshot taken when this bundle was built; on the platform a flow reads the database as it is when the flow runs."

A bundle without responses ("Code & documents") names the files the flows read but does not contain them. Its README says: "This bundle was made without data. Put the files the flows read (below) in place, or download the bundle with data from History, before run.sh." run.sh checks for them first and stops with "missing data/responses.csv: this bundle has no data — download it with data from History, or put the file there". Only the files a flow that run.sh runs would read are asked for. A file you supply is read as it is: no environment or completion filter is applied to it.

Step by step (the README lists the exact lines):

pip install -r environment/requirements.txt
siamang validate survey/questionnaire.py
siamang preview survey/questionnaire.py
export PYTHONPATH=.
python scripts/satisfaction.py --data data/responses.main.csv

These lines do not set the report footer. For it, also run export SIAMANG_PROVENANCE="$(cat PROVENANCE.md)" before the flow scripts, as run.sh does while the footer setting is on.

The reports

Each flow writes outputs/<flow>.md and, with Also save HTML, its .html twin, in the look of its own Save report node — the theme written into scripts/<flow>.py — so a report renders here as it did in Studio. A flow that names no look renders with the engine's defaults, here as there. report/theme.json is the project's house style at this Save, which Studio stamps into new Save report nodes and renders the combined report with; nothing in the bundle reads it. To render a flow that names no look in the house style, run that flow with SIAMANG_REPORT_THEME="$PWD/report/theme.json".

PROVENANCE.md

A table stating what everything in the bundle was made from:

Row	Example
Project	`acme-research/brand-awareness-2026`
Questionnaire	`Save #17 — Add charging question (2026-06-04 14:32 UTC, sha256 a3f2c81d09be), environments pilot, main`
Data	`2026-06-20 09:00 UTC · 1247 responses (latest) · sha256 9f1c0b7e44d2` — or "not included — analysis reads the project database"; then one row per other data file: `data/responses.main.csv` · 1180 rows · sha256 5b0e41c9a7d3
Flow	one row per flow: `flows/satisfaction.flow.json @ Save #17 (sha256 d8194e5a0c77)`
Engine	`siamang 0.6.0 · python 3.11` — the Python Studio ran the flows on
Generated by	`Siamang Studio <version> (codegen flow 1.0, questionnaire <version>)`
Validation at Save	`valid`, `warnings` or `error`
Pre-registration	only when a Save is tagged: "this Save (#17) is the pre-registration", or the registered Save and the documents "changed since"
Bundle built	when the zip was made

It ends with "Hashes are SHA-256 of the files in this bundle (sha256sum survey/questionnaire.json)." — check them by hand. While Settings → Reports → End every report with the provenance footer was on at that Save, every report produced by run.sh repeats this table in its footer; with it off, run.sh does not set the footer. (Reports produced on the platform carry a shorter footer — see The provenance footer.)

What reproduces, and what does not

Result	From the bundle?
The questionnaire as a program (`validate`, `preview`)	Yes, with the engine pin described above (`preview` shows the pinned engine's survey runtime)
Simulated data	Yes — the seed is in the flow; at a pin that lags behind Studio the node stops with an error
Tables, tests and weights of a flow whose source is Responses	Yes, for the same data file and engine — at a pin that lags behind Studio, the weighted MaxDiff and conjoint estimates, bar charts, heatmaps of means, principal components, scale reliability and TURF's frequency come out unweighted
t-test, Correlation matrix, Paired tests, Factor analysis, Descriptive statistics, Data check, MaxDiff scores, Bands, Key drivers, Perceptual map, Price sensitivity, TURF, Regression's `ordinal` model, and a test chosen by hand (Correlation `pearson` / `kendall`, a Group means Test other than `auto`, Compare groups with Dunn's test, Crosstab with Fisher's test)	Yes, with the engine Studio ran — deterministic, including Fisher's Monte Carlo p and parallel analysis, which draw from fixed seeds. At a pin that lags behind Studio these nodes stop with an error
Likert chart, Result chart, Trend, Tab book (Excel), a Bar chart with Show `percent`, Split by, Sort `value`, Top N, Confidence intervals or Layout `histogram` or `donut`, a Heatmap with Pearson or Kendall, a chart of Palette `theme` and a Look's chart colors, Save report's tables in Excel	Yes, with the engine Studio ran (charts visually, as below; a Trend over `created_at` reads the timestamps the bundle's data files carry). At a pin that lags behind Studio these stop with an error
Export file	Yes — the same files under `outputs/`, the R bundle (`.R`) and codebook (`.json`) included; at a pin that lags behind Studio every Export file node stops with an error
Environment and Only completed responses on a Responses node	Yes in a bundle with data: each node reads a file already filtered its way. With a file you supply, it is read as it is
Speeders (interview length) and other fieldwork columns	Yes in a bundle with data: `duration_s`, `started_at`, `captcha`, `tab_switches`, `hidden_seconds` and `pastes` are in the files
URL parameters (`url_*`)	Only those a flow names; the others — and always the invitation token — are left out of the files
A flow that reads a Project table written by another flow	Yes, from a snapshot: it reads `data/tables/<table>.csv`, the table as it was when the bundle was built, with its variables. Running the writing flow in the bundle does not refresh it
Write table	Skipped (there is no project database)
Code open answers	Yes — the codeframe is in the bundle; no model is called. At a pin that lags behind Studio the node stops with an error
Data file nodes	Yes — a bundle with data carries the uploads the flows name; otherwise place the file at the path the node names
Charts	Visually yes; byte-identical only with the same matplotlib version and fonts
Reports (`.md`, `.html`)	Yes, in each flow's own look (the engine's defaults where a flow names none, as in Studio), with the bundle's provenance footer when the footer setting is on
Live tiles	No — they exist only on the platform
The combined report of Run all	No — `run.sh` runs the flows but does not assemble it
Live responses	No, by design — a bundle carries a snapshot of the data

The honest summary: with the engine Studio ran and a bundle made with data, every flow reruns on the data a run in Studio would have read when the bundle was made — filtered the same way, in the same order, chained flows included — and gives the same tables and reports. What differs: tables other flows wrote are a snapshot rather than rewritten, Write table, Live and the combined report are platform-only, and an engine pin that lags behind Studio can stop a flow or change weighted numbers — the README says when that applies.

Which upload a run read

A flow reads an upload under Files as it is when the run starts, and an upload replaced with new content keeps its name. So that two runs over a replaced file can be told apart, the log of every run — of one flow, Run all, a scheduled run — names each upload it copied in, with the start of its content's SHA-256:

read assets/panel_wave2.csv (sha256 4be1c07a93d2f518)

To check that a file you hold is the one a run read, compare the first 16 characters of sha256sum panel_wave2.csv with that line. A research bundle with data records the full hash of each upload it carries in data/responses.manifest.json, under uploads. On Files, a file's Updated date says when its content last changed (see The Files screen).

METHODS.md and CITATION.cff

METHODS.md is a draft Methods section built from the Save's documents — "nothing is invented; passages marked [...] need you". Sections: Participants (with the response count when data is included), Instrument, Measures, Procedure, Data handling and analysis (one entry per flow, its steps in order, each analysis with the test it runs — the correlation method, the t-test's design, a post-hoc test and its adjustment, the factor extraction and rotation; what Dedup respondents, Speeders & partials and Missing values dropped and by which rule; "This flow also fed a live dashboard during fieldwork." for live flows), Pre-registration, Software, and the authors on record. The same text opens from History → a Save → More ▾ → Methods (Rendered | Markdown, Copy Markdown, .md). See History and Versions.

CITATION.cff (Citation File Format 1.2.0) makes the Save citable by GitHub, Zenodo and reference managers:

Field	Taken from
`title`	Study title, else the project name, else the questionnaire title
`authors`	Authors of Study & citation — given and family names, `orcid` as `https://orcid.org/<id>`, `affiliation`; if none, the organization
`version`	`save-N`
`date-released`	the Save's date
`abstract`, `keywords`, `license`, `doi`	the matching Study & citation fields, when set
`type`	`dataset`
`references`	the siamang engine and its version

Citing a study

Fill in Settings → General → Study & citation: Study title, Authors (one per line, Name; ORCID; affiliation), License, Keywords, DOI, Abstract, then Save study metadata (this creates a Save). See Project Settings.
Open the Save to cite in History and choose More ▾ → Deposit to send its bundle to Zenodo (which mints a DOI) or OSF. The box "Use sandbox.zenodo.org (test DOI; needs a sandbox token)" is checked by default — uncheck it, with a zenodo.org token, for a real DOI. With "Include the data collected so far: data/responses.csv, and the project tables and uploaded files the flows read. Survey-link parameters (panel ids) are included only where a flow reads one; invitation tokens never are." checked, the deposit publishes the same data files as a bundle with the responses. See Depositing to Zenodo or OSF.
Put the DOI back into Study & citation so later bundles carry it.
Cite the deposit, and attach the bundle as supplementary material.

Your paper then ships the executable instrument and analysis, not a screenshot of a questionnaire. If you pre-registered, tag the Save before fieldwork: every later bundle's PROVENANCE.md and Methods draft then list what changed since.

Licensing note

The siamang engine that runs your questionnaire and analysis is source-available and free for non-commercial use — personal use, research, education and non-commercial organizations — so a bundle runs on it at no cost for such purposes. Paid Studio plans include a commercial license for the engine. For a commercial license for using the engine outside Studio, write to info@siamang-team.org. See Plans, Trial and Billing.

Moving to the siamang library

A bundle is also a starting point if you want to work with the code directly. Install the engine (Installation), then use questionnaire.py and the flow scripts as ordinary programs; the CLI Reference covers validate, preview, codegen and the rest. After editing a JSON document, regenerate its code with siamang codegen <document>, as the README says.

Studio Reproducibility

Reproducibility: Code, Bundles and Citation

Download .py

questionnaire.py

A flow script

The research bundle

What is inside

Installing the engine

Running a bundle

The reports

PROVENANCE.md

What reproduces, and what does not

Which upload a run read

METHODS.md and CITATION.cff

Citing a study

Licensing note

Moving to the siamang library

See also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!