v6.6.0 — Plugin system: extensible providers, live trading, and an interactive data UI

The largest release since the 6.5 line, and a change of kind rather than degree. PyneCore stops being a single fixed program and becomes a platform: data sources, live streaming, order execution and even CLI commands are now plugins, discovered automatically and layered into a capability stack. Built on that foundation this release adds a live run mode, an interactive terminal UI for downloading market data, structured visual output for charting, and — at the top of the stack — trading against real exchanges. All of the 6.5.x TradingView-parity work (interpreter, strategy simulator, request.security()) is carried forward unchanged.

The live and broker layers are new; see the Status section before running them against a real account.

The plugin system

Everything extensible now hangs off one Python entry-point group, pyne.plugin. Install a package that registers it and PyneCore finds it at startup — no registration code, no config edits, no imports. A single class hierarchy decides what each plugin can do, and every layer strictly extends the one beneath it:

• ProviderPlugin — an offline OHLCV data source (pyne data download).
• LiveProviderPlugin — a provider that can also stream bars in real time. Because it is a provider, every live source can also download history.
• BrokerPlugin — a live provider that can also route orders. Because it is a live provider, a broker also delivers the market data its orders trade against.
• CLIPlugin — a parallel axis: adds pyne <name> … subcommands and injects flags into existing commands.

Multiple inheritance combines them, so one package can provide data, stream it and add management commands at once. The layering is the point: capabilities compose upward instead of every integration re-implementing the whole surface.

Around that core:

• pyne plugin list and pyne plugin info <name> — enumerate installed plugins with their capabilities (provider / broker / cli / library), and show a plugin's config fields, minimum PyneCore version and — for brokers — a colour-coded exchange-capability table. --json output for tooling.
• Self-healing config. Each plugin gets a workdir/config/plugins/<name>.toml, generated from a typed Config dataclass: defaults appear commented out, user edits survive regeneration, new fields appear and removed fields disappear automatically. The Config type parameter flows into self.config, so IDEs type it precisely.
• Pluggable CLI. A plugin can mount its own command group under pyne <name> and inject --flags into run and data download; the values arrive in ctx.plugin_params, name/option collisions are detected and rejected with a warning, and built-in command names are protected.
• Package convention. Official plugins are pynesys-pynecore-<name>, third-party ones pynecore-<name>; CLI-capable plugins depend on the [cli] extra.

An authoring guide (docs/development/plugin-system.md) walks through building one from a five-line "hello" plugin upward.

A discoverable plugin registry — so installed and available plugins can be browsed and found in one place — is planned for a future release.

Interactive data download (terminal UI)

Downloading data no longer requires knowing a symbol's exact ticker in advance. Run pyne data download with just a provider name (or an unresolved selector) on a real terminal and PyneCore drops into a full-screen, two-pane UI:

• Browse and inspect. A scrollable, filterable symbol list on the left; a live SymInfo panel on the right (description, type, tick size, point value, currencies, opening hours, and where available spreads and fees). Symbol details are fetched lazily on a background worker, debounced so fast scrolling doesn't flood the provider, and cached.
• Search. Press / for an incremental, case-insensitive filter; arrow keys, PgUp/PgDn, Home/End navigate.
• Download in place. Enter opens an inline wizard — timeframe, from/to (relative presets like "30 days back" or custom dates), and a truncate toggle that appears only when the target file already exists — then a live progress strip with elapsed time and ETA. Pick and download many symbols in one session without leaving the UI.
• Pickers. With no provider argument you get a provider picker; a multi-broker provider (CCXT reaches 100+ exchanges, cTrader every account's broker) first shows a broker picker. A failed open (e.g. an exchange needing credentials) re-displays with an error instead of exiting, so the picker → browser flow is a loop.

The whole UI is keyboard-driven, resize-aware, and TTY-only — non-interactive runs still take an explicit symbol.

Provider strings and data download

A single addressing syntax now spans pyne data download, pyne run and request.security():

provider:[broker:]symbol[@timeframe]

e.g. ccxt:BYBIT:BTC/USDT:USDT@1D, ccxt:BINANCE:ETH/USDT@240, capitalcom:EURUSD@60. The old --symbol / --timeframe split still works.

• Multi-broker providers select the exchange from the segment after the provider name; --list-brokers enumerates them.
• --chunk-size overrides the per-request bar count; --extra-data additionally downloads ask/bid/spread into a sidecar CSV.
• Re-download from the file. The canonical provider string is persisted next to each download, so pyne data download <file> -f continue resumes it without reconstructing the CLI arguments.
• Databento CSV import is auto-detected by convert-from; PYNE_DATA_DIR relocates the data directory; a .pyne marker file lets any directory act as a workdir.
• Data-gap reporting. After a provider download, run warns when it received fewer bars than requested and localises the cause — venue history horizon, in-session missing slots, or known-closed sessions — with the exact missing intervals.

Live mode

pyne run --live (-l) runs a script beyond history: after replaying the historical window it transitions seamlessly into a real-time stream from a LiveProviderPlugin. Indicators update on every intra-bar tick and strategies run tick-accurately, matching TradingView's live semantics — correct barstate per tick, and varip accumulating across ticks while var rolls back to the bar-open snapshot. Historical-phase strategy calls are suppressed so no phantom trades are booked before the live handoff.

• Provider mode: the data argument accepts a provider string and downloads history automatically; --from -N means "N bars back" (default -500).
• A feed-liveness watchdog forces a reconnect on a stale or half-open feed; reconnection is retried indefinitely with backoff, so a bot survives long outages.
• Graceful shutdown on Ctrl-C/SIGTERM polls the provider's can_shutdown() up to --shutdown-timeout seconds; a periodic heartbeat line replaces the spinner under a durable log.

Without --broker, live fills are paper-simulated with intra-bar order processing.

Broker trading

pyne run --broker sits at the top of the plugin stack: it routes real orders to an exchange through a BrokerPlugin instead of simulating them. The core owns the hard part — a venue-agnostic execution engine with exactly-once client-order ids, idempotent restart and reconnect recovery, ownership across concurrent runs on one account, software fallbacks for protection/bracket/trailing legs a venue can't express natively, and a fail-closed read path that refuses to open exposure on an unconfirmed view — while the plugin only wires the exchange's REST/WebSocket calls. State persists to a shared SQLite journal with automatic stale-run cleanup.

Three broker plugins ship alongside this release, each as its own package:

• Capital.com (pynesys-pynecore-capitalcom) — CFD, native brackets/trailing/OCA, hedging and netting.
• Bybit (pynesys-pynecore-bybit) — crypto spot plus linear and inverse perpetuals.
• cTrader (pynesys-pynecore-ctrader) — CFD across cTrader brokers over the Open API.

Structured visual output

pyne run --viz (-vz) writes a compact NDJSON stream (<script>_viz.ndjson) describing everything a chart would render — per-plot style metadata, per-bar values, only-on-change dynamic colours, and the full drawing registry (lines, labels, boxes, tables, polylines, linefills) — that the flat plot CSV cannot capture. --viz-journal additionally emits per-bar create/update/delete drawing events. The plot CSV is never altered.

This required completing the whole Pine plot family (plot, plotshape, plotchar, plotarrow, plotcandle, plotbar, bgcolor, barcolor, hline and a three-form fill), which now record their full styling. Programmatically, ScriptRunner exposes plot_meta, drawings() and an assignable viz_events callback, so a headless run — or the IDE (coming soon) — can reconstruct a full chart.

Pine language and engine

• request.security() lookahead modes. Alongside the default repaint-free closed-bar behaviour, this release adds a new PyneSys-native barmerge.lookahead_last_closed (repaint-free in both historical and live) and TradingView-compatible barmerge.lookahead_on handling. ticker.heikinashi() now works inside request.security() in live mode.
• @pyne edge script variant. A script may open with """@pyne edge""" to mark code confined to a strict Pine-equivalent Python subset — a no-op at runtime that enables future tooling such as Pyne-to-Pine conversion; PyneComp emits it automatically for compiled Pine.
• The interpreter, strategy simulator and the rest of the TradingView-parity surface shipped through the 6.5.x line are carried forward unchanged.

Developer experience

• Offline broker conformance lab (python -m pynecore.testing.broker_lab) exercises a broker plugin against the real sync engine, store and position layer without a live venue — deterministic lifecycle, fault-injection and idempotency scenarios, with a machine-checked coverage contract.
• Debugger-safe source locations for transformed Pyne modules, so breakpoints land on the right line instead of firing mid-prologue.
• Improved IDE typing for module-property builtins, and wide durable logging that keeps long order/UUID tokens on one grep-able line when output is redirected.

Status

The plugin system, the interactive data UI and structured visual output are stable and ready to use. The live and broker layers are new and should be treated as a release candidate.

The execution architecture has been validated end-to-end against three independent brokers — Capital.com, Bybit and cTrader. On live demo venues the order, position, protection and partial-close flows, restart and reconnect recovery, idempotency, and ownership across concurrent runs are all green, together with short fault-injection. What remains before we call the subsystem production-grade is multi-day unattended soak testing and a full bot-platform pilot.

Until those gates close, run the broker plugins against demo/paper accounts, and expect the plugin-authoring API to still refine in minor ways. The rest of the core is unchanged and stable.