Highlights
The .ohlcv data file has a new, self-describing format.
The v1 file was a headerless array of 24-byte records, addressed arithmetically as
(timestamp - first_timestamp) // interval — with the interval itself guessed from the first two
records. A single off-grid bar therefore shifted every following index permanently, gaps had to be
papered over with phantom records marked volume = -1, and nothing about the file was declared:
not its timeframe, not whether it was gap-free, not what the columns meant. The uint32 second
timestamps also ran out in 2106.
v2 declares all of it. An 8-byte magic and a 64-byte fixed header carry the version, the period, a
writer-verified DENSE flag, the record count and the first and last timestamps; 24-byte column
descriptors give each column its role, dtype and explicit byte offset. Addressing is a bisect over
the stored timestamp column, so an irregular feed is an ordinary case instead of a corruption
source. Timestamps are int64 milliseconds.
high/low/close are stored as f32 deltas from the bar's open, which keeps a mintick-exact tick
even for instruments whose absolute price exceeds f32 precision; when mintick says the delta cannot
round-trip, those three columns promote to absolute f64 and the record grows from 36 to 48 bytes.
OHLCVReader.open() reads the magic once and binds either the v2 or the legacy reader, so callers
never branch on the format. v1 files stay readable — there is no v1 writer.
Breaking changes
OHLCVWriter requires the period
Because the format is no longer derivable from the bytes, the writer needs the period up front:
OHLCVWriter(path, period, *, mintick=None, timezone="UTC", truncate=False)It analyses the data it writes and exposes tick size, price scale, min move, a mincontract
candidate derived from the volume column, and the session intervals implied by the bar times — the
latter bucketed in the symbol's timezone rather than the host's.
The v2 writer refuses to append to a v1 file and tells the user to re-download.
OHLCV.timestamp is milliseconds
OHLCV.timestamp is in milliseconds everywhere now, including the provider interface. Call sites
migrated accordingly: currency rate lookup, the security resolver, the aggregator, bar magnifier,
live runner, LTF collector, script runner, the CSV bridge and the CLI.
core/ohlcv_file.py is split
The module is now core/ohlcv.py (format, reader, writer), with core/ohlcv_importers.py for the
CSV/TXT/JSON imports and core/ohlcv_legacy.py for the read-only v1 implementation.
pynecore.core.ohlcv is the single public entry point.
A module-level record_count(path) answers "how many records does this file hold" without opening
a reader, and returns 0 for anything unusable — so it works directly as a cache-intactness gate.
Fixes
Downloads no longer predict where the next bar opens
The period alone cannot say it: calendar months vary and daylight-saving shifts move the opening
instant in UTC. A continuation now re-requests from the stored last bar, and
ProviderPlugin.save_ohlcv_data() drops everything up to that boundary.
Reopening a writer for append also read the last timestamp by extrapolating
first + interval * (size - 1). Because a real feed is not evenly spaced (monthly bars are 28-31
days apart by definition, and any halt or missing session breaks it too), that value ran far ahead
of the file and rejected every legitimate append as out of order. It is read from the last record
now.
Conversion is atomic
The output is built as <stem>.converting.ohlcv and published only on success, with both
destinations set aside first and restored if any step of the swap fails, so a failed refresh can no
longer destroy the previous good dataset.
Declared and inferred periods are compared by meaning, so a hand-written period = "D" no longer
collides with an inferred 1D. Period inference recognises calendar months from the wall-clock
stamps — month-end dating and missing months included — instead of inferring them from elapsed
days, which confused a leap-February month with 29 days and an evenly spaced four-week cadence with
a month.
The extra-fields sidecar is kept aligned with the binary record count on open and after a failed
write, and a record is committed only once both parts are on disk, so an interrupted append can no
longer leave the pair out of step and render the whole dataset unreadable.
Text-file detection no longer treats a decodable prefix as proof of text: a valid record whose
bytes all fall below 0x80 decodes as ASCII just as well. A record file is a whole number of
fixed-size records, so the length settles it first.
Legacy range queries search the stored timestamps directly. The old interval arithmetic returned a
stale bar for a window past the end of the file, a negative size for one before its start, and
wrong bounds for single-record and irregularly spaced files.
Overload dispatch rejected calls that omitted optional parameters
The slow path type-checked the __dyn_default__ sentinel that DynamicDefaultTransformer
substitutes for a lib.* default (= na above all). The sentinel is a bare object() and matches
no annotation, so every overload whose optional parameters the caller omitted was rejected with
"No matching implementation found". Such arguments are skipped now.
plotshape/plotchar/plotarrow keep the series value
They store the series as-is and narrow only a genuine bool to 0/1. TradingView exports the value
a numeric marker series carries, not a truthiness flag, so the usual cond ? high : na idiom keeps
its price in the exported column.
request.security() data resolution
- The chart's own symbol at a different timeframe no longer falls back to the chart feed. A coarser
context would start at the chart's first bar instead of carrying its own history, and a finer one
needs sub-bars the chart feed does not hold; both silently produced wrong series. Such a context
now reaches the canonical "no data" error that tells the caller to supply a feed. - The global symbol map skips the chart's own symbol, which Pine guarantees is the same instrument
as the chart, so a map entry can no longer route it to another venue's prices. - A mapped symbol whose derived file is missing degrades to a hint appended to the canonical error
instead of raising its own, which callers cannot match on to discover contexts. - Same-context detection accepts both the bare and the exchange-qualified spelling of the chart
symbol.
Number formatting of non-finite values
An infinite value reached the digit patterns, where round() raises OverflowError and Decimal
raises InvalidOperation. str.tostring() and str.format() disagree on the output, so they are
handled separately: the chart formatter prints NaN for every format but format.mintick (raw
double), while DecimalFormat prints the infinity symbol carrying the pattern's prefix and suffix.
Percent NA also keeps its % in str.tostring().
ta.crossover/ta.crossunder first-bar seed
They seeded their previous-relation state with NA(bool). Pine has no na bool: with no earlier
relation there is no cross, so the first bar where both sources are defined yields false.
Daily, weekly and monthly bars anchor to the session
The resampler anchors single-period D/W/M bars to trading session opens and handles session
boundaries using opening hours, while preserving midnight behaviour for 24/7 markets. Session
metadata is carried through security resolution. Regression coverage added for FX, equities, crypto
and scheduled day boundaries.
Comma-separated session ranges
Multiple session ranges with shared day filters are parsed, and a bar matches if it falls in any of
them. Coverage added for daytime, overnight, malformed and day-filtered sessions.
Log message column no longer expands
The rich log table no longer stretches the message column to the full terminal width.
Features
Corporate action ticker ID helpers
__dividends_tickerid(), __earnings_tickerid() and __splits_tickerid() map a regular ticker
identifier onto TradingView's corporate-action feeds, so request.security() can read dividend,
earnings and split events as a daily series. TV-measured: NASDAQ:AAPL becomes
ESD_FACTSET:NASDAQ;AAPL;DIVIDENDS — a purely syntactic rewrite.
Changes
Session checks moved to core.session
Session overlap and point-in-time checks are centralised in pynecore/core/session.py; the live
runner, the CLI and the library callers use those functions. The is_in_session module property is
registered.
Documentation
Persistent and IBPersistent rollback behaviour and historical series support are documented, as
are the Series and PersistentSeries typing semantics. The OHLCV reader/writer, extra fields,
CSV reader/writer, data CLI and programmatic docs are updated for v2.