fix(runtime): fill intraday tail rebalances at the execution bar's VWAP, and divide corporate actions out of holding returns - #75
Merged
Conversation
The tail model filled at the CLOSE of the earliest 1min bar in the execution window — a single tick (that minute's last print), which on a thin name can be an outlier or a stale price and which one small trade can move. The volume weighted average of every trade in that minute (amount / volume) is the honest proxy for an order worked across the minute, and it takes real volume to move. `IntradayExecutionConfig.execution_price_basis` / `IntradayCfg.execution_price_basis` select it; the default is now `bar_vwap`. `bar_close` is kept and reproduces the previous ledger bit-for-bit (the I4 locks in test_intraday_execution.py pin it and assert the original values). WHICH bar executes (`execution_model`) is unchanged. A bar whose VWAP is undefined — volume or amount missing, NaN, non-finite or non-positive — is an EXPLICIT block on the existing missing-price path, keeping its reason and its bar timestamp. It never falls back to the bar close and never to the daily close; a name with no traded shares in its execution minute simply does not trade that rebalance. The I5b price-limit gate is unchanged in order and basis: the bar is selected first, then the gate compares the RAW price that actually executes against raw stk_limit. A bar VWAP is raw amount over raw volume, so it is raw by construction and no adjustment is introduced. An unpriceable bar blocks before the gate runs and therefore requires no limit-coverage row. Both intraday reports now disclose the active basis.
A limit-locked minute prices at the limit, but tushare rounds `amount`, so the bar VWAP only approximately equals that price. Measured over real cached 1min bars (25 shard files, 16,704 locked minutes): amount/volume equals the locked price exactly 78.5% of the time and falls more than 1e-6 below it 9.3% of the time (0.8% beyond one fen, max 0.21 RMB). With the default `limit_tolerance=1e-6` such a bar therefore no longer trips the up-limit gate, where a close-based gate saw the limit exactly. The default is left untouched — this only records the interaction as an executable fact so the band is calibrated by decision rather than by rounding.
…ll price Filling at the bar VWAP silently weakened the I5b limit gate. A limit-locked minute ends AT the limit, so its close equals the limit price and an exact comparison catches it. Its VWAP does not: every print on the way into the lock drags the average below the limit, so a genuinely limit-up name stopped being recognised as at-limit and the buy that I5b exists to block went through. Measured on real cached 14:51 bars joined to raw stk_limit (6,451 bars, 97 of them closing at the up limit): gating on the VWAP with the default limit_tolerance=1e-6 fails to block 5.2% of genuinely limit-up minutes, and 1.0% are more than one fen below the limit. Worst observed: 601858.SH 2023-05-15, low 45.57 -> locked close 46.13 == up_limit, VWAP 46.048583 — 0.0814 RMB (0.18%) under the limit. No fixed tolerance separates that from a legitimate near-limit trade, so widening the band is not a fix. "What did the trade pay" and "was this name locked" are different questions and no longer share an input. ExecutionFill carries the selected bar's raw close as `limit_reference_price` alongside `exec_price`; the gate reads the former, the book pays the latter. Feasibility is therefore byte-identical across price bases and unchanged from before this branch, while fills move to the VWAP. A selected bar with a usable VWAP but a NaN close leaves the gate unable to check that pair: it is counted as unchecked and disclosed, never passed.
…ility Property over locked / sealing / normal / unpriceable execution minutes: every (symbol, direction) the VWAP basis allows must also have been allowed on the bar_close basis. Verified independently against main over a 45-pair synthetic cross-section: zero pairs more permissive, 13 strictly stricter (all zero-volume bars, now unpriceable and blocked both ways). With every bar priceable, feasibility is byte-identical to main on both bases.
…le tolerance Reverses the gate input chosen in 76a1782, per the user's decision, and encodes the mechanism that makes it the more faithful test. A limit-up execution minute has exactly two shapes. LOCKED (封死涨停): every trade prints at the limit, so the VWAP equals the limit up to rounding — there is no seller except at the limit and the queue is hopeless, so the buy must be blocked. OPENED (盘中打开): some trades printed below the limit, so the VWAP sits below it — those prints are direct evidence a fill was achievable, so the buy must go through. "Can I buy?" maps onto exactly that distinction, and the VWAP encodes it directly; the bar close misclassifies both edges (closed at the limit but opened during the minute -> over-blocks an achievable fill; closed below but locked most of the minute -> under-blocks). Calibration follows and is the easy part to get wrong: limit_tolerance stays at ROUNDING scale and is NOT widened. It was never changed on this branch — 1e-6 before, 1e-6 after, in both qt/config.py and the model signature; 33d064f only added a test and its reasoning is superseded here. Measured on real cached 14:51 bars joined to raw stk_limit (13,699 bars; 149 closing at the up limit): the 146 LOCKED sit a median 0.000000% and at most 0.0021% below the limit, the 3 OPENED sit 0.013%-0.017% below — an order of magnitude apart. At 1e-6 every OPENED minute is correctly allowed and 2.1% of LOCKED are misclassified as opened by sub-tick rounding; widening to 0.01 RMB would misclassify 100% of OPENED minutes as locked, re-blocking fills that demonstrably were achievable. "Opened but thin" is a CAPACITY question and stays in the I5f layer, not here. The divergence from a close-based gate is counted, not silent: opened_limit_up_minutes() / opened_limit_down_minutes() report the (date, symbol) pairs where the bar closed at a limit but traded through it. ExecutionFill keeps limit_reference_price for that labelling only; it never decides feasibility.
IntradayTailEventModel.holding_returns computed exec_price(exit)/exec_price(entry)
on RAW minute prices. The intraday cache stores unadjusted bars by design, so a
monthly holding period spanning an ex-dividend or split date booked the
mechanical price drop as a loss. Over a 5-year monthly study each name has
roughly five affected periods, each biased by its dividend yield (~1-3%, and
A-share ex-dates cluster in May-July); if the long and short legs carry
different dividend exposure the spread itself is biased.
Returns are now computed on adjusted prices:
(raw_exit * adj_factor(exit)) / (raw_entry * adj_factor(entry)) - 1
The per-symbol anchor that data/clean/adjust.py divides by cancels in the ratio,
so no qfq series is re-derived and the result is exact and window-invariant —
the same property that makes front_adjust safe for batch == incremental.
adj_factor comes from the daily panel already passed as calendar_panel (it is a
required core column and front_adjust preserves it), restricted to anchor dates.
No new plumbing, no new fetch, and no runner change. A missing / NaN /
non-positive factor drops that name's return for the period and is counted in
missing_adj_factor_pairs() — never silently treated as 1.0, which would
reintroduce the bias being fixed.
Scope held: only the RETURN is adjusted. Fills still pay the raw execution price
and the I5b price-limit gate stays RAW-vs-RAW, since exchange limits are quoted
on raw prices — locked by test_adjustment_does_not_leak_into_the_raw_price_limit_gate.
Decisions and fills remain minute-only; adj_factor is a corporate-action ratio,
not a price, so no EOD price leaks into the 14:50 decision. The module docstring
claim that the model "never prices off the daily panel" is corrected accordingly.
… residual The tolerance stays at 1e-6. The docstring now states what that costs and why it is not tuned: a locked minute misread as opened lets through a buy that was NOT achievable, so the residual errs OPTIMISTIC. It is bounded — limit-closing bars are 149/13,699 ~ 1.1% of execution bars and 2.1% of those are misread, so ~0.02% of bars overall — and re-fitting the cutoff on a 149-observation sample would be the post-hoc tuning this project refuses elsewhere. Revisiting it needs a fresh, larger sample, not this one. No behaviour change: both defaults remain 1e-6.
StackOverFlow11
added a commit
that referenced
this pull request
Jul 21, 2026
…t class Review of the first pass found the SAME stale wording surviving in _write_report()'s "Limitations" section, untouched by that commit and identical to what shipped before PR #75. Every I5b/I5c/I5f report would therefore have carried two contradictory descriptions of one check in a single document: the corrected one mid-report and the pre-#75 one at the end. The dev-facing _load_price_limits docstring had it too. Both now derive from the active execution_price_basis. The more useful half is why the first pass missed it. None of the three tests render _write_report at all -- they exercise limit_basis_lines() in isolation and _feasibility_lines() through a stub -- so that location was structurally unreachable by them, and enumerating the places I happened to know about was never going to converge. The new test scans both report modules for any phrasing that asserts the gate compares a CLOSE, so an (N+1)th copy fails rather than ships. "bar_close" the basis name and "that bar's single closing tick" the basis description are legitimate and deliberately not matched. Mutation evidence: restoring the stale Limitations sentence fails the guard, and it names the offending line: AssertionError: intraday_tail_framework.py still claims the price-limit gate compares a CLOSE. ... Derive the wording from execution_price_basis instead of restating it: line 966: f"execution-minute close to raw limits (see the section " pytest 1732 passed (1731 + 1), ruff clean, phase0 unchanged.
This was referenced Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes to the intraday tail execution path, both prompted by the user questioning why a signal known at 14:50 was being scored against a price formed at 15:00.
1. Fill at the execution bar's VWAP, not its close
The fill price is now the execution bar's
amount / volume— the volume-weighted mean of every trade in that minute — instead of its closing tick. A single-tick close is noisy, can be moved by one small print, and is a poor proxy for an order worked across the minute; the VWAP requires real volume and is the honest approximation. SameΣamount/Σvolumeidentity already used and hand-verified in the merged PR-I / PR-J factors. Configurable viaIntradayCfg.execution_price_basis: "bar_vwap" | "bar_close", defaulting tobar_vwap;bar_closereproduces the old behaviour bit-for-bit (verified by cross-worktree byte-comparison of the price matrix and the full fill ledger against main, not merely by assertions).An undefined VWAP (volume≤0, amount≤0, non-finite) routes to the existing explicit block path. It never falls back to the bar close and never to the daily close — silent degradation is a standing red line here.
The price-limit gate now compares the executed VWAP against raw
stk_limit, which is more faithful than gating on the close. A limit-up minute has exactly two shapes: locked, where every print is at the limit so VWAP == limit; and opened, where trades printed below the limit so VWAP < limit — and 'could I buy?' maps onto exactly that distinction. Gating on the close misclassifies both edges. This is a deliberate behaviour change and it is counted, not silent:opened_limit_up_minutes()/opened_limit_down_minutes()report every divergence from close-based gating. The 'opened' test is sourced from real cached data — 601858.SH 2023-05-15 14:51, close == up_limit 46.13, VWAP 46.048583, a 0.18% gap far above rounding scale.Tolerance deliberately left at 1e-6. Measured over 13,699 real 14:51 bars (149 closing at a limit): locked bars sit ≤0.0021% below the limit, opened bars 0.013–0.017% — non-overlapping populations. At 1e-6 every opened minute is correctly allowed, at the cost of 2.1% of locked bars being misread as opened (≈0.02% of all bars). A tolerance of 0.001 would be 0% in both directions on this sample, but recalibrating a threshold on 149 observations is the post-hoc tuning this project refuses elsewhere, and the effect is 0.02%. The residual errs optimistic and that direction is recorded in the docstring alongside the numbers.
2. Divide corporate actions out of holding returns (pre-existing defect)
holding_returnscomputedexec(exit)/exec(entry) − 1on raw, unadjusted minute prices across the whole rebalance interval. The intraday cache stores raw by design andqt/intraday_group_backtest.pypasses bars through unmodified, while the daily event model prices off the qfq panel — so the two paths were asymmetric and every ex-dividend/split inside a holding period was read as a loss. Now(raw(exit)×af(exit)) / (raw(entry)×af(entry)) − 1; the qfq anchor cancels in the ratio so no adjusted series is needed. A missingadj_factoris not silently treated as 1.0 — it joins the existing drop path and is counted bymissing_adj_factor_pairs(). Adjustment does not leak into the price-limit gate, which stays raw-vs-raw; a dedicated test pins this (a name with af=2.0 whose adjusted price would clear an 11.0 limit is still blocked on its raw price).Measured incidence — this is larger than a rounding correction:
fb/fa−1: mean / medianMean far above median indicates a right tail of bonus/rights issues where
adj_factorroughly doubles — meaning the raw price halves and the uncorrected return read about −50%. Over 59 periods the correction is of order +20% on each quantile's NAV, so the level statements in the existing I5d/I5e write-ups ("Q1 fell below 1.0", "every CSI300 quantile annualised negative") are unlikely to survive. Whether the spread and monotonicity conclusions survive depends on the differential correction between quantiles — if dividend exposure is similar across MMP quintiles the correction is common-mode and Q5−Q1 barely moves. Only the re-run decides that, and it is not asserted here.Test plan
bar_closebit-identity proven by byte-comparing the price matrix and full fill ledger (includingexec_timeand blockreason) against main run in a separate worktree over 240 constructed cells incl. NaN closes, zero volume, non-positive amount and out-of-window barsruntime/+qt/+ tests.analytics/eval/, the eleven factor modules anddata/cache/untouchedNote on the commit history
76a1782moves the limit gate to the bar close and3bb4c51moves it back to the VWAP;527a0e2adds an invariance test asserting the VWAP basis 'never widens feasibility' and it was subsequently deleted because that property is false under the correct design — an opened limit-up minute should become feasible. The history is kept unsquashed deliberately: it records that close-gating was considered and overturned on evidence, and that a self-authored test encoding a wrong premise was removed rather than left to lock in over-blocking.