fix: TradingView range-end accounting for open positions; pad the TV-entry gate by the feed's previous bar - #173
Closed
luisleo526 wants to merge 2 commits into
Closed
fix: TradingView range-end accounting for open positions; pad the TV-entry gate by the feed's previous bar#173luisleo526 wants to merge 2 commits into
luisleo526 wants to merge 2 commits into
Conversation
…s range-end close TradingView's deep-backtest report (the ws-report-v1 tape the campaign grades against) does not leave the last position open. It reports it as a closed trade whose exit leg is the range's last bar at that bar's close, with an empty exit Signal, and counts it in closedTrades. The canonical row is orb-lite on NYSE:F 1D: Entry short 2026-03-16 @ 11.82, Exit 2026-04-30 @ 12.08 -- 12.08 is the last close of the range, metrics closedTrades:1. On the f-1d spark set 8/10 engine runs held the same position to the last bar (bars-in-market = TV Duration + 1) and 7/10 had a mark-to-market open_pl equal to TV's row to the cent: the row is exactly the engine's open position marked at the last close. The engine exported closed trades only (trades_ grows in emit_close_trade alone; neither run loop flattened at end of feed), so every such probe was one trade short against the tape and the verifier scored the missing row as an unmatched TV trade. The browser export writes the same row with Signal "Open", which the verifier already recognises; the ws tape gives it no marker at all. Operator decision (2026-09-02): the engine emulates TradingView's range-end accounting, in the report. After the last script bar is dispatched and its equity point recorded, record_range_end_close_trades builds the rows a close of the open position at the last bar's close would record -- one per pyramid slice through the same arithmetic as every other close (build_close_trade, split out of emit_close_trade) -- into range_end_trades_, which fill_trades_section merges behind the script's closed trades. The live position, pending orders, trades_ and the realized sums are left as the bar loop left them: a stream's realtime bars continue the warmup position, Pine accessors never see the row, and the broker-mechanics tests keep inspecting the open lot. Pricing: the row is a mark at the close, not an order the broker fills -- the raw close rounded to the nearest tick (bar_fill_price, finding-446; the tape's 12.08 is the on-tick close), no slippage ticks, commission per the strategy's rules on the exit leg like any close (the tape in hand has commission 0, so that is the operator's call, not a measurement), dated on the script bar's label. Accounting: the report's closed-trade count, net profit, win/loss tallies and commission paid include the rows. Every equity point before the last is untouched; the last point is re-marked to the flat account (open_profit 0, equity = capital + net profit including the rows) so the report identity equity == initial_capital + net_profit + open_pl that test_metrics pins still holds on the last bar -- with the tape's commission 0 and an on-tick close that point's equity VALUE is the gross mark it already was. A first cut re-marked the point and left the scalar extremes at the bar loop's gross fold; review showed that broke the compute_equity_stats walk == scalar identity on every commissioned run whose last bar was the trough or the peak (the fold had seen the gross mark, the walk the net one), so the scalars are re-folded from the curve after the re-mark (fold_equity_extreme, split out of update_equity_extremes): the walk and the scalars agree again, and every earlier bar's drawdown / run-up is unchanged. A strategy flat after the final bar is unaffected. The measurement must END where TradingView's range ends, or the row is booked on the wrong bar. The engine marks after the last bar it is FED, and the ETH chart feed (eth-corpus-15 / eth-scraped-15) runs on to 2026-05-04 15:00 UTC while every ETH tape ends at 2026-05-01 00:00 UTC (286 of 305 with an Open row there @ 2261.44 = that bar's close); the ws-report lanes (F, AAPL, NIFTY, BTC, XAU, ES, EURUSD, NQ) end on the tape's last bar already. run_strategy.py now bounds the feed to the tape's latest row (_load_tv_range_end_ms over entries, exits and the open-position row -> ohlcv_end_ms, on both runners; the docker runner pre-trims through _trim_ohlcv_csv) whenever the TV window is in use. It hides nothing the emit window did not already: engine trades entered after TV's last entry were dropped regardless. The browser export's Open row is priced at the export-time quote, not a bar's print: all 12 sampled OANDA:EURUSD tapes end with Open rows @ 1.17256 while the feed's last bar is o 1.17289 h 1.17299 l 1.17282 c 1.1729 (246 of 358 EURUSD tapes end with such a row). The engine's row at the close can never price-match a mark. verify_corpus.py treats a TV Open row whose engine counterpart exits on the SAME bar as count-only (counted, covered, entry compared; exit/pnl deltas left out and the pair reported as open_mark_pairs); an engine exit on a different bar keeps its deltas, since TV held to the end and the engine did not. ws tapes write the row with no Signal and never take that path. pineforge-utils/validator/validate.py tracks this rubric and needs the same rule. pf_trade_t gains open_at_end (1 on the range-end row) -- ABI v3, with the ctypes mirrors (run_strategy, docker/run_json, tutorial, benchmarks) and the docs updated; run_strategy.py carries the flag on the report dict and writes the CSV row as an ordinary exit with no marker, the way the ws tape prints it. run_stream_corpus compares the trades the two brokers made, without the range-end rows. tests/test_range_end_close.cpp pins the row (long, short, flat, pyramid slices, commission on both legs, rounded close without slippage, aggregated-path label, earlier curve points untouched and the last re-marked flat, walk == scalar extremes with commission and the last bar as trough and as peak, re-run reset); test_run_inputs_overrides and test_accounting_reconciliation foot the report to the blotter plus the range-end rows; scripts/test_run_strategy_range_end.py pins the tape bound on the ETH shape for both runners and the harness's EXPECTED_PF_ABI to the header's PF_ABI_VERSION (the f-1d spark pre-check of 2026-09-02 failed all 64 probes with ".so reports 3, harness expects 2" while that guard was still 2 -- the verifier runs every probe through run_strategy.py); scripts/test_verify_corpus_open_mark.py pins the EURUSD Open-row pairing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXAE7TVTKZYbmtenHDHVJo
…endar The validator opens its emit / trade-start window one bar interval before TradingView's first entry so the signal bar that produced that entry can place its order, and trading_is_active pads the gate by one more script TF in milliseconds. Both pads are calendar-blind, and the daily lanes exposed it. The bar before a Monday is Friday, three calendar days back: orb-lite on NYSE:F 1D fires its short on the 2026-03-13 (Friday) bar and TradingView fills it on 2026-03-16 (Monday), its first entry. The window opened on Sunday (Monday minus 86,400,000 ms), the engine's buffer reached Saturday, and Friday's strategy.entry fell before the gate -- dropped, so the engine's tape started from a different placement entirely. The same shape hides on every session gap: a Friday-close signal on an intraday equity feed, a holiday, the 17:00 ET forex open whose opening minutes the tape does not print. The feed knows which bar precedes the first in-window bar, so the pad is now ONE bar, by INDEX on the feed, in ONE place. The engine resolves the script bar immediately preceding the first script bar at/after trade_start_time_ once per run (compute_trade_start_preceding_script_bar -- a preview of the same aggregation the bar loop performs, same TFs, timezone and session, so the index it finds is the bar_index_ the loop assigns), and when that bar resolves trading_is_active admits exactly it and every bar at/after the gate; the millisecond buffer survives only where the index cannot resolve (a gate past the feed -- a stream's warmup -- or before it). The validator sets the gate ON TradingView's first entry bar (run_strategy.py _load_tv_entry_window returns (first entry, last entry)) and no longer pads itself: the trades filter therefore also stops keeping a fill dated one bar before TV's first entry, where TV has none. A first cut kept the millisecond buffer beside the index rule and moved the validator's pad to the previous FEED bar; review showed that stacks two pads -- with the gate on Friday the buffer reaches Thursday, and a strategy firing on both of the two bars before TV's first entry books an engine-only trade the >= window-start filter keeps. One pad, in the engine, closes both holes. On a continuous feed the index rule names the bar the old subtraction named, so the 1m-feed/15m-script gate of validation/62 is unchanged bar for bar; a same-TF 15m feed, which the old pair of pads let two bars through, now admits one. tests/test_trade_start_gap_pad.cpp pins the gate: Friday admitted and Thursday refused with the gate on Sunday or Monday; one pad under a Friday gate (Thursday in, Wednesday out) and a mid-week gate; the aggregated 1m->15m path across an overnight gap under TV's label and the old 1m pad; the continuous 1m pin of test_strategy_commands_extra unchanged; one pad on a continuous 15m feed; no gate, a gate past the feed, a gate before it. scripts/test_run_strategy_gap_pad.py pins the window: it opens on the first entry (not Sunday, not Friday), ends on the last entry, honours the tape's timezone and the meta plumbing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXAE7TVTKZYbmtenHDHVJo
2 tasks
Collaborator
Author
|
Superseded by the M2-only PR: the gap-pad factor (M1) drops the first trade of the 15m window on six hard-lane probes (gate |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two commuting fixes for the daily-timeframe lanes, measured as a 2×2 on spark and as a six-lane experiment on Cloud Run.
M2 — range-end close (TradingView emulation). TradingView's deep-backtest report lists the position still open at the range's last bar as a closed trade at that bar's close. The engine exported closed trades only, so on 1D lanes where that is the only trade in the window the engine reported 0 trades ("no-trades", tier minimal) although it held the same position with the same mark-to-market P/L (spark: 7/10 to the cent). The engine now emits a report-only row for each lot still open after the final bar of the range (
record_range_end_close_trades,pf_trade_t::open_at_end, ABI v3), priced at the last close on-tick, commission per the strategy's rules, no slippage; broker state, Pine accessors and the equity curve before the last bar are untouched. The harness bounds the run to the tape's last row (ohlcv_end_ms) so the row lands on TradingView's range end, not the feed's end.M1 — TV-entry gate pad. The TV-entry emit window padded by one bar-interval in milliseconds, so a Friday signal filled on Monday fell outside the gate across a weekend. The gate now admits the feed's previous script bar by index (
trading_is_active,_window_start_before_first_entry).Measurement
exp-1d-range-end-20260902(all six 1D lanes, 1148 probes, 21 cases, 5 min): excellent+strong 74.0% → 93.6% overall — BTCUSDT-1D 77.6→93.8%, ES1-1D 79.5→96.6%, NQ1-1D 75.0→92.2%, NIFTY-1D 78.6→97.2%, NYSE-F-1D 68.4→92.8%, XAUUSD-1D 70.6→91.1%; engine-0 probes 134 → 12; 225 entrants, 1 leaver (3commas-ena-grid-botXAUUSD-1D: engine 28 vs TV 25 — one range-end row per pyramided open lot where TV carries fewer; follow-up).gate-cand-1d-range-end-20260902).Test plan
-UNDEBUG(newtest_range_end_close,test_trade_start_gap_pad, Pythontest_run_strategy_gap_pad/test_run_strategy_range_end); patches commute (treebc9b3797)gate_candidatePASS recorded🤖 Generated with Claude Code
https://claude.ai/code/session_01BXAE7TVTKZYbmtenHDHVJo