We built 19 Borsa Istanbul research systems but still cannot prove a real trading edge — what are we missing? #906
Unanswered
sedatguner2000-coder
asked this question in
Q&A
Replies: 8 comments
|
This is one of the most carefully framed questions we've had here, so let me answer the central one directly instead of proposing a twentieth architecture. Scope note first, so I don't waste your time: Vibe-Trading has no Borsa Istanbul engine. Our backtest engines cover A-shares, US/HK, India, Korea, crypto, futures and FX; BIST costs, circuit rules and lot conventions are not modelled. So read everything below as framework-independent methodology — it applies to whatever stack AVCI already runs on. The short answer: no test run on your historical period can settle this any more. You wrote that yourself, and it is the most important sentence in your post. After thousands of experiments and 19 architectures on the same history, that data is spent. Any further in-sample statistic — including an excellent one — is contaminated by the search that produced it. So the decisive experiment cannot be a backtest. It has to be prospective and pre-registered. Concretely, here is what I would pre-register. 1. Freeze and timestamp. Before the window opens, publish a hash (a git commit plus a public timestamp) covering the decision code, every parameter, the universe definition, the decision time-of-day, the cost model, and the allocation / holding / replacement / exit rules. Nothing inside that hash may change during the test; if anything changes, the clock restarts. This is what makes the result immutable instead of negotiable afterwards. 2. Declare the verdict rule in advance — including the failure branch. Before seeing any data, write down the horizon (I would want at least six months of trading sessions), the metric, the threshold that counts as success, the threshold that counts as failure, and the stopping rule. The failure branch matters most: pre-committing to "if after-cost return over N sessions is below X, we conclude there is no extractable edge" is the only thing that structurally prevents architecture #20. 3. Run a same-universe random control in parallel. This is the experiment I would prioritise above all others. Every session, alongside AVCI's real decision, draw K random baskets from the same eligible universe on the same date, and push them through the identical allocation, holding, replacement, exit and cost machinery. Then ask one question: does AVCI's realised after-cost curve sit outside the distribution of its own random twins? That single control separates the three possibilities you say you cannot distinguish. If AVCI sits inside the random distribution, the selector contributes nothing and the historical result was market movement plus search luck. If it sits outside, you have a real selector — and only then is it worth asking where the edge leaks. (We implement exactly this in 4. An ablation ladder, to localise where the edge dies — your possibility (2). Hold one stage real and neutralise the rest, over the same dates: (a) real signal + equal weights + fixed holding period + zero costs; (b) add real allocation; (c) add real holding and replacement; (d) add real costs; (e) add realistic execution. The rung where the curve collapses is your answer. A system of 20 modules cannot be debugged as a whole, and this is the only way I know to localise the loss rather than guess at it. 5. Execution realism, sized to your actual capital — your possibility (3). In a market with BIST's liquidity profile, the two assumptions that most often destroy a paper edge are fill price and participation. Cap each fill at a fixed fraction of that bar's traded volume, make slippage a function of that participation rather than a constant, then re-run the ladder at 1×, 3× and 10× your intended capital. If the edge survives only at sizes you would never actually trade, it is not an edge for you — and you get a capacity number, which you need anyway. 6. Adjust for the search itself. You ran thousands of hypotheses, so your best result is by construction an extreme order statistic. The Deflated Sharpe Ratio and the probability of backtest overfitting (Bailey and López de Prado) exist precisely for this and take the number of trials as an explicit input. Reporting an undeflated Sharpe after 19 architectures is the most common way a dead system still looks alive. On the missing ledger: you mentioned you cannot reconstruct a complete real-money ledger for the older results. I would stop trying. Treat everything before the freeze as hypothesis generation rather than evidence, and let the pre-registered window be the only thing you defend. That reframing costs you nothing you actually had, and it makes the eventual result defensible either way. If you do run this, I would genuinely like to see the outcome posted here — including if it is negative. "22 months, 19 architectures, one pre-registered prospective test, no extractable edge after costs" is a more useful public artifact than most positive results, and making that kind of verdict routine is precisely what the |
0 replies
|
Subject: Your guidance changed the direction of AVCI — thank you, and may I
ask for one further review?
Dear Haozhe,
I want to begin with my deepest and most sincere gratitude.
You are on the other side of the world, you do not know me personally, and
you had no obligation to spend your time understanding our project. Yet you
read our problem carefully and gave us a detailed, honest, and constructive
answer.
After approximately 22 months of work, 19 different architectures, and
thousands of hypotheses and simulations, repeated disappointment had made
it difficult for me to feel hopeful. While reading your response, I was
genuinely moved. Not because you promised success, but because you treated
our work seriously and showed us a defensible way to discover the truth.
Your kindness affected me deeply. I will always remember that someone far
away chose to help a stranger simply because he recognised a difficult
research problem. Thank you.
We have completed the retrospective part of your methodology and prepared,
but have not yet activated, the prospective stage. We did this without
creating architecture number twenty.
Everything below concerning historical results refers only to the C_SAFE30
validation path. It does not validate the full AVCI system.
What we did after your response
For the C_SAFE30 review, we re-downloaded official daily-session files and
rebuilt the complete 300-session sequence from clean source data.
The old published result of approximately +133.02% could not be reproduced
and was therefore rejected as evidence.
A real error was found in the old market comparison: on 12 days, stocks
with missing next-session results had been silently removed from the
comparison.
We then reconstructed each day’s eligible stock universe using only
information available at the decision time.
Using the existing frozen C_SAFE30 selection path:
Initial capital: TRY 10,000
Historical final capital: TRY 28,797.83
Historical growth: +187.98%
Maximum drawdown: −8.30%
Capital used per session: 25%
Assumed total cost: 0.70%
We created 100,000 random paths from the same daily eligible universe,
using the same capital, costs, holding rules and dates:
Random full-universe paths beating C_SAFE30: 0 / 100,000
Random paths selecting only among C_SAFE30’s daily Top 3 and beating
C_SAFE30: 3 / 100,000
We understand that this is only a strong historical signal—not independent
proof—because these 300 days were already exposed during the previous
research process.
We also tested the system as a ladder:
Selection signal without costs
Real 25% capital allocation
Holding and replacement rules
The normal 0.70% cost
A heavier 1.00% adverse execution stress
The historical advantage survived the cost increases. However, realistic
historical execution, partial fills, settlement delays and carried
positions cannot be reconstructed completely from the old records.
How AVCI currently works
The current project has 20 main modules and 23 supporting files. It
operates like a factory:
Delayed daily and one-minute BIST data are collected.
Timestamps, missing records and corporate actions are checked.
A point-in-time eligible stock universe is created.
Price, volume, market, sector and risk indicator families are calculated.
Our proprietary İz–Koku–DNA layers examine market traces, related movement
and historical similarities.
Previous winning patterns and similar historical DNA structures are used as
reinforcement information.
A broad candidate pool is formed.
Weak or inconsistent candidates are eliminated.
The remaining candidates are ranked and passed through risk, allocation,
holding and replacement controls.
The system produces a small final candidate list, generally Top 1–3.
The decision is sealed before the outcome is known.
Execution, costs, capital growth and random control paths are then recorded
separately.
What is still missing
We have prepared a new pre-registered prospective test with:
A frozen decision rule
100 same-universe random paths per day
TRY 10,000 as the real-capital path
TRY 30,000 and TRY 100,000 as capacity-stress scenarios, not live capital
deployments
Full-share quantities only
Partial-fill and unsold-position rules
No session may be removed after its outcome is known
Corporate-action accounting
File hashes and restart after any rule change
Predefined success and failure conditions
But the decisive evidence stage has not begun.
We still do not have:
An external public timestamp for the final freeze
A completed unseen prospective window
Real broker fill and fee records
Real execution evidence at the TRY 10,000 capital level
Execution-capacity evidence for the TRY 30,000 and TRY 100,000 stress
scenarios
We have pre-specified a provisional success threshold. At 300 official
sessions, the candidate must remain above its initial capital after all
real costs, avoid a drawdown of 20% or worse, and remain significantly
above its same-universe random paths. We would value your criticism of
whether this threshold is sufficiently strict.
We are therefore considering a clean-room rebuild of the implementation,
not a twentieth selection strategy. The purpose would be to preserve the
core AVCI logic while separating:
research data,
validation data,
prospective evidence,
execution records,
and final capital accounting
from the first day, so that the same historical contamination and ledger
problems cannot occur again.
May I ask for your opinion on four points?
Would you preserve C_SAFE30 as the frozen prospective candidate, or rebuild
the same logic in a clean implementation before starting?
Is our provisional success threshold against the random distribution
sufficiently strict?
Is our proposed ladder sufficient to identify where the advantage
disappears?
What minimum architecture and data-separation rules would you require
before trusting a new prospective test?
If you are comfortable with it, I would be deeply grateful to send you a
concise architecture diagram and pre-registration draft for a brief review.
I do not want to turn your kindness into an expectation of unpaid
consulting. Even a short warning about a serious methodological mistake
would be extremely valuable.
Your response did not give us another optimistic backtest. It gave us a
method for distinguishing a real advantage from a false one.
After 22 difficult months, that means more to me than I can fully express.
With my deepest gratitude,
After reviewing our records again in light of your message, I believe your
central diagnosis is correct:
Our historical data can no longer provide independent evidence.
The same periods were exposed to 19 architectures and thousands of research
decisions. Even when a later result appeared economically strong, we could
no longer know whether it represented a real effect or an unusually
favourable result selected from a very large search process.
We also do not have a complete, immutable economic ledger for the original
AVCI research chain. We cannot fully reconstruct every historical decision
together with real executable quantity, realised fills, all costs,
unresolved execution, corporate actions, and the complete daily capital
path.
Some promising results also came from separate research branches rather
than the complete AVCI production chain. Therefore, we cannot honestly
attribute every promising historical result to the full system.
We now accept that everything before the new freeze must be treated as
hypothesis generation, not proof.
Your proposed framework gives us a disciplined way to separate three
possibilities:
A persistent advantage may not exist.
A small selection advantage may exist but disappear during allocation,
holding, replacement, or risk controls.
An apparent paper advantage may disappear once realistic execution, traded
quantity, costs, corporate actions, and full capital accounting are applied.
Nihat Güner
Independent Software Engineer and AVCI Researcher
Türkiye
Haozhe Wu ***@***.***>, 30 Tem 2026 Per, 14:01 tarihinde şunu
yazdı:
… This is one of the most carefully framed questions we've had here, so let
me answer the central one directly instead of proposing a twentieth
architecture.
*Scope note first, so I don't waste your time:* Vibe-Trading has no Borsa
Istanbul engine. Our backtest engines cover A-shares, US/HK, India, Korea,
crypto, futures and FX; BIST costs, circuit rules and lot conventions are
not modelled. So read everything below as framework-independent methodology
— it applies to whatever stack AVCI already runs on.
*The short answer: no test run on your historical period can settle this
any more.* You wrote that yourself, and it is the most important sentence
in your post. After thousands of experiments and 19 architectures on the
same history, that data is spent. Any further in-sample statistic —
including an excellent one — is contaminated by the search that produced
it. So the decisive experiment cannot be a backtest. It has to be
prospective and pre-registered.
Concretely, here is what I would pre-register.
*1. Freeze and timestamp.* Before the window opens, publish a hash (a git
commit plus a public timestamp) covering the decision code, every
parameter, the universe definition, the decision time-of-day, the cost
model, and the allocation / holding / replacement / exit rules. Nothing
inside that hash may change during the test; if anything changes, the clock
restarts. This is what makes the result immutable instead of negotiable
afterwards.
*2. Declare the verdict rule in advance — including the failure branch.*
Before seeing any data, write down the horizon (I would want at least six
months of trading sessions), the metric, the threshold that counts as
success, the threshold that counts as failure, and the stopping rule. The
failure branch matters most: pre-committing to "if after-cost return over N
sessions is below X, we conclude there is no extractable edge" is the only
thing that structurally prevents architecture #20
<#20>.
*3. Run a same-universe random control in parallel.* This is the
experiment I would prioritise above all others. Every session, alongside
AVCI's real decision, draw K random baskets from the same eligible universe
on the same date, and push them through the identical allocation, holding,
replacement, exit and cost machinery. Then ask one question: does AVCI's
realised after-cost curve sit outside the distribution of its own random
twins?
That single control separates the three possibilities you say you cannot
distinguish. If AVCI sits inside the random distribution, the selector
contributes nothing and the historical result was market movement plus
search luck. If it sits outside, you have a real selector — and only then
is it worth asking where the edge leaks.
(We implement exactly this in alpha bench --strict: a same-universe
random control plus an out-of-sample gate. Different market, but the repo
is MIT and the control's construction may be worth reading —
agent/src/factors/bench_runner_strict.py.)
*4. An ablation ladder, to localise where the edge dies — your possibility
(2).* Hold one stage real and neutralise the rest, over the same dates:
(a) real signal + equal weights + fixed holding period + zero costs; (b)
add real allocation; (c) add real holding and replacement; (d) add real
costs; (e) add realistic execution. The rung where the curve collapses is
your answer. A system of 20 modules cannot be debugged as a whole, and this
is the only way I know to localise the loss rather than guess at it.
*5. Execution realism, sized to your actual capital — your possibility
(3).* In a market with BIST's liquidity profile, the two assumptions that
most often destroy a paper edge are fill price and participation. Cap each
fill at a fixed fraction of that bar's traded volume, make slippage a
function of that participation rather than a constant, then re-run the
ladder at 1×, 3× and 10× your intended capital. If the edge survives only
at sizes you would never actually trade, it is not an edge for you — and
you get a capacity number, which you need anyway.
*6. Adjust for the search itself.* You ran thousands of hypotheses, so
your best result is by construction an extreme order statistic. The
Deflated Sharpe Ratio and the probability of backtest overfitting (Bailey
and López de Prado) exist precisely for this and take the number of trials
as an explicit input. Reporting an undeflated Sharpe after 19 architectures
is the most common way a dead system still looks alive.
*On the missing ledger:* you mentioned you cannot reconstruct a complete
real-money ledger for the older results. I would stop trying. Treat
everything before the freeze as hypothesis generation rather than evidence,
and let the pre-registered window be the only thing you defend. That
reframing costs you nothing you actually had, and it makes the eventual
result defensible either way.
If you do run this, I would genuinely like to see the outcome posted here
— including if it is negative. "22 months, 19 architectures, one
pre-registered prospective test, no extractable edge after costs" is a more
useful public artifact than most positive results, and making that kind of
verdict routine is precisely what the --strict gate in this repo was
built for.
—
Reply to this email directly, view it on GitHub
<#906?email_source=notifications&email_token=B3NTTT264PKZ6TJUZC2E2ST5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17839330>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/B3NTTT3XN754IVCZ3ZRZRP35HMTHVAVCNFSNUABJKJSXA33TNF2G64TZHMYTCOJYGMYDMOBRGI5UI2LTMN2XG43JN5XDWMJQGUZDANBSHCQXMAQ>
.
Triage notifications, keep track of coding agent tasks and review pull
requests on the go with GitHub Mobile for iOS
<https://github.com/notifications/mobile/ios/B3NTTTY3H4TXHQ5Q3JX23535HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM>
and Android
<https://github.com/notifications/mobile/android/B3NTTT5C2UKBA4GUTBD4YPT5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>.
Download it today!
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
0 replies
|
Before deciding whether to activate the prospective test, I would be
especially grateful for your view on the architecture itself.
If you were rebuilding AVCI from zero today, what would be the smallest
architecture you would trust?
Which parts of the current system would you preserve, and which parts would
you remove completely before beginning a new prospective test?
Would you preserve the frozen C_SAFE30 selection rule during a clean
rebuild, or stop it and rebuild the same logic from zero under a new clean
protocol?
What strict separation would you require between:
research data,
validation data,
sealed decisions,
execution records,
and capital accounting?
Which continuing weaknesses in the current system do you consider most
dangerous: data quality, decision timing, candidate selection, ranking,
allocation, holding rules, execution assumptions, or the capital ledger?
What should we test, remove, or simplify before trusting any new
prospective result?
In what order would you proceed so that AVCI becomes a simpler and
healthier system rather than a twentieth architecture?
Finally, are our provisional success threshold, random-control
construction, and ablation ladder sufficiently strict, or would you change
them before the freeze?
Nihat GÜNER ***@***.***>, 30 Tem 2026 Per, 16:22 tarihinde
şunu yazdı:
… Subject: Your guidance changed the direction of AVCI — thank you, and may
I ask for one further review?
Dear Haozhe,
I want to begin with my deepest and most sincere gratitude.
You are on the other side of the world, you do not know me personally, and
you had no obligation to spend your time understanding our project. Yet you
read our problem carefully and gave us a detailed, honest, and constructive
answer.
After approximately 22 months of work, 19 different architectures, and
thousands of hypotheses and simulations, repeated disappointment had made
it difficult for me to feel hopeful. While reading your response, I was
genuinely moved. Not because you promised success, but because you treated
our work seriously and showed us a defensible way to discover the truth.
Your kindness affected me deeply. I will always remember that someone far
away chose to help a stranger simply because he recognised a difficult
research problem. Thank you.
We have completed the retrospective part of your methodology and prepared,
but have not yet activated, the prospective stage. We did this without
creating architecture number twenty.
Everything below concerning historical results refers only to the C_SAFE30
validation path. It does not validate the full AVCI system.
What we did after your response
For the C_SAFE30 review, we re-downloaded official daily-session files and
rebuilt the complete 300-session sequence from clean source data.
The old published result of approximately +133.02% could not be reproduced
and was therefore rejected as evidence.
A real error was found in the old market comparison: on 12 days, stocks
with missing next-session results had been silently removed from the
comparison.
We then reconstructed each day’s eligible stock universe using only
information available at the decision time.
Using the existing frozen C_SAFE30 selection path:
Initial capital: TRY 10,000
Historical final capital: TRY 28,797.83
Historical growth: +187.98%
Maximum drawdown: −8.30%
Capital used per session: 25%
Assumed total cost: 0.70%
We created 100,000 random paths from the same daily eligible universe,
using the same capital, costs, holding rules and dates:
Random full-universe paths beating C_SAFE30: 0 / 100,000
Random paths selecting only among C_SAFE30’s daily Top 3 and beating
C_SAFE30: 3 / 100,000
We understand that this is only a strong historical signal—not independent
proof—because these 300 days were already exposed during the previous
research process.
We also tested the system as a ladder:
Selection signal without costs
Real 25% capital allocation
Holding and replacement rules
The normal 0.70% cost
A heavier 1.00% adverse execution stress
The historical advantage survived the cost increases. However, realistic
historical execution, partial fills, settlement delays and carried
positions cannot be reconstructed completely from the old records.
How AVCI currently works
The current project has 20 main modules and 23 supporting files. It
operates like a factory:
Delayed daily and one-minute BIST data are collected.
Timestamps, missing records and corporate actions are checked.
A point-in-time eligible stock universe is created.
Price, volume, market, sector and risk indicator families are calculated.
Our proprietary İz–Koku–DNA layers examine market traces, related movement
and historical similarities.
Previous winning patterns and similar historical DNA structures are used
as reinforcement information.
A broad candidate pool is formed.
Weak or inconsistent candidates are eliminated.
The remaining candidates are ranked and passed through risk, allocation,
holding and replacement controls.
The system produces a small final candidate list, generally Top 1–3.
The decision is sealed before the outcome is known.
Execution, costs, capital growth and random control paths are then
recorded separately.
What is still missing
We have prepared a new pre-registered prospective test with:
A frozen decision rule
100 same-universe random paths per day
TRY 10,000 as the real-capital path
TRY 30,000 and TRY 100,000 as capacity-stress scenarios, not live capital
deployments
Full-share quantities only
Partial-fill and unsold-position rules
No session may be removed after its outcome is known
Corporate-action accounting
File hashes and restart after any rule change
Predefined success and failure conditions
But the decisive evidence stage has not begun.
We still do not have:
An external public timestamp for the final freeze
A completed unseen prospective window
Real broker fill and fee records
Real execution evidence at the TRY 10,000 capital level
Execution-capacity evidence for the TRY 30,000 and TRY 100,000 stress
scenarios
We have pre-specified a provisional success threshold. At 300 official
sessions, the candidate must remain above its initial capital after all
real costs, avoid a drawdown of 20% or worse, and remain significantly
above its same-universe random paths. We would value your criticism of
whether this threshold is sufficiently strict.
We are therefore considering a clean-room rebuild of the implementation,
not a twentieth selection strategy. The purpose would be to preserve the
core AVCI logic while separating:
research data,
validation data,
prospective evidence,
execution records,
and final capital accounting
from the first day, so that the same historical contamination and ledger
problems cannot occur again.
May I ask for your opinion on four points?
Would you preserve C_SAFE30 as the frozen prospective candidate, or
rebuild the same logic in a clean implementation before starting?
Is our provisional success threshold against the random distribution
sufficiently strict?
Is our proposed ladder sufficient to identify where the advantage
disappears?
What minimum architecture and data-separation rules would you require
before trusting a new prospective test?
If you are comfortable with it, I would be deeply grateful to send you a
concise architecture diagram and pre-registration draft for a brief review.
I do not want to turn your kindness into an expectation of unpaid
consulting. Even a short warning about a serious methodological mistake
would be extremely valuable.
Your response did not give us another optimistic backtest. It gave us a
method for distinguishing a real advantage from a false one.
After 22 difficult months, that means more to me than I can fully express.
With my deepest gratitude,
After reviewing our records again in light of your message, I believe your
central diagnosis is correct:
Our historical data can no longer provide independent evidence.
The same periods were exposed to 19 architectures and thousands of
research decisions. Even when a later result appeared economically strong,
we could no longer know whether it represented a real effect or an
unusually favourable result selected from a very large search process.
We also do not have a complete, immutable economic ledger for the original
AVCI research chain. We cannot fully reconstruct every historical decision
together with real executable quantity, realised fills, all costs,
unresolved execution, corporate actions, and the complete daily capital
path.
Some promising results also came from separate research branches rather
than the complete AVCI production chain. Therefore, we cannot honestly
attribute every promising historical result to the full system.
We now accept that everything before the new freeze must be treated as
hypothesis generation, not proof.
Your proposed framework gives us a disciplined way to separate three
possibilities:
A persistent advantage may not exist.
A small selection advantage may exist but disappear during allocation,
holding, replacement, or risk controls.
An apparent paper advantage may disappear once realistic execution, traded
quantity, costs, corporate actions, and full capital accounting are applied.
Nihat Güner
Independent Software Engineer and AVCI Researcher
Türkiye
Haozhe Wu ***@***.***>, 30 Tem 2026 Per, 14:01 tarihinde
şunu yazdı:
> This is one of the most carefully framed questions we've had here, so let
> me answer the central one directly instead of proposing a twentieth
> architecture.
>
> *Scope note first, so I don't waste your time:* Vibe-Trading has no
> Borsa Istanbul engine. Our backtest engines cover A-shares, US/HK, India,
> Korea, crypto, futures and FX; BIST costs, circuit rules and lot
> conventions are not modelled. So read everything below as
> framework-independent methodology — it applies to whatever stack AVCI
> already runs on.
>
> *The short answer: no test run on your historical period can settle this
> any more.* You wrote that yourself, and it is the most important
> sentence in your post. After thousands of experiments and 19 architectures
> on the same history, that data is spent. Any further in-sample statistic —
> including an excellent one — is contaminated by the search that produced
> it. So the decisive experiment cannot be a backtest. It has to be
> prospective and pre-registered.
>
> Concretely, here is what I would pre-register.
>
> *1. Freeze and timestamp.* Before the window opens, publish a hash (a
> git commit plus a public timestamp) covering the decision code, every
> parameter, the universe definition, the decision time-of-day, the cost
> model, and the allocation / holding / replacement / exit rules. Nothing
> inside that hash may change during the test; if anything changes, the clock
> restarts. This is what makes the result immutable instead of negotiable
> afterwards.
>
> *2. Declare the verdict rule in advance — including the failure branch.*
> Before seeing any data, write down the horizon (I would want at least six
> months of trading sessions), the metric, the threshold that counts as
> success, the threshold that counts as failure, and the stopping rule. The
> failure branch matters most: pre-committing to "if after-cost return over N
> sessions is below X, we conclude there is no extractable edge" is the only
> thing that structurally prevents architecture #20
> <#20>.
>
> *3. Run a same-universe random control in parallel.* This is the
> experiment I would prioritise above all others. Every session, alongside
> AVCI's real decision, draw K random baskets from the same eligible universe
> on the same date, and push them through the identical allocation, holding,
> replacement, exit and cost machinery. Then ask one question: does AVCI's
> realised after-cost curve sit outside the distribution of its own random
> twins?
>
> That single control separates the three possibilities you say you cannot
> distinguish. If AVCI sits inside the random distribution, the selector
> contributes nothing and the historical result was market movement plus
> search luck. If it sits outside, you have a real selector — and only then
> is it worth asking where the edge leaks.
>
> (We implement exactly this in alpha bench --strict: a same-universe
> random control plus an out-of-sample gate. Different market, but the repo
> is MIT and the control's construction may be worth reading —
> agent/src/factors/bench_runner_strict.py.)
>
> *4. An ablation ladder, to localise where the edge dies — your
> possibility (2).* Hold one stage real and neutralise the rest, over the
> same dates: (a) real signal + equal weights + fixed holding period + zero
> costs; (b) add real allocation; (c) add real holding and replacement; (d)
> add real costs; (e) add realistic execution. The rung where the curve
> collapses is your answer. A system of 20 modules cannot be debugged as a
> whole, and this is the only way I know to localise the loss rather than
> guess at it.
>
> *5. Execution realism, sized to your actual capital — your possibility
> (3).* In a market with BIST's liquidity profile, the two assumptions
> that most often destroy a paper edge are fill price and participation. Cap
> each fill at a fixed fraction of that bar's traded volume, make slippage a
> function of that participation rather than a constant, then re-run the
> ladder at 1×, 3× and 10× your intended capital. If the edge survives only
> at sizes you would never actually trade, it is not an edge for you — and
> you get a capacity number, which you need anyway.
>
> *6. Adjust for the search itself.* You ran thousands of hypotheses, so
> your best result is by construction an extreme order statistic. The
> Deflated Sharpe Ratio and the probability of backtest overfitting (Bailey
> and López de Prado) exist precisely for this and take the number of trials
> as an explicit input. Reporting an undeflated Sharpe after 19 architectures
> is the most common way a dead system still looks alive.
>
> *On the missing ledger:* you mentioned you cannot reconstruct a complete
> real-money ledger for the older results. I would stop trying. Treat
> everything before the freeze as hypothesis generation rather than evidence,
> and let the pre-registered window be the only thing you defend. That
> reframing costs you nothing you actually had, and it makes the eventual
> result defensible either way.
>
> If you do run this, I would genuinely like to see the outcome posted here
> — including if it is negative. "22 months, 19 architectures, one
> pre-registered prospective test, no extractable edge after costs" is a more
> useful public artifact than most positive results, and making that kind of
> verdict routine is precisely what the --strict gate in this repo was
> built for.
>
> —
> Reply to this email directly, view it on GitHub
> <#906?email_source=notifications&email_token=B3NTTT264PKZ6TJUZC2E2ST5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17839330>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/B3NTTT3XN754IVCZ3ZRZRP35HMTHVAVCNFSNUABJKJSXA33TNF2G64TZHMYTCOJYGMYDMOBRGI5UI2LTMN2XG43JN5XDWMJQGUZDANBSHCQXMAQ>
> .
> Triage notifications, keep track of coding agent tasks and review pull
> requests on the go with GitHub Mobile for iOS
> <https://github.com/notifications/mobile/ios/B3NTTTY3H4TXHQ5Q3JX23535HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM>
> and Android
> <https://github.com/notifications/mobile/android/B3NTTT5C2UKBA4GUTBD4YPT5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>.
> Download it today!
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
0 replies
|
.
Nihat GÜNER ***@***.***>, 30 Tem 2026 Per, 16:22 tarihinde
şunu yazdı:
… Subject: Your guidance changed the direction of AVCI — thank you, and may
I ask for one further review?
Dear Haozhe,
I want to begin with my deepest and most sincere gratitude.
You are on the other side of the world, you do not know me personally, and
you had no obligation to spend your time understanding our project. Yet you
read our problem carefully and gave us a detailed, honest, and constructive
answer.
After approximately 22 months of work, 19 different architectures, and
thousands of hypotheses and simulations, repeated disappointment had made
it difficult for me to feel hopeful. While reading your response, I was
genuinely moved. Not because you promised success, but because you treated
our work seriously and showed us a defensible way to discover the truth.
Your kindness affected me deeply. I will always remember that someone far
away chose to help a stranger simply because he recognised a difficult
research problem. Thank you.
We have completed the retrospective part of your methodology and prepared,
but have not yet activated, the prospective stage. We did this without
creating architecture number twenty.
Everything below concerning historical results refers only to the C_SAFE30
validation path. It does not validate the full AVCI system.
What we did after your response
For the C_SAFE30 review, we re-downloaded official daily-session files and
rebuilt the complete 300-session sequence from clean source data.
The old published result of approximately +133.02% could not be reproduced
and was therefore rejected as evidence.
A real error was found in the old market comparison: on 12 days, stocks
with missing next-session results had been silently removed from the
comparison.
We then reconstructed each day’s eligible stock universe using only
information available at the decision time.
Using the existing frozen C_SAFE30 selection path:
Initial capital: TRY 10,000
Historical final capital: TRY 28,797.83
Historical growth: +187.98%
Maximum drawdown: −8.30%
Capital used per session: 25%
Assumed total cost: 0.70%
We created 100,000 random paths from the same daily eligible universe,
using the same capital, costs, holding rules and dates:
Random full-universe paths beating C_SAFE30: 0 / 100,000
Random paths selecting only among C_SAFE30’s daily Top 3 and beating
C_SAFE30: 3 / 100,000
We understand that this is only a strong historical signal—not independent
proof—because these 300 days were already exposed during the previous
research process.
We also tested the system as a ladder:
Selection signal without costs
Real 25% capital allocation
Holding and replacement rules
The normal 0.70% cost
A heavier 1.00% adverse execution stress
The historical advantage survived the cost increases. However, realistic
historical execution, partial fills, settlement delays and carried
positions cannot be reconstructed completely from the old records.
How AVCI currently works
The current project has 20 main modules and 23 supporting files. It
operates like a factory:
Delayed daily and one-minute BIST data are collected.
Timestamps, missing records and corporate actions are checked.
A point-in-time eligible stock universe is created.
Price, volume, market, sector and risk indicator families are calculated.
Our proprietary İz–Koku–DNA layers examine market traces, related movement
and historical similarities.
Previous winning patterns and similar historical DNA structures are used
as reinforcement information.
A broad candidate pool is formed.
Weak or inconsistent candidates are eliminated.
The remaining candidates are ranked and passed through risk, allocation,
holding and replacement controls.
The system produces a small final candidate list, generally Top 1–3.
The decision is sealed before the outcome is known.
Execution, costs, capital growth and random control paths are then
recorded separately.
What is still missing
We have prepared a new pre-registered prospective test with:
A frozen decision rule
100 same-universe random paths per day
TRY 10,000 as the real-capital path
TRY 30,000 and TRY 100,000 as capacity-stress scenarios, not live capital
deployments
Full-share quantities only
Partial-fill and unsold-position rules
No session may be removed after its outcome is known
Corporate-action accounting
File hashes and restart after any rule change
Predefined success and failure conditions
But the decisive evidence stage has not begun.
We still do not have:
An external public timestamp for the final freeze
A completed unseen prospective window
Real broker fill and fee records
Real execution evidence at the TRY 10,000 capital level
Execution-capacity evidence for the TRY 30,000 and TRY 100,000 stress
scenarios
We have pre-specified a provisional success threshold. At 300 official
sessions, the candidate must remain above its initial capital after all
real costs, avoid a drawdown of 20% or worse, and remain significantly
above its same-universe random paths. We would value your criticism of
whether this threshold is sufficiently strict.
We are therefore considering a clean-room rebuild of the implementation,
not a twentieth selection strategy. The purpose would be to preserve the
core AVCI logic while separating:
research data,
validation data,
prospective evidence,
execution records,
and final capital accounting
from the first day, so that the same historical contamination and ledger
problems cannot occur again.
May I ask for your opinion on four points?
Would you preserve C_SAFE30 as the frozen prospective candidate, or
rebuild the same logic in a clean implementation before starting?
Is our provisional success threshold against the random distribution
sufficiently strict?
Is our proposed ladder sufficient to identify where the advantage
disappears?
What minimum architecture and data-separation rules would you require
before trusting a new prospective test?
If you are comfortable with it, I would be deeply grateful to send you a
concise architecture diagram and pre-registration draft for a brief review.
I do not want to turn your kindness into an expectation of unpaid
consulting. Even a short warning about a serious methodological mistake
would be extremely valuable.
Your response did not give us another optimistic backtest. It gave us a
method for distinguishing a real advantage from a false one.
After 22 difficult months, that means more to me than I can fully express.
With my deepest gratitude,
After reviewing our records again in light of your message, I believe your
central diagnosis is correct:
Our historical data can no longer provide independent evidence.
The same periods were exposed to 19 architectures and thousands of
research decisions. Even when a later result appeared economically strong,
we could no longer know whether it represented a real effect or an
unusually favourable result selected from a very large search process.
We also do not have a complete, immutable economic ledger for the original
AVCI research chain. We cannot fully reconstruct every historical decision
together with real executable quantity, realised fills, all costs,
unresolved execution, corporate actions, and the complete daily capital
path.
Some promising results also came from separate research branches rather
than the complete AVCI production chain. Therefore, we cannot honestly
attribute every promising historical result to the full system.
We now accept that everything before the new freeze must be treated as
hypothesis generation, not proof.
Your proposed framework gives us a disciplined way to separate three
possibilities:
A persistent advantage may not exist.
A small selection advantage may exist but disappear during allocation,
holding, replacement, or risk controls.
An apparent paper advantage may disappear once realistic execution, traded
quantity, costs, corporate actions, and full capital accounting are applied.
Nihat Güner
Independent Software Engineer and AVCI Researcher
Türkiye
Haozhe Wu ***@***.***>, 30 Tem 2026 Per, 14:01 tarihinde
şunu yazdı:
> This is one of the most carefully framed questions we've had here, so let
> me answer the central one directly instead of proposing a twentieth
> architecture.
>
> *Scope note first, so I don't waste your time:* Vibe-Trading has no
> Borsa Istanbul engine. Our backtest engines cover A-shares, US/HK, India,
> Korea, crypto, futures and FX; BIST costs, circuit rules and lot
> conventions are not modelled. So read everything below as
> framework-independent methodology — it applies to whatever stack AVCI
> already runs on.
>
> *The short answer: no test run on your historical period can settle this
> any more.* You wrote that yourself, and it is the most important
> sentence in your post. After thousands of experiments and 19 architectures
> on the same history, that data is spent. Any further in-sample statistic —
> including an excellent one — is contaminated by the search that produced
> it. So the decisive experiment cannot be a backtest. It has to be
> prospective and pre-registered.
>
> Concretely, here is what I would pre-register.
>
> *1. Freeze and timestamp.* Before the window opens, publish a hash (a
> git commit plus a public timestamp) covering the decision code, every
> parameter, the universe definition, the decision time-of-day, the cost
> model, and the allocation / holding / replacement / exit rules. Nothing
> inside that hash may change during the test; if anything changes, the clock
> restarts. This is what makes the result immutable instead of negotiable
> afterwards.
>
> *2. Declare the verdict rule in advance — including the failure branch.*
> Before seeing any data, write down the horizon (I would want at least six
> months of trading sessions), the metric, the threshold that counts as
> success, the threshold that counts as failure, and the stopping rule. The
> failure branch matters most: pre-committing to "if after-cost return over N
> sessions is below X, we conclude there is no extractable edge" is the only
> thing that structurally prevents architecture #20
> <#20>.
>
> *3. Run a same-universe random control in parallel.* This is the
> experiment I would prioritise above all others. Every session, alongside
> AVCI's real decision, draw K random baskets from the same eligible universe
> on the same date, and push them through the identical allocation, holding,
> replacement, exit and cost machinery. Then ask one question: does AVCI's
> realised after-cost curve sit outside the distribution of its own random
> twins?
>
> That single control separates the three possibilities you say you cannot
> distinguish. If AVCI sits inside the random distribution, the selector
> contributes nothing and the historical result was market movement plus
> search luck. If it sits outside, you have a real selector — and only then
> is it worth asking where the edge leaks.
>
> (We implement exactly this in alpha bench --strict: a same-universe
> random control plus an out-of-sample gate. Different market, but the repo
> is MIT and the control's construction may be worth reading —
> agent/src/factors/bench_runner_strict.py.)
>
> *4. An ablation ladder, to localise where the edge dies — your
> possibility (2).* Hold one stage real and neutralise the rest, over the
> same dates: (a) real signal + equal weights + fixed holding period + zero
> costs; (b) add real allocation; (c) add real holding and replacement; (d)
> add real costs; (e) add realistic execution. The rung where the curve
> collapses is your answer. A system of 20 modules cannot be debugged as a
> whole, and this is the only way I know to localise the loss rather than
> guess at it.
>
> *5. Execution realism, sized to your actual capital — your possibility
> (3).* In a market with BIST's liquidity profile, the two assumptions
> that most often destroy a paper edge are fill price and participation. Cap
> each fill at a fixed fraction of that bar's traded volume, make slippage a
> function of that participation rather than a constant, then re-run the
> ladder at 1×, 3× and 10× your intended capital. If the edge survives only
> at sizes you would never actually trade, it is not an edge for you — and
> you get a capacity number, which you need anyway.
>
> *6. Adjust for the search itself.* You ran thousands of hypotheses, so
> your best result is by construction an extreme order statistic. The
> Deflated Sharpe Ratio and the probability of backtest overfitting (Bailey
> and López de Prado) exist precisely for this and take the number of trials
> as an explicit input. Reporting an undeflated Sharpe after 19 architectures
> is the most common way a dead system still looks alive.
>
> *On the missing ledger:* you mentioned you cannot reconstruct a complete
> real-money ledger for the older results. I would stop trying. Treat
> everything before the freeze as hypothesis generation rather than evidence,
> and let the pre-registered window be the only thing you defend. That
> reframing costs you nothing you actually had, and it makes the eventual
> result defensible either way.
>
> If you do run this, I would genuinely like to see the outcome posted here
> — including if it is negative. "22 months, 19 architectures, one
> pre-registered prospective test, no extractable edge after costs" is a more
> useful public artifact than most positive results, and making that kind of
> verdict routine is precisely what the --strict gate in this repo was
> built for.
>
> —
> Reply to this email directly, view it on GitHub
> <#906?email_source=notifications&email_token=B3NTTT264PKZ6TJUZC2E2ST5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17839330>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/B3NTTT3XN754IVCZ3ZRZRP35HMTHVAVCNFSNUABJKJSXA33TNF2G64TZHMYTCOJYGMYDMOBRGI5UI2LTMN2XG43JN5XDWMJQGUZDANBSHCQXMAQ>
> .
> Triage notifications, keep track of coding agent tasks and review pull
> requests on the go with GitHub Mobile for iOS
> <https://github.com/notifications/mobile/ios/B3NTTTY3H4TXHQ5Q3JX23535HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM>
> and Android
> <https://github.com/notifications/mobile/android/B3NTTT5C2UKBA4GUTBD4YPT5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>.
> Download it today!
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
0 replies
|
Hi Haozhe Wu,
Greetings from Turkey. How are you? I hope you are doing well. What have
you been working on today?
It’s 10:00 PM here, and I’ve honestly spent the whole day waiting for a
message from you.
Have you had a chance to look at my project? Do you think there is any
possibility you could help me? I would really appreciate your support — I’m
reaching out from the other side of the world.
Thank you very much.
Nihat GÜNER ***@***.***>, 30 Tem 2026 Per, 16:33 tarihinde
şunu yazdı:
… .
Nihat GÜNER ***@***.***>, 30 Tem 2026 Per, 16:22 tarihinde
şunu yazdı:
> Subject: Your guidance changed the direction of AVCI — thank you, and may
> I ask for one further review?
>
> Dear Haozhe,
>
> I want to begin with my deepest and most sincere gratitude.
>
> You are on the other side of the world, you do not know me personally,
> and you had no obligation to spend your time understanding our project. Yet
> you read our problem carefully and gave us a detailed, honest, and
> constructive answer.
>
> After approximately 22 months of work, 19 different architectures, and
> thousands of hypotheses and simulations, repeated disappointment had made
> it difficult for me to feel hopeful. While reading your response, I was
> genuinely moved. Not because you promised success, but because you treated
> our work seriously and showed us a defensible way to discover the truth.
>
> Your kindness affected me deeply. I will always remember that someone far
> away chose to help a stranger simply because he recognised a difficult
> research problem. Thank you.
>
> We have completed the retrospective part of your methodology and
> prepared, but have not yet activated, the prospective stage. We did this
> without creating architecture number twenty.
>
> Everything below concerning historical results refers only to the
> C_SAFE30 validation path. It does not validate the full AVCI system.
>
> What we did after your response
>
> For the C_SAFE30 review, we re-downloaded official daily-session files
> and rebuilt the complete 300-session sequence from clean source data.
>
> The old published result of approximately +133.02% could not be
> reproduced and was therefore rejected as evidence.
>
> A real error was found in the old market comparison: on 12 days, stocks
> with missing next-session results had been silently removed from the
> comparison.
>
> We then reconstructed each day’s eligible stock universe using only
> information available at the decision time.
>
> Using the existing frozen C_SAFE30 selection path:
>
> Initial capital: TRY 10,000
> Historical final capital: TRY 28,797.83
> Historical growth: +187.98%
> Maximum drawdown: −8.30%
> Capital used per session: 25%
> Assumed total cost: 0.70%
>
> We created 100,000 random paths from the same daily eligible universe,
> using the same capital, costs, holding rules and dates:
>
> Random full-universe paths beating C_SAFE30: 0 / 100,000
> Random paths selecting only among C_SAFE30’s daily Top 3 and beating
> C_SAFE30: 3 / 100,000
>
> We understand that this is only a strong historical signal—not
> independent proof—because these 300 days were already exposed during the
> previous research process.
>
> We also tested the system as a ladder:
>
> Selection signal without costs
> Real 25% capital allocation
> Holding and replacement rules
> The normal 0.70% cost
> A heavier 1.00% adverse execution stress
>
> The historical advantage survived the cost increases. However, realistic
> historical execution, partial fills, settlement delays and carried
> positions cannot be reconstructed completely from the old records.
>
> How AVCI currently works
>
> The current project has 20 main modules and 23 supporting files. It
> operates like a factory:
>
> Delayed daily and one-minute BIST data are collected.
> Timestamps, missing records and corporate actions are checked.
> A point-in-time eligible stock universe is created.
> Price, volume, market, sector and risk indicator families are calculated.
> Our proprietary İz–Koku–DNA layers examine market traces, related
> movement and historical similarities.
> Previous winning patterns and similar historical DNA structures are used
> as reinforcement information.
> A broad candidate pool is formed.
> Weak or inconsistent candidates are eliminated.
> The remaining candidates are ranked and passed through risk, allocation,
> holding and replacement controls.
> The system produces a small final candidate list, generally Top 1–3.
> The decision is sealed before the outcome is known.
> Execution, costs, capital growth and random control paths are then
> recorded separately.
>
> What is still missing
>
> We have prepared a new pre-registered prospective test with:
>
> A frozen decision rule
> 100 same-universe random paths per day
> TRY 10,000 as the real-capital path
> TRY 30,000 and TRY 100,000 as capacity-stress scenarios, not live capital
> deployments
> Full-share quantities only
> Partial-fill and unsold-position rules
> No session may be removed after its outcome is known
> Corporate-action accounting
> File hashes and restart after any rule change
> Predefined success and failure conditions
>
> But the decisive evidence stage has not begun.
>
> We still do not have:
>
> An external public timestamp for the final freeze
> A completed unseen prospective window
> Real broker fill and fee records
> Real execution evidence at the TRY 10,000 capital level
> Execution-capacity evidence for the TRY 30,000 and TRY 100,000 stress
> scenarios
>
> We have pre-specified a provisional success threshold. At 300 official
> sessions, the candidate must remain above its initial capital after all
> real costs, avoid a drawdown of 20% or worse, and remain significantly
> above its same-universe random paths. We would value your criticism of
> whether this threshold is sufficiently strict.
>
> We are therefore considering a clean-room rebuild of the implementation,
> not a twentieth selection strategy. The purpose would be to preserve the
> core AVCI logic while separating:
>
> research data,
> validation data,
> prospective evidence,
> execution records,
> and final capital accounting
>
> from the first day, so that the same historical contamination and ledger
> problems cannot occur again.
>
> May I ask for your opinion on four points?
>
> Would you preserve C_SAFE30 as the frozen prospective candidate, or
> rebuild the same logic in a clean implementation before starting?
>
> Is our provisional success threshold against the random distribution
> sufficiently strict?
>
> Is our proposed ladder sufficient to identify where the advantage
> disappears?
>
> What minimum architecture and data-separation rules would you require
> before trusting a new prospective test?
>
> If you are comfortable with it, I would be deeply grateful to send you a
> concise architecture diagram and pre-registration draft for a brief review.
> I do not want to turn your kindness into an expectation of unpaid
> consulting. Even a short warning about a serious methodological mistake
> would be extremely valuable.
>
> Your response did not give us another optimistic backtest. It gave us a
> method for distinguishing a real advantage from a false one.
>
> After 22 difficult months, that means more to me than I can fully express.
>
> With my deepest gratitude,
>
> After reviewing our records again in light of your message, I believe
> your central diagnosis is correct:
>
> Our historical data can no longer provide independent evidence.
>
> The same periods were exposed to 19 architectures and thousands of
> research decisions. Even when a later result appeared economically strong,
> we could no longer know whether it represented a real effect or an
> unusually favourable result selected from a very large search process.
>
> We also do not have a complete, immutable economic ledger for the
> original AVCI research chain. We cannot fully reconstruct every historical
> decision together with real executable quantity, realised fills, all costs,
> unresolved execution, corporate actions, and the complete daily capital
> path.
>
> Some promising results also came from separate research branches rather
> than the complete AVCI production chain. Therefore, we cannot honestly
> attribute every promising historical result to the full system.
>
> We now accept that everything before the new freeze must be treated as
> hypothesis generation, not proof.
>
> Your proposed framework gives us a disciplined way to separate three
> possibilities:
>
> A persistent advantage may not exist.
> A small selection advantage may exist but disappear during allocation,
> holding, replacement, or risk controls.
> An apparent paper advantage may disappear once realistic execution,
> traded quantity, costs, corporate actions, and full capital accounting are
> applied.
>
> Nihat Güner
> Independent Software Engineer and AVCI Researcher
> Türkiye
>
> Haozhe Wu ***@***.***>, 30 Tem 2026 Per, 14:01 tarihinde
> şunu yazdı:
>
>> This is one of the most carefully framed questions we've had here, so
>> let me answer the central one directly instead of proposing a twentieth
>> architecture.
>>
>> *Scope note first, so I don't waste your time:* Vibe-Trading has no
>> Borsa Istanbul engine. Our backtest engines cover A-shares, US/HK, India,
>> Korea, crypto, futures and FX; BIST costs, circuit rules and lot
>> conventions are not modelled. So read everything below as
>> framework-independent methodology — it applies to whatever stack AVCI
>> already runs on.
>>
>> *The short answer: no test run on your historical period can settle this
>> any more.* You wrote that yourself, and it is the most important
>> sentence in your post. After thousands of experiments and 19 architectures
>> on the same history, that data is spent. Any further in-sample statistic —
>> including an excellent one — is contaminated by the search that produced
>> it. So the decisive experiment cannot be a backtest. It has to be
>> prospective and pre-registered.
>>
>> Concretely, here is what I would pre-register.
>>
>> *1. Freeze and timestamp.* Before the window opens, publish a hash (a
>> git commit plus a public timestamp) covering the decision code, every
>> parameter, the universe definition, the decision time-of-day, the cost
>> model, and the allocation / holding / replacement / exit rules. Nothing
>> inside that hash may change during the test; if anything changes, the clock
>> restarts. This is what makes the result immutable instead of negotiable
>> afterwards.
>>
>> *2. Declare the verdict rule in advance — including the failure branch.*
>> Before seeing any data, write down the horizon (I would want at least six
>> months of trading sessions), the metric, the threshold that counts as
>> success, the threshold that counts as failure, and the stopping rule. The
>> failure branch matters most: pre-committing to "if after-cost return over N
>> sessions is below X, we conclude there is no extractable edge" is the only
>> thing that structurally prevents architecture #20
>> <#20>.
>>
>> *3. Run a same-universe random control in parallel.* This is the
>> experiment I would prioritise above all others. Every session, alongside
>> AVCI's real decision, draw K random baskets from the same eligible universe
>> on the same date, and push them through the identical allocation, holding,
>> replacement, exit and cost machinery. Then ask one question: does AVCI's
>> realised after-cost curve sit outside the distribution of its own random
>> twins?
>>
>> That single control separates the three possibilities you say you cannot
>> distinguish. If AVCI sits inside the random distribution, the selector
>> contributes nothing and the historical result was market movement plus
>> search luck. If it sits outside, you have a real selector — and only then
>> is it worth asking where the edge leaks.
>>
>> (We implement exactly this in alpha bench --strict: a same-universe
>> random control plus an out-of-sample gate. Different market, but the repo
>> is MIT and the control's construction may be worth reading —
>> agent/src/factors/bench_runner_strict.py.)
>>
>> *4. An ablation ladder, to localise where the edge dies — your
>> possibility (2).* Hold one stage real and neutralise the rest, over the
>> same dates: (a) real signal + equal weights + fixed holding period + zero
>> costs; (b) add real allocation; (c) add real holding and replacement; (d)
>> add real costs; (e) add realistic execution. The rung where the curve
>> collapses is your answer. A system of 20 modules cannot be debugged as a
>> whole, and this is the only way I know to localise the loss rather than
>> guess at it.
>>
>> *5. Execution realism, sized to your actual capital — your possibility
>> (3).* In a market with BIST's liquidity profile, the two assumptions
>> that most often destroy a paper edge are fill price and participation. Cap
>> each fill at a fixed fraction of that bar's traded volume, make slippage a
>> function of that participation rather than a constant, then re-run the
>> ladder at 1×, 3× and 10× your intended capital. If the edge survives only
>> at sizes you would never actually trade, it is not an edge for you — and
>> you get a capacity number, which you need anyway.
>>
>> *6. Adjust for the search itself.* You ran thousands of hypotheses, so
>> your best result is by construction an extreme order statistic. The
>> Deflated Sharpe Ratio and the probability of backtest overfitting (Bailey
>> and López de Prado) exist precisely for this and take the number of trials
>> as an explicit input. Reporting an undeflated Sharpe after 19 architectures
>> is the most common way a dead system still looks alive.
>>
>> *On the missing ledger:* you mentioned you cannot reconstruct a
>> complete real-money ledger for the older results. I would stop trying.
>> Treat everything before the freeze as hypothesis generation rather than
>> evidence, and let the pre-registered window be the only thing you defend.
>> That reframing costs you nothing you actually had, and it makes the
>> eventual result defensible either way.
>>
>> If you do run this, I would genuinely like to see the outcome posted
>> here — including if it is negative. "22 months, 19 architectures, one
>> pre-registered prospective test, no extractable edge after costs" is a more
>> useful public artifact than most positive results, and making that kind of
>> verdict routine is precisely what the --strict gate in this repo was
>> built for.
>>
>> —
>> Reply to this email directly, view it on GitHub
>> <#906?email_source=notifications&email_token=B3NTTT264PKZ6TJUZC2E2ST5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17839330>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/B3NTTT3XN754IVCZ3ZRZRP35HMTHVAVCNFSNUABJKJSXA33TNF2G64TZHMYTCOJYGMYDMOBRGI5UI2LTMN2XG43JN5XDWMJQGUZDANBSHCQXMAQ>
>> .
>> Triage notifications, keep track of coding agent tasks and review pull
>> requests on the go with GitHub Mobile for iOS
>> <https://github.com/notifications/mobile/ios/B3NTTTY3H4TXHQ5Q3JX23535HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM>
>> and Android
>> <https://github.com/notifications/mobile/android/B3NTTT5C2UKBA4GUTBD4YPT5HMTHVA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYGM4TGMZQUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>.
>> Download it today!
>> You are receiving this because you authored the thread.Message ID:
>> ***@***.***>
>>
>
|
0 replies
|
Nihat — apologies for the delay, and thank you for the kind words. Let me answer the architecture If I were rebuilding AVCI from zero, the smallest architecture I'd trust is roughly this:
That's four components. Everything else is analysis you can rebuild on top at any time. On what to preserve: I would not carry C_SAFE30 forward as a frozen rule into a clean rebuild. On which weakness is most dangerous: ranked, for a system of this shape — decision timing On order of work: freeze the four components above, run the prospective test on the existing On the ablation ladder and random control: those are the right instruments. Your threshold Now the honest part: I can't take on an ongoing review role for AVCI. It's a single-maintainer I hope this is useful, and I genuinely wish you luck with it. |
0 replies
|
Dear Guy,
Thank you once again for your thoughtful reply. Your advice has been incredibly valuable, and we truly appreciate the time you took to help us.
However, despite all our work, we are still stuck on one fundamental problem.
We cannot find a way to build a consistently profitable strategy using only delayed (historical) data.
Do you have any suggestions for a different direction we should explore?
Or, if you were in our position, how would you design a profitable strategy using only delayed public data?
This is the part we simply cannot solve. We have spent many months testing countless ideas, architectures, and hypotheses, but we still haven't found a reliable source of predictive information that can generate consistent profits after transaction costs.
Any advice or guidance you could share would mean a great deal to us.
Thank you again for your kindness, your experience, and your willingness to help.
With sincere gratitude,
**Nihat Güner**
Karabük, Türkiye
iOS için Outlook<https://aka.ms/o0ukef> uygulamasını edinin
…________________________________
Gönderen: Haozhe Wu ***@***.***>
Gönderildi: Monday, 03 August 2026 16:52:39
Kime: HKUDS/Vibe-Trading ***@***.***>
Bilgi: sedatguner2000-coder ***@***.***>; Author ***@***.***>
Konu: Re: [HKUDS/Vibe-Trading] We built 19 Borsa Istanbul research systems but still cannot prove a real trading edge — what are we missing? (Discussion #906)
Nihat — apologies for the delay, and thank you for the kind words. Let me answer the architecture
question directly, and then be honest about what I can and can't offer, because I'd rather you plan
around the real thing than wait on me.
If I were rebuilding AVCI from zero, the smallest architecture I'd trust is roughly this:
1. One data store with an available_at column on every row, and one query function that takes
an as_of timestamp and cannot return anything stamped later. Not a convention — a function
signature that makes the leak impossible to write. Almost every "edge that evaporates" I've seen
traces back to something knowable-later being visible earlier, and it hides beautifully inside 19
architectures.
2. A sealed decision log. Decisions written before the outcome window opens, hashed, append-only.
If a decision can be edited after the fact, no downstream result means anything — and this is the
one component you cannot retrofit onto old results, only start fresh.
3. An execution record kept separate from the decision log, with its own fills, fees, and
slippage. Never infer execution from the decision.
4. A capital ledger derived from the execution record only — never from the research pipeline.
That's four components. Everything else is analysis you can rebuild on top at any time.
On what to preserve: I would not carry C_SAFE30 forward as a frozen rule into a clean rebuild.
Not because the rule is wrong, but because it was selected on the same history you'd test it on, and
you have no way left to distinguish "this rule is good" from "this rule was chosen." Rebuild the
same logic from zero under the new protocol and let it re-earn its place — if it's real, that costs
you a few months; if it isn't, it saves you architecture number twenty.
On which weakness is most dangerous: ranked, for a system of this shape — decision timing
(leakage) first, by a wide margin; candidate selection second (survivorship and selection-on-history
enter here); execution assumptions third; capital accounting fourth. Data quality matters, but it's
rarely what kills a result that looked good in-sample.
On order of work: freeze the four components above, run the prospective test on the existing
rule for a fixed pre-committed window, and change nothing during it. The result of that single test
tells you more than the twentieth architecture will.
On the ablation ladder and random control: those are the right instruments. Your threshold
should be pre-registered and never adjusted after seeing results — if the test doesn't clear the
bar, the answer is "more observations", never "a lower bar."
Now the honest part: I can't take on an ongoing review role for AVCI. It's a single-maintainer
project here and I owe the queue on this repo my time — so please don't hold your evenings waiting
on a reply from me. What I can do is answer specific, self-contained methodology questions in this
thread when they come up, without any promised turnaround. If you post the results of that
prospective test when it completes, I'll read them with interest.
I hope this is useful, and I genuinely wish you luck with it.
—
Reply to this email directly, view it on GitHub<#906?email_source=notifications&email_token=B3NTTTYKBHJ7JC32PHAPK5L5ICKKPA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYHAYTKNRTUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17881563>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/B3NTTT7ZWR74DLF47A3DXUL5ICKKPAVCNFSNUABJKJSXA33TNF2G64TZHMYTCOJYGMYDMOBRGI5UI2LTMN2XG43JN5XDWMJQGUZDANBSHCQXMAQ>.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS<https://github.com/notifications/mobile/ios/B3NTTT4XXODTNKLOIGIEGYT5ICKKPA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYHAYTKNRTUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM> and Android<https://github.com/notifications/mobile/android/B3NTTT7NXHXJJ4CZMYVK2UT5ICKKPA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZYHAYTKNRTUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>. Download it today!
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
0 replies
|
Nihat — thank you for asking it this plainly. I don't have a comfortable answer, but I'd rather give you an honest one than a hopeful one. I don't think "consistently profitable" is something delayed public data yields on demand. With end-of-day public data on a liquid market, you're working from the same inputs as everyone else, a little later than everyone else. That doesn't make an edge impossible — but it does mean the edge can't come from seeing something first. It has to come from somewhere else, and in my experience there are only a few honest candidates:
What isn't on that list is a better signal computed from the same daily bars — and I suspect that's close to what the nineteen architectures have been reaching for. On where that leaves you. If a pre-registered prospective test on a frozen architecture doesn't clear the bar, I don't think the right conclusion is "try a twentieth architecture." I think it's that no edge of the size you need is available in this particular data, and the real choice then is between changing the input — order flow, fundamentals at a horizon nobody trades, something local to Turkish markets that you can reach and outsiders can't — or accepting that the market is doing its job here and indexing. I want to be careful not to make that sound like a consolation prize. Arriving at it deliberately, with a clean test behind it, is a genuinely better place to be than most people who search this carefully ever reach — the more common ending is architectures twenty through forty. One cheap thing worth checking first, because it could change everything above: what does your best strategy look like after Borsa İstanbul transaction costs, at the position sizes you'd actually trade, on the small caps where the effect is strongest? A good share of "no edge after costs" results turn out to be "the edge is real but sits entirely below our cost floor" — and that's a different problem, with different solutions (fewer, larger, longer-held positions), than "there's no signal here." And just so you can plan around it: I'm glad to answer specific methodology questions here whenever they come up, but I can't take on an ongoing review role — so please don't let anything wait on a reply from me. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We have been developing an independent decision-support system for Borsa Istanbul for approximately 22 months.
The system uses only delayed market information that was available at the exact decision time. It does not use future data, real-time private feeds, other stock markets, or automated order execution. It does not manage client money. Its purpose is to examine the market, identify candidates, compare their relative strength and risk, and produce a manual daily decision.
How the system works
Over time, we built and tested 19 different research architectures. These were not simple parameter changes; they examined different combinations of price behaviour, intraday movement, market and sector conditions, historical similarities, candidate ranking, downside protection, capital allocation, holding periods, and exit decisions.
The current AVCI architecture contains:
Historical daily and real one-minute BIST price and trading data have also been examined. Information created after the decision time is not supposed to enter the decision process.
The unresolved problem
Despite thousands of hypotheses, simulations, tests, and several complete architecture changes, we have not been able to prove a repeatable and executable after-cost edge.
Promising historical results often weaken or disappear when:
A further problem is that much of the historical period has already been examined during research. After thousands of experiments, even an apparently excellent historical result may simply be a false discovery caused by overfitting and repeated testing.
There are also unresolved differences between a paper result and actual capital growth. A correct candidate does not automatically mean a profitable trade. Entry price, liquidity, slippage, tradable quantity, transaction costs, corporate actions, holding time, and exit timing can all change the result.
Our historical records also do not provide a complete real-money ledger containing every decision, executed quantity, entry, exit, cost, and daily capital change. For this reason, some old results cannot be reconstructed as genuine executable performance.
At present, we cannot confidently distinguish between three possibilities:
What we are looking for
We are not looking for:
We are looking for experienced researchers, graduate students, quantitative developers, market-microstructure specialists, or independent practitioners who are willing to examine this problem carefully and patiently.
The central question is:
We are especially interested in people with experience in:
A negative conclusion is acceptable. The objective is not to make an unsuccessful system appear successful. The objective is to determine, with defensible evidence, whether a real edge exists, where it disappears, or why it cannot be extracted under the current constraints.
This is not a quick question that can be solved with one indicator or a few comments. We are looking for serious contributors who are willing to understand the architecture and help define a small number of decisive experiments.
A concise anonymized technical summary can first be shared with serious contributors. Proprietary selection rules, the complete source code, and raw data that we do not have the right to redistribute will not be posted publicly.
Our core question is:
Why, although this large research infrastructure appears able to identify strong candidates, can we not convert that ability into repeatable, executable, after-cost capital growth across different market periods?
All reactions