Repository navigation
💡 RFC: Causal Reinforcement Learning & Reward Function Design in DecisionIntent #183
NeoNix-Lab
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Context & Objective
In Quant Platform, all trading decisions flow through the learner-agnostic
DecisionIntentinterface (ADR-0009). This decouples mathematical decision logic from execution mechanics and ledger accounting.With Wave 5 focusing on Experimentation & Supervised Learning, a natural research question emerges: How do we design robust, non-overfitting Reinforcement Learning (RL) agents that produce viable
DecisionIntentactions?Key Challenges in Financial Reinforcement Learning
Proposed Framework
1. Reward Function Formulation
Instead of raw PnL, we propose testing risk-adjusted, cost-penalized differential rewards:
$$\mathcal{R}_t = \Delta \text{PnL}_t - \lambda_1 \cdot \text{FeeExpense}_t - \lambda_2 \cdot \text{SlippagePenalty}_t - \lambda_3 \cdot \text{DrawdownPenalty}_t$$
Or differential Sharpe Ratio / Sortino Ratio reward shaping.
2. Action Space & Risk Budgets
Rather than direct unconstrained order sizing, the agent outputs normalized directional intensity$a_t \in [-1, 1]$ , which is conditioned by our frozen risk gates ($G03$ Position Sizing ceilings and $G04$ Post-Loss Cooldown).
3. Causal Verification & Anti-Overfitting
Evaluating trained policies exclusively through:
Open Questions for Discussion
ReplayEnginebe of interest to the community?Share your insights, papers, or preferred RL frameworks below!
All reactions