Black Trader

Risk-first bootstrap for an automated crypto trading system.

This starter implementation focuses on one critical recommendation from the feasibility review: never promote to larger capital unless objective quality gates pass.

What is implemented

• Gate evaluator CLI (black-readiness) for Gate 0-4 decisions.
• Policy-driven thresholds in JSON (configs/risk_policy.json).
• Example metrics input (data/sample_metrics.json).
• Freqtrade DB ingestion CLI (black-ingest) to generate real readiness metrics from tradesv3.sqlite.
• Backtest ingestion CLI (black-backtest-ingest) to populate Gate 1 from backtest exports.
• Trade summary CLI (black-trades) for realized PnL / PF / drawdown metrics.
• Daily loss-limit CLI (black-monitor) for stop-trading decisions.
• Exchange abstraction + spot execution simulator (black-sim).
• Strategy lifecycle controller (black-lifecycle) for start/stop/stopentry/forceexit/status/health actions.
• Multi-channel notifier (black-notify) for Telegram/Slack alerts.
• Dev runtime bootstrap CLI (black-dev-runtime) to create host-local configs and seeded dry-run DB.
• Gate 3 portfolio risk controls for multi-symbol exposure and correlation caps.
• Unattended verification loop script until a target clock time.
• Unit tests for readiness, ingestion, simulation, lifecycle, notifier, and monitoring.

Project layout

• src/black_trader/readiness.py: gate logic + report output
• src/black_trader/freqtrade_ingest.py: Freqtrade SQLite -> readiness metrics
• src/black_trader/backtest_ingest.py: backtest JSON -> Gate 1 metrics
• src/black_trader/exchange_simulator.py: exchange abstraction + spot order simulator
• src/black_trader/trades.py: trade summary utilities + CLI
• src/black_trader/monitoring.py: daily risk checks + CLI
• src/black_trader/lifecycle.py: strategy lifecycle command/API controller
• src/black_trader/notify.py: Telegram/Slack notifier fan-out
• src/black_trader/dev_runtime.py: local runtime bootstrap for dev host paths/configs/DB
• user_data/strategies/RAPv1Strategy.py: recommended production candidate strategy (spot, long-only)
• configs/risk_policy.json: thresholds and constraints
• configs/metrics_context_example.json: operator/security/risk context values merged with DB-derived metrics
• configs/lifecycle_config_example.json: lifecycle controller config template
• configs/notifier_config_example.json: notifier channel config template
• data/sample_metrics.json: example metrics payload
• data/sample_trades.json: example closed/open trades
• data/sample_backtest_result.json: example backtest export payload
• data/sample_simulation_scenario.json: example simulation scenario
• tests/test_backtest_ingest.py: backtest ingestion tests
• tests/test_dev_runtime.py: dev runtime bootstrap tests
• tests/test_exchange_simulator.py: simulation tests
• tests/test_lifecycle.py: lifecycle tests
• tests/test_notify.py: notifier tests
• tests/test_readiness.py: readiness tests
• tests/test_trades_monitoring.py: trade and monitoring tests
• scripts/run_full_readiness_pipeline.sh: db ingest + backtest ingest + readiness one-shot pipeline
• scripts/setup_dev_runtime.sh: initialize runtime/freqtrade for dev usage
• scripts/run_walkforward_backtests.sh: run RAPv1 walk-forward backtests with exported artifacts
• scripts/run_strategy_bias_checks.sh: run lookahead + recursive bias checks with archived logs/results
• scripts/run_stress_tag_analysis.sh: run fee-stress walk-forward backtests + entry/exit tag analysis + stress gate
• scripts/run_holdout_validation.sh: run untouched holdout backtest + gate report for final confirmation
• scripts/run_strategy_release_gate.sh: run full promotion gate (walk-forward + bias + stress + holdout) with one pass/fail result
• scripts/run_trend_regression_checks.sh: run trend regression checks against release gate artifacts
• scripts/update_gate3_context.sh: refresh Gate 3 exposure/correlation metrics context from runtime config/db/candles
• .github/workflows/release-gate.yml: PR workflow running quality checks + full release gate with artifact upload
• .github/workflows/nightly-release-gate.yml: scheduled nightly strict release gate + trend regression checks
• scripts/improvement_loop_until_2am.sh: unattended verification loop
• docs/IMPLEMENTATION_NOTES.md: scope and next engineering steps
• docs/REAL_STRATEGY_PLAN.md: recommended real strategy blueprint (RAP-v1)

Quick start

cd /home/tommy/dev/Black
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
./scripts/setup_dev_runtime.sh
black-readiness --policy configs/risk_policy.json --metrics data/sample_metrics.json
black-trades --trades data/sample_trades.json --starting-balance 1000
black-monitor --trades data/sample_trades.json --day 2026-03-04 --day-start-balance 1000 --max-daily-loss-pct 3
black-sim --scenario data/sample_simulation_scenario.json
black-backtest-ingest --backtest-json data/sample_backtest_result.json --metrics-out /tmp/backtest_metrics.json --fee-slippage-included
black-readiness --policy configs/risk_policy.json --metrics data/sample_metrics_nogo.json --notify-config configs/notifier_config_example.json --notify-on nogo

Initialize host-local dev runtime paths (idempotent):

./scripts/setup_dev_runtime.sh

This creates:

• /home/tommy/dev/Black/runtime/freqtrade/config.dryrun.json
• /home/tommy/dev/Black/runtime/freqtrade/config.live.json
• /home/tommy/dev/Black/runtime/freqtrade/tradesv3.dryrun.sqlite
• /home/tommy/dev/Black/runtime/freqtrade/tradesv3.readiness_seed.sqlite
• /home/tommy/dev/Black/runtime/freqtrade/tradesv3.live.sqlite
• /home/tommy/dev/Black/runtime/freqtrade/lifecycle_config.json
• /home/tommy/dev/Black/runtime/freqtrade/metrics_context.dev.json

Default strategy in generated runtime configs: RAPv1Strategy.

Generate metrics from a Freqtrade DB and run readiness:

./scripts/build_metrics_from_freqtrade_db.sh
black-readiness --policy configs/risk_policy.json --metrics data/metrics_from_freqtrade_db.json

Refresh Gate 3 context from runtime config, open trades, and local candle data:

./scripts/update_gate3_context.sh

For Gate 3 portfolio caps, provide gate3.symbol_exposure_usdt and gate3.symbol_correlation_matrix in your context JSON. If these are missing, readiness applies conservative fallback assumptions.

Run full pipeline (DB ingest + backtest ingest + readiness):

./scripts/run_full_readiness_pipeline.sh /path/to/tradesv3.sqlite /path/to/backtest_result.json

If runtime/freqtrade/tradesv3.readiness_seed.sqlite exists, the pipeline script uses it by default.

Run walk-forward backtests for RAPv1Strategy:

./scripts/run_walkforward_backtests.sh

Results are written under user_data/backtest_results/walkforward/<timestamp>/.

Run anti-bias checks for RAPv1Strategy:

./scripts/run_strategy_bias_checks.sh

Optional compact window / thresholds:

./scripts/run_strategy_bias_checks.sh runtime/freqtrade/config.dryrun.json RAPv1Strategy 20250301-20250630 user_data/bias_checks 20 60

Results are written under user_data/bias_checks/<timestamp>/, including:

• lookahead_analysis.csv
• lookahead.log
• recursive.log
• bias_checks_summary.json

Run fee/slippage stress + tag analysis:

./scripts/run_stress_tag_analysis.sh

Optional gate tuning:

THRESHOLD_25_PROFIT_PCT_DROP=1.2 THRESHOLD_50_PROFIT_PCT_DROP=2.5 ./scripts/run_stress_tag_analysis.sh

Results are written under user_data/backtest_results/stress_tags/<timestamp>/, including:

• stress_tag_summary.json
• stress_tag_report.md

Run holdout validation (untouched confirmation window):

./scripts/run_holdout_validation.sh

Optional holdout gate tuning:

HOLDOUT_MIN_TRADES=3 HOLDOUT_MIN_PROFIT_PCT=0.0 HOLDOUT_MIN_PROFIT_FACTOR=1.0 HOLDOUT_MAX_DRAWDOWN=0.02 ./scripts/run_holdout_validation.sh runtime/freqtrade/config.dryrun.json RAPv1Strategy 20260301-20260305

Results are written under user_data/backtest_results/holdout/<timestamp>/, including:

• holdout_summary.json
• holdout_report.md

Run full strategy release gate (all stages + final decision):

./scripts/run_strategy_release_gate.sh

Optional holdout override inside release gate:

HOLDOUT_MIN_PROFIT_FACTOR=0.95 ./scripts/run_strategy_release_gate.sh runtime/freqtrade/config.dryrun.json RAPv1Strategy user_data/backtest_results/release_gate 20260301-20260305

Results are written under user_data/backtest_results/release_gate/<timestamp>/, including:

• release_gate_summary.json
• release_gate_report.md

Run trend regression checks from the latest release gate:

./scripts/run_trend_regression_checks.sh

Run trend regression checks against a specific release summary:

./scripts/run_trend_regression_checks.sh user_data/backtest_results/release_gate/<timestamp>/release_gate_summary.json

Results are written under user_data/backtest_results/trend_regression/<timestamp>/, including:

• trend_regression_summary.json
• trend_regression_report.md

Nightly CI:

• PR checks: .github/workflows/release-gate.yml
• 02:00 JST scheduled strict gate + regression trend checks: .github/workflows/nightly-release-gate.yml

Lifecycle control examples:

black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action start-dry-run
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action start-dry-run --detach --log-file logs/freqtrade_dryrun.log
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action start --api-url http://127.0.0.1:8080 --api-user freqtrader --api-pass '***'
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action stopentry --api-url http://127.0.0.1:8080 --api-user freqtrader --api-pass '***'
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action forceexit-all --api-url http://127.0.0.1:8080 --api-user freqtrader --api-pass '***'
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action status --api-url http://127.0.0.1:8080 --api-user freqtrader --api-pass '***'
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action health --api-url http://127.0.0.1:8080 --api-user freqtrader --api-pass '***'
black-lifecycle --config runtime/freqtrade/lifecycle_config.json --action health --api-url http://127.0.0.1:8080 --api-user freqtrader --api-pass '***' --notify-config configs/notifier_config_example.json --notify-on fail

Notifier example:

black-notify --config configs/notifier_config_example.json --title \"Risk Alert\" --body \"Daily loss limit reached\" --severity critical

Expected output

The commands print:

• Each gate status (PASS/FAIL)
• Reasons for failure
• Final recommendation (GO / NO-GO)
• DB-ingested gate metrics file compatible with black-readiness
• Backtest-ingested Gate 1 metrics merged into pipeline metrics
• Trade KPIs (trade count, win rate, PF, max drawdown)
• Daily loss status and action (CONTINUE / STOP_TRADING)
• Gate 3 exposure/correlation cap status for portfolio concentration control
• Spot execution simulation result for scenario replay
• Lifecycle action result payload for operational hooks
• Notification send result per channel

Notification hooks:

• black-readiness: supports --notify-config with --notify-on {nogo,warnings,always}
• black-monitor: supports --notify-config with --notify-on {breach,always}
• black-lifecycle: supports --notify-config with --notify-on {fail,always}

Autonomous Loop (Until 02:00 JST)

Run unattended verification cycles until 02:00 on 2026-03-05:

cd /home/tommy/dev/Black
systemd-run --user --unit=black-improvement-loop \
  /bin/bash -lc 'cd /home/tommy/dev/Black && ./scripts/improvement_loop_until_2am.sh'

Optional overrides:

END_TIME_LOCAL="2026-03-05 02:00:00 JST" INTERVAL_SECS=120 ./scripts/improvement_loop_until_2am.sh

Check active loop status and latest log:

./scripts/check_loop_status.sh

Next steps

1. Define next strategy iteration objective (parameter search, market regime expansion, or live rollout controls).