Skip to content

Fix initialization, filter, and timestamp-contract issues (#99-#103)#104

Merged
dceoy merged 7 commits into
mainfrom
fix-issues-99-103
Jul 7, 2026
Merged

Fix initialization, filter, and timestamp-contract issues (#99-#103)#104
dceoy merged 7 commits into
mainfrom
fix-issues-99-103

Conversation

@dceoy

@dceoy dceoy commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes five issues found while debugging downstream live-trading consumers:

Intentional behavior changes

  • with Mt5Client(...) and auto-initialization now raise Mt5RuntimeError on a dead terminal/connection instead of continuing silently; *_total methods raise instead of returning None.
  • Conflicting filter combinations raise ValueError (the logging decorator lets ValueError propagate unwrapped); the history symbol filter is now an exact match.

Release

  • Version bumped to 1.1.1

QA

  • .agents/skills/local-qa/scripts/qa.sh — all checks passed
  • uv run pytest — 421 passed, 33 skipped (Windows-only), 100% branch coverage
  • uv run ruff check . / uv run pyright — clean
  • uv run mkdocs build — succeeds

Windows/MT5 note: all changes are exercised via the existing mock-based test suite; no live-terminal behavior was verified (CI runs on Windows).

Closes #99
Closes #100
Closes #101
Closes #102
Closes #103

🤖 Generated with Claude Code

agent and others added 5 commits July 7, 2026 09:26
Capture last_error() immediately after a failed initialize() or login()
and embed that value in the raised Mt5RuntimeError, instead of querying
the terminal after shutdown() has already closed the IPC connection.

Closes #100

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mt5Client.__enter__() and _initialize_if_needed() previously discarded
the boolean result of initialize(), so a dead terminal or connection
"succeeded" silently. Both now raise Mt5RuntimeError with last_error()
details via a shared _initialize_or_raise() helper, matching
Mt5DataClient.__enter__() behavior.

The five *_total methods also validated nothing and passed None through
despite their -> int annotations; they now use
_validate_mt5_response_is_not_none like the other getters.

Closes #101

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The filterable getters silently prioritized mutually exclusive filters
(ticket > group > symbol; ticket/position over date range), returning
data filtered differently from what the caller asked for. orders_get,
positions_get, history_orders_get, and history_deals_get now raise
ValueError on conflicting combinations, and the logging decorator lets
ValueError propagate instead of wrapping it in Mt5RuntimeError.

The history symbol filter also matched substrings via the *{symbol}*
group wildcard (returning broker-suffixed variants such as EURUSD.m)
and silently discarded a caller-supplied group. history_*_get_as_dicts
now reject symbol combined with group and post-filter the returned rows
on exact symbol equality.

Closes #102

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MT5 epoch timestamps are trade-server wall-clock labels (typically
UTC+2/UTC+3), and pdmt5 converts them to timezone-naive values that
preserve those labels. Nothing stated this, so downstream consumers
treated the converted datetimes as UTC and built live-trading logic on
a 2-3 hour bias.

Add a "Timestamps and Timezones" section to README.md, docs/index.md,
and the conversion rules in docs/api/utils.md, plus one-line notes on
detect_and_convert_time_to_datetime and the date_from/date_to
parameters of the history and copy_* accessors in both layers.

Closes #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- docs/api/utils.md showed @detect_and_convert_time_to_datetime as the
  outer decorator, the opposite of every call site; with that order the
  raw epoch column would be moved into the index before conversion.
  Show the real order and explain why set_index_if_possible stays
  outermost. Also correct the "conversions only happen when requested"
  claim (conversion is on by default, opt-out via skip_to_datetime).
- Remove redundant client.initialize()/client.login() calls inside
  `with` blocks across docs/api/mt5.md, docs/index.md,
  docs/api/index.md, and docs/api/dataframe.md; the context managers
  already initialize (and Mt5DataClient logs in via config). The
  removed client.login(config.login, ...) calls also failed pyright
  strict since config.login is int | None.
- Update mt5.md Best Practices and Error Handling for the context
  manager raising Mt5RuntimeError on failure, and fix the Logging
  section to reference INFO level (the wrapper logs at INFO/WARNING,
  not DEBUG).

Closes #103

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8fd232a9b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread README.md

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Review tooling noise — superseded by the final review below.]

Comment thread tests/test_dataframe.py

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Review tooling noise — superseded by the final review below.]

Comment thread pdmt5/mt5.py
Comment thread pdmt5/mt5.py
Comment thread pdmt5/dataframe.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Review tooling noise — superseded by the final review below.]

Comment thread tests/test_dataframe.py
Comment thread pdmt5/utils.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed code quality, performance, tests, documentation, security, types, and silent-failure concerns. Overall this is a solid, focused set of fixes. A few important issues to address before merging:

Implementation issues

  • pdmt5/mt5.py:73_log_mt5_last_status_code still wraps internally-raised Mt5RuntimeError into a generic message (see inline comment).
  • pdmt5/mt5.py:1073_validate_history_filters allows group without a date range, contradicting its docstring.
  • pdmt5/dataframe.py:1126_validate_history_input does not reject symbol/group combined with ticket/position, producing a confusing downstream error.

Documentation

  • docs/api/mt5.md:221 — the claim that all methods raise Mt5RuntimeError is too broad.
  • Many pdmt5/mt5.py docstrings still say "or None" for methods that now raise on None: account_info, terminal_info, symbols_get, symbol_info, symbol_info_tick, market_book_get, copy_rates_from, copy_rates_from_pos, copy_rates_range, copy_ticks_from, copy_ticks_range, orders_get, order_calc_margin, order_calc_profit, order_check, order_send, positions_get, history_orders_get, history_deals_get.
  • docs/api/utils.md:66 — the millisecond rule says any field ending in _msc is converted, but the code only converts keys that start with time_ and end with _msc.

Test gaps

  • tests/test_dataframe.py lacks coverage for Mt5DataClient.__enter__ raising on initialization failure and for conflicting filters on the orders_get_as_df/positions_get_as_df path.

Suggestions

  • Narrow **filters: Any in _validate_filters_are_exclusive to str | int | None.
  • Update one-line summaries in orders_get/positions_get/history_orders_get/history_deals_get to mention group.
  • Consider filtering the raw MT5 sequence before dict conversion to reduce allocation for large histories.

None of these block merge, but the docstring/implementation inconsistencies and the Mt5RuntimeError wrapping are worth fixing.

Comment thread pdmt5/mt5.py
Comment thread pdmt5/mt5.py
Comment thread pdmt5/dataframe.py Outdated
Comment thread tests/test_dataframe.py
Comment thread tests/test_dataframe.py
Comment thread pdmt5/utils.py Outdated
@opencode-agent

opencode-agent Bot commented Jul 7, 2026

Copy link
Copy Markdown

Posted a review on PR #104: #104 (review)

I ran the full review agent set (code quality, performance, tests, documentation, security, code guidelines, types, comments, and silent-failure). The review is a COMMENT (not blocking) with six inline comments and a summary body.

Key findings noted:

  • _log_mt5_last_status_code still wraps internally-raised Mt5RuntimeError into a generic message.
  • _validate_history_filters allows group without a date range, contradicting its docstring.
  • _validate_history_input doesn't reject symbol/group combined with ticket/position, leading to confusing downstream errors.
  • Several docstrings still say "or None" for methods that now raise on None.
  • docs/api/mt5.md overstates that all methods raise Mt5RuntimeError.
  • A few test gaps remain in the DataFrame wrapper paths.

Note: uv was not available in this environment, so I couldn't run .agents/skills/local-qa/scripts/qa.sh locally; the PR body already reports all checks passed.

github run

@dceoy dceoy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the MT5 initialization/error-path changes, mutually-exclusive filter validation, exact symbol post-filtering, timestamp/timezone documentation updates, and the reported CI status. I do not see a blocking issue in this diff. The behavior changes appear covered by the expanded mock-based tests, and the latest CI/CD run is green.

@dceoy dceoy self-assigned this Jul 7, 2026
- Stop re-wrapping internally raised Mt5RuntimeError in the status decorator
- Reject history queries missing either date when no ticket/position is given
- Validate symbol/group against ticket/position in Mt5DataClient history input
- Drop stale "or None" return docs and mention group in getter summaries
- Clarify server-time vs UTC epoch contract in README and fix docs claims
- Cover as_df/as_dicts filter conflicts, exact-symbol dict path, and context
  manager initialization failure in tests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@dceoy dceoy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the implementation, tests, and docs for PR #104.

No additional blocking findings.

What I checked:

  • Initialization failure now raises Mt5RuntimeError through __enter__() / auto-initialization, and tests cover both direct context-manager and lazy-initialization paths.
  • *_total methods now validate None responses while preserving valid zero counts.
  • orders_get / positions_get / history getters now reject ambiguous filter combinations before delegating to MT5.
  • History symbol filtering now fetches via the required MT5 wildcard group and post-filters exact symbol matches, with tests covering broker-suffixed variants.
  • Decorator ordering for datetime conversion before index setting is consistent with the documented behavior.
  • Docs have been updated to align with the intentional breaking changes and timestamp contract.
  • CI for head commit 0801b2972a62740caa81a55287ba51cb84c37b21 is green.

One non-blocking note: the timestamp/server-time contract intentionally differs from the literal wording in the official MQL5 docs. The PR documents that nuance clearly enough for now, but downstream live-terminal validation remains valuable before relying on it as a hard public contract.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@dceoy dceoy merged commit 0ad6eeb into main Jul 7, 2026
5 checks passed
@dceoy dceoy deleted the fix-issues-99-103 branch July 7, 2026 15:46
dceoy pushed a commit to dceoy/mt5cli that referenced this pull request Jul 7, 2026
pdmt5 1.1.1 (dceoy/pdmt5#104) fixes initialization error reporting,
makes Mt5Client.__enter__()/_initialize_if_needed() and the *_total
methods raise Mt5RuntimeError instead of failing silently, raises
ValueError on mutually exclusive orders/positions/history filters
instead of silently applying precedence, and makes the history symbol
filter an exact match (no longer returning broker-suffixed variants).

mt5cli only calls the *_as_df wrappers with single or caller-supplied
optional filters, never combines them internally, and never uses the
raw pdmt5.Mt5Client base class, so none of the raised-instead-of-silent
behavior changes require code changes here. Full test suite (1175
tests, 100% coverage) passes unchanged against pdmt5 1.1.1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4
dceoy added a commit to dceoy/mt5cli that referenced this pull request Jul 7, 2026
* Resolve broker filling mode for position closes instead of assuming IOC

close_open_positions() previously hardcoded the IOC filling mode for every
close order, so closes were rejected on brokers whose symbols only support
FOK or RETURN even though entries succeeded. Add an optional
order_filling_mode parameter (also exposed as --filling-mode on
close-positions); when omitted, resolve the mode per symbol with
resolve_broker_filling_mode().

Closes #103

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4

* Backfill dsr and trial_count onto streamed SQLite result rows

In --stream --output-sqlite3 runs, result rows were inserted before the
full trial statistics were known and the enriched top-N records were
never written back, leaving the dsr/trial_count columns NULL for every
streamed run. Update the retained top-N rows in place (matched by
run_id and parameters) once the summary diagnostics are computed.

Addresses Codex review feedback on #360.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4

* Expand test coverage for the close-positions filling-mode change

Parametrize the Request/Instant IOC-implied-support tests into a full
matrix (execution mode x filling_mode 0/None x preferred order), add a
multi-symbol close_open_positions test asserting per-symbol type_filling
with one symbol_info lookup per unique symbol, parametrize explicit
filling-mode passthrough over IOC/FOK/RETURN (including case-insensitive
CLI input) at both the trading and CLI layers, broaden the invalid
--filling-mode CLI test beyond a single bogus value, and assert
close-positions --help documents --filling-mode.

Addresses review feedback on #110.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4

* Bump pdmt5 dependency to >=1.1.1

pdmt5 1.1.1 (dceoy/pdmt5#104) fixes initialization error reporting,
makes Mt5Client.__enter__()/_initialize_if_needed() and the *_total
methods raise Mt5RuntimeError instead of failing silently, raises
ValueError on mutually exclusive orders/positions/history filters
instead of silently applying precedence, and makes the history symbol
filter an exact match (no longer returning broker-suffixed variants).

mt5cli only calls the *_as_df wrappers with single or caller-supplied
optional filters, never combines them internally, and never uses the
raw pdmt5.Mt5Client base class, so none of the raised-instead-of-silent
behavior changes require code changes here. Full test suite (1175
tests, 100% coverage) passes unchanged against pdmt5 1.1.1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4

* Bump version to 1.1.4

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment