Fix initialization, filter, and timestamp-contract issues (#99-#103)#104
Conversation
Capture last_error() immediately after a failed initialize() or login() and embed that value in the raised Mt5RuntimeError, instead of querying the terminal after shutdown() has already closed the IPC connection. Closes #100 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mt5Client.__enter__() and _initialize_if_needed() previously discarded the boolean result of initialize(), so a dead terminal or connection "succeeded" silently. Both now raise Mt5RuntimeError with last_error() details via a shared _initialize_or_raise() helper, matching Mt5DataClient.__enter__() behavior. The five *_total methods also validated nothing and passed None through despite their -> int annotations; they now use _validate_mt5_response_is_not_none like the other getters. Closes #101 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The filterable getters silently prioritized mutually exclusive filters
(ticket > group > symbol; ticket/position over date range), returning
data filtered differently from what the caller asked for. orders_get,
positions_get, history_orders_get, and history_deals_get now raise
ValueError on conflicting combinations, and the logging decorator lets
ValueError propagate instead of wrapping it in Mt5RuntimeError.
The history symbol filter also matched substrings via the *{symbol}*
group wildcard (returning broker-suffixed variants such as EURUSD.m)
and silently discarded a caller-supplied group. history_*_get_as_dicts
now reject symbol combined with group and post-filter the returned rows
on exact symbol equality.
Closes #102
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MT5 epoch timestamps are trade-server wall-clock labels (typically UTC+2/UTC+3), and pdmt5 converts them to timezone-naive values that preserve those labels. Nothing stated this, so downstream consumers treated the converted datetimes as UTC and built live-trading logic on a 2-3 hour bias. Add a "Timestamps and Timezones" section to README.md, docs/index.md, and the conversion rules in docs/api/utils.md, plus one-line notes on detect_and_convert_time_to_datetime and the date_from/date_to parameters of the history and copy_* accessors in both layers. Closes #99 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- docs/api/utils.md showed @detect_and_convert_time_to_datetime as the outer decorator, the opposite of every call site; with that order the raw epoch column would be moved into the index before conversion. Show the real order and explain why set_index_if_possible stays outermost. Also correct the "conversions only happen when requested" claim (conversion is on by default, opt-out via skip_to_datetime). - Remove redundant client.initialize()/client.login() calls inside `with` blocks across docs/api/mt5.md, docs/index.md, docs/api/index.md, and docs/api/dataframe.md; the context managers already initialize (and Mt5DataClient logs in via config). The removed client.login(config.login, ...) calls also failed pyright strict since config.login is int | None. - Update mt5.md Best Practices and Error Handling for the context manager raising Mt5RuntimeError on failure, and fix the Logging section to reference INFO level (the wrapper logs at INFO/WARNING, not DEBUG). Closes #103 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d8fd232a9b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Reviewed code quality, performance, tests, documentation, security, types, and silent-failure concerns. Overall this is a solid, focused set of fixes. A few important issues to address before merging:
Implementation issues
pdmt5/mt5.py:73—_log_mt5_last_status_codestill wraps internally-raisedMt5RuntimeErrorinto a generic message (see inline comment).pdmt5/mt5.py:1073—_validate_history_filtersallowsgroupwithout a date range, contradicting its docstring.pdmt5/dataframe.py:1126—_validate_history_inputdoes not rejectsymbol/groupcombined withticket/position, producing a confusing downstream error.
Documentation
docs/api/mt5.md:221— the claim that all methods raiseMt5RuntimeErroris too broad.- Many
pdmt5/mt5.pydocstrings still say "or None" for methods that now raise onNone:account_info,terminal_info,symbols_get,symbol_info,symbol_info_tick,market_book_get,copy_rates_from,copy_rates_from_pos,copy_rates_range,copy_ticks_from,copy_ticks_range,orders_get,order_calc_margin,order_calc_profit,order_check,order_send,positions_get,history_orders_get,history_deals_get. docs/api/utils.md:66— the millisecond rule says any field ending in_mscis converted, but the code only converts keys that start withtime_and end with_msc.
Test gaps
tests/test_dataframe.pylacks coverage forMt5DataClient.__enter__raising on initialization failure and for conflicting filters on theorders_get_as_df/positions_get_as_dfpath.
Suggestions
- Narrow
**filters: Anyin_validate_filters_are_exclusivetostr | int | None. - Update one-line summaries in
orders_get/positions_get/history_orders_get/history_deals_getto mentiongroup. - Consider filtering the raw MT5 sequence before dict conversion to reduce allocation for large histories.
None of these block merge, but the docstring/implementation inconsistencies and the Mt5RuntimeError wrapping are worth fixing.
|
Posted a review on PR #104: #104 (review) I ran the full review agent set (code quality, performance, tests, documentation, security, code guidelines, types, comments, and silent-failure). The review is a Key findings noted:
Note: |
dceoy
left a comment
There was a problem hiding this comment.
Reviewed the MT5 initialization/error-path changes, mutually-exclusive filter validation, exact symbol post-filtering, timestamp/timezone documentation updates, and the reported CI status. I do not see a blocking issue in this diff. The behavior changes appear covered by the expanded mock-based tests, and the latest CI/CD run is green.
- Stop re-wrapping internally raised Mt5RuntimeError in the status decorator - Reject history queries missing either date when no ticket/position is given - Validate symbol/group against ticket/position in Mt5DataClient history input - Drop stale "or None" return docs and mention group in getter summaries - Clarify server-time vs UTC epoch contract in README and fix docs claims - Cover as_df/as_dicts filter conflicts, exact-symbol dict path, and context manager initialization failure in tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dceoy
left a comment
There was a problem hiding this comment.
Reviewed the implementation, tests, and docs for PR #104.
No additional blocking findings.
What I checked:
- Initialization failure now raises
Mt5RuntimeErrorthrough__enter__()/ auto-initialization, and tests cover both direct context-manager and lazy-initialization paths. *_totalmethods now validateNoneresponses while preserving valid zero counts.orders_get/positions_get/ history getters now reject ambiguous filter combinations before delegating to MT5.- History
symbolfiltering now fetches via the required MT5 wildcard group and post-filters exact symbol matches, with tests covering broker-suffixed variants. - Decorator ordering for datetime conversion before index setting is consistent with the documented behavior.
- Docs have been updated to align with the intentional breaking changes and timestamp contract.
- CI for head commit
0801b2972a62740caa81a55287ba51cb84c37b21is green.
One non-blocking note: the timestamp/server-time contract intentionally differs from the literal wording in the official MQL5 docs. The PR documents that nuance clearly enough for now, but downstream live-terminal validation remains valuable before relying on it as a hard public contract.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
pdmt5 1.1.1 (dceoy/pdmt5#104) fixes initialization error reporting, makes Mt5Client.__enter__()/_initialize_if_needed() and the *_total methods raise Mt5RuntimeError instead of failing silently, raises ValueError on mutually exclusive orders/positions/history filters instead of silently applying precedence, and makes the history symbol filter an exact match (no longer returning broker-suffixed variants). mt5cli only calls the *_as_df wrappers with single or caller-supplied optional filters, never combines them internally, and never uses the raw pdmt5.Mt5Client base class, so none of the raised-instead-of-silent behavior changes require code changes here. Full test suite (1175 tests, 100% coverage) passes unchanged against pdmt5 1.1.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4
* Resolve broker filling mode for position closes instead of assuming IOC close_open_positions() previously hardcoded the IOC filling mode for every close order, so closes were rejected on brokers whose symbols only support FOK or RETURN even though entries succeeded. Add an optional order_filling_mode parameter (also exposed as --filling-mode on close-positions); when omitted, resolve the mode per symbol with resolve_broker_filling_mode(). Closes #103 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4 * Backfill dsr and trial_count onto streamed SQLite result rows In --stream --output-sqlite3 runs, result rows were inserted before the full trial statistics were known and the enriched top-N records were never written back, leaving the dsr/trial_count columns NULL for every streamed run. Update the retained top-N rows in place (matched by run_id and parameters) once the summary diagnostics are computed. Addresses Codex review feedback on #360. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4 * Expand test coverage for the close-positions filling-mode change Parametrize the Request/Instant IOC-implied-support tests into a full matrix (execution mode x filling_mode 0/None x preferred order), add a multi-symbol close_open_positions test asserting per-symbol type_filling with one symbol_info lookup per unique symbol, parametrize explicit filling-mode passthrough over IOC/FOK/RETURN (including case-insensitive CLI input) at both the trading and CLI layers, broaden the invalid --filling-mode CLI test beyond a single bogus value, and assert close-positions --help documents --filling-mode. Addresses review feedback on #110. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4 * Bump pdmt5 dependency to >=1.1.1 pdmt5 1.1.1 (dceoy/pdmt5#104) fixes initialization error reporting, makes Mt5Client.__enter__()/_initialize_if_needed() and the *_total methods raise Mt5RuntimeError instead of failing silently, raises ValueError on mutually exclusive orders/positions/history filters instead of silently applying precedence, and makes the history symbol filter an exact match (no longer returning broker-suffixed variants). mt5cli only calls the *_as_df wrappers with single or caller-supplied optional filters, never combines them internally, and never uses the raw pdmt5.Mt5Client base class, so none of the raised-instead-of-silent behavior changes require code changes here. Full test suite (1175 tests, 100% coverage) passes unchanged against pdmt5 1.1.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EVZUF9t6ezmw18QwKj76H4 * Bump version to 1.1.4 --------- Co-authored-by: Claude <noreply@anthropic.com>
Summary
Fixes five issues found while debugging downstream live-trading consumers:
Mt5DataClient.initialize_and_login_mt5()now captureslast_error()immediately after a failedinitialize()/login()(beforeshutdown()closes the IPC connection), so the raisedMt5RuntimeErrorreports the real login/authorization error instead of the post-shutdown status.Mt5Client.__enter__()and_initialize_if_needed()now raiseMt5RuntimeError(withlast_error()details) wheninitialize()returnsFalse, matchingMt5DataClientbehavior. The five*_totalmethods validate their responses with_validate_mt5_response_is_not_none, so they no longer returnNonedespite-> int.orders_get/positions_get/history_orders_get/history_deals_getraiseValueErrorwhen mutually exclusive filters are combined (previously they silently appliedticket > group > symbolprecedence). The historysymbolfilter now post-filters rows on exact symbol equality after the*{symbol}*group-wildcard fetch, so broker-suffixed variants (EURUSD.m,EURUSD.raw) are no longer returned; combiningsymbolwithgroupalso raises instead of silently droppinggroup.README.md,docs/index.md, anddocs/api/utils.md, plus docstring notes: MT5 epochs are trade-server wall-clock labels (typically UTC+2/+3) and pdmt5's converted datetimes are timezone-naive server time, not UTC;date_from/date_toarguments are interpreted in server time by the MetaTrader5 API.docs/api/utils.md(set_index_if_possiblemust stay outermost), removed the redundantinitialize()/login()calls insidewithblocks across the docs (the context managers already handle both; the removedclient.login(config.login, ...)examples also failed pyright strict), corrected the logging-level (INFO, not DEBUG) and conversion-default prose, and aligned Best Practices with the new raising behavior.Intentional behavior changes
with Mt5Client(...)and auto-initialization now raiseMt5RuntimeErroron a dead terminal/connection instead of continuing silently;*_totalmethods raise instead of returningNone.ValueError(the logging decorator letsValueErrorpropagate unwrapped); the historysymbolfilter is now an exact match.Release
QA
.agents/skills/local-qa/scripts/qa.sh— all checks passeduv run pytest— 421 passed, 33 skipped (Windows-only), 100% branch coverageuv run ruff check ./uv run pyright— cleanuv run mkdocs build— succeedsWindows/MT5 note: all changes are exercised via the existing mock-based test suite; no live-terminal behavior was verified (CI runs on Windows).
Closes #99
Closes #100
Closes #101
Closes #102
Closes #103
🤖 Generated with Claude Code