Conversation
Batch A of the new-ops initiative — stdlib-only, no new dependencies. - text_ops: FA_file_split, FA_file_merge, FA_encoding_convert, FA_line_count, FA_sed_replace (literal + regex). All writes atomic. - diff_ops: FA_diff_files / FA_diff_dirs / FA_apply_patch. Extends the existing diff_text_files / diff_dirs helpers with apply_text_patch (unified-diff driver with strict context-line verification) and a JSON-friendly diff_dirs_summary wrapper. - data_ops: FA_csv_filter (column projection + where clause), FA_csv_to_jsonl, FA_jsonl_iter (with limit), FA_jsonl_append. - Facade + __all__ updated; new exception types TextOpsException and DataOpsException added under the shared exceptions hierarchy. Tests: 42 new cases — all pass. Full suite: 686 passed, 8 skipped.
Batch B — introduces three required dependencies: PyYAML, pyarrow, opentelemetry-api/sdk (all added to requirements.txt + dev.toml + stable.toml). YAML / Parquet: - FA_yaml_get, FA_yaml_set, FA_yaml_delete — dotted-path access with yaml.safe_load (never yaml.load) so untrusted config can't construct arbitrary Python objects. - FA_parquet_read (with limit + columns projection), FA_parquet_write, FA_csv_to_parquet. All writes atomic. Tracing: - core.tracing.init_tracing installs a global TracerProvider with a configurable SpanExporter (defaults to a null exporter so spans can still be inspected without wiring up a backend). - action_span context manager is a zero-cost no-op until tracing is initialised. - ActionExecutor._execute_event wraps every action in an "automation_file.action" span tagged with fa.action=<name>. - FA_tracing_init exposed through the registry. Tests: 22 new — all pass. Full suite: 708 passed, 8 skipped.
Batch C — two new cloud backends mirroring the existing S3 / Azure / Dropbox pattern, plus matching PySide6 tabs. OneDrive (Microsoft Graph via MSAL): - OneDriveClient holds an OAuth2 access token and a requests.Session, exposes both later_init(token) and device_code_login(client_id) paths. graph_request() centralises auth header + error wrapping. - Ops: upload_file / upload_dir (4 MiB simple-upload cap), download_file, delete_item, list_folder (follows @odata.nextLink). - FA_onedrive_* actions registered, UI tab added to Transfer sidebar and Home status probes. Box (boxsdk): - BoxClient wraps boxsdk.OAuth2 + Client with access-token init. - Ops: upload_file (returns new file id), upload_dir (flattens tree since Box requires pre-existing folder ids), download_file, delete_file, delete_folder (recursive flag), list_folder. - FA_box_* actions registered, UI tab added. Deps added to requirements.txt + stable.toml + dev.toml: msal, boxsdk. Both declared as required runtime deps, consistent with the existing boto3 / azure-storage-blob / dropbox / paramiko treatment. Tests: 31 new — all pass. Full suite: 739 passed, 8 skipped. Real Graph / Box endpoints are out of CI; tests exercise registry wiring, guard clauses, and exception-wrapping paths via in-memory fakes.
CI lint (mypy, 9 errors): - text_ops.file_merge: replace the iter(lambda) chunker with a plain while loop so mypy can infer the chunk type. - data_ops: _resolve_fieldnames now accepts Sequence[str] | None to match csv.DictReader.fieldnames. - tracing: drop unused # type: ignore[attr-defined] comments now that mypy resolves the Once / TracerProvider references on its own. - tracing: replace # type: ignore[override] on the _NullExporter methods with plain signatures that already match the parent. - box.upload_ops: wrap box_upload_file in a helper that returns bool so walk_and_upload's Callable[[Path, str], bool] is satisfied. Codacy (11 new issues): - Bandit B105 in test_onedrive_ops: promote the literal "fake-token" string to a named local and mark with # nosec B105 + comment. - PyLint W0212 across test_onedrive_ops + test_tracing: add file-level # pylint: disable=protected-access with rationale — the tests legitimately poke tracing._state and OneDriveClient._session / _access_token to inject fakes. - PyLint R0914 in data_ops.csv_filter (17 locals): refactor into csv_filter → _stream_csv_filter → _write_filtered_rows, each below the 15-locals cap. - PyLint R0913 in onedrive.client.graph_request (8 args): collapse params / json_body / data / headers into **request_kwargs forwarded to requests.Session.request. - PyLint R0917 too-many-positional-arguments on the new data_ops helpers: justified disable-next per helper. SonarCloud (4 issues): - python:S108 empty-body blocks in test_tracing: replace the `pass`-only with blocks with meaningful assertions / operations so the span bodies are non-empty. - pythonsecurity:S2083 in apply_text_patch: document that the caller is the trust boundary for target (matching the rest of the local ops) and mark the write_text call with # NOSONAR pythonsecurity:S2083 + explanation. - python:S3776 cognitive complexity in _apply_unified_patch (30) and _iter_hunks (17): split both into small helpers (_copy_up_to, _apply_hunk_ops, _verify_live_line for the former; a _HunkParseState dataclass + _consume_patch_line for the latter). Each helper now scores well under the 15-complexity cap. Verified: `ruff check`, `ruff format --check`, `mypy`, and `pytest` (739 passed, 8 skipped).
…ackends Add text/diff/data ops, YAML/Parquet, OTel tracing, and OneDrive/Box backends
Replace the legacy mermaid flowchart with a comprehensive overview that covers every current subsystem (DAG executor, triggers, scheduler, notifications, observability, all remote backends including OneDrive/ Box/WebDAV/SMB/FTP/fsspec, MCP/metrics/web UI servers, crypto, audit, FIM, config + secrets) and uses bold borders, bold labels, and thick arrows for readability. Mirror the new diagram into README.zh-TW.md / README.zh-CN.md and add a System overview section to docs/source*/architecture.rst, wired through sphinxcontrib-mermaid (added to docs/requirements.txt and all three Sphinx conf.py extension lists).
…ackends Refresh system architecture diagram in README and docs
execute_action's result dict keyed entries on f"execute: {action}", so two
identical actions in one batch would collide and the first result would be
overwritten. With substitute=True, the substituted action (with ${env:...}
already expanded) was also written into the log line and the result key,
exposing secrets pulled in via ${env:...}.
- Key results as f"execute[{index}]: {display}" so duplicate actions keep
both results, matching execute_action_parallel's existing format.
- Track the original (un-substituted) action separately and use it for the
result key, the success log line, and the dry_run payload log; only the
executed callable receives the substituted payload.
- Update docs (en / zh-TW / zh-CN) and the http server test for the new
key format.
- Add regression tests for both fixes in test_action_executor.
…r-keys Fix action result key collisions and secret leak in substitute mode
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 387 |
| Duplication | 14 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codacy/Bandit flagged the literal string assigned to FA_EXEC_SECRET as a hardcoded password (B105). Switch to pytest's monkeypatch (matching test_substitution.py's existing pattern), rename the env var so the key itself does not contain "SECRET", and use a non-credential-looking sentinel value. Behaviour under test is unchanged.
3 tasks
…test-secret Silence Codacy Bandit B105 false positive in executor leak test
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promote the current
devhead tomainfor a stable release. Triggers the publish workflow (version bump instable.toml/dev.toml, build + upload to PyPI, tagvX.Y.Z, GitHub release).Changes since last main release
e650dd9Add text_ops, diff_ops patching, and CSV/JSONL data_ops9f6e6abAdd YAML / Parquet data_ops and OpenTelemetry tracing8995da1Add OneDrive and Box backends (+ UI tabs)c05aaf6Address CI lint, Codacy, and SonarCloud findings on PR Add text/diff/data ops, YAML/Parquet, OTel tracing, and OneDrive/Box backends #5955d2041Refresh system architecture diagram in README and docsd1b4f20Fix action result key collisions and secret leak in substitute mode (Fix action result key collisions and secret leak in substitute mode #61)Test plan