FD-238: Open source ML auditing system in btcopilot by patrickkidd · Pull Request #1 · patrickkidd/btcopilot

patrickkidd · 2025-09-23T04:22:11Z

No description provided.

…ngs, ollama / mistrel

gitguardian · 2025-09-23T04:22:16Z

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
21808676	Triggered	Generic Password	`42628dc`	btcopilot/pro/tasks.py	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secret safely. Learn here the best practices.
Revoke and rotate this secret.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

Three fixes to f1_metrics.py PairBond matching accuracy: 1. Fix parent ID resolution in match_people: Person.parents references a PairBond ID, but PairBond IDs are never in the id_map (only person IDs are). The old code fell through to comparing raw IDs which always failed. Now skips parent comparison when the parent ID can't be resolved, breaking the circular dependency between people matching and PairBond matching. 2. Add _augment_committed_id_map: resolves committed (positive) person ID mismatches when diagram_data has duplicate entries for the same person. 3. Add _augment_duplicate_person_id_map: when AI creates duplicate people (same name, different IDs), match_people only maps one (1:1). This adds many-to-one mappings so PairBonds referencing unmapped duplicates resolve correctly. Also adds PairBond completeness checklist to DATA_FULL_EXTRACTION_CONTEXT prompt, addressing the #1 extraction failure mode (under-extraction of PairBonds — disc 36 had 0 bonds extracted). Results (disc 36/37/39/48): - PairBond F1: 0.37 → 0.52 (target ≥ 0.50) - People F1: 0.72 → 0.79 - All 451 tests pass (308 training + 143 schema/personal) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Raise PairBond F1 from 0.37 to 0.52 via matching fixes and prompt update Three fixes to f1_metrics.py PairBond matching accuracy: 1. Fix parent ID resolution in match_people: Person.parents references a PairBond ID, but PairBond IDs are never in the id_map (only person IDs are). The old code fell through to comparing raw IDs which always failed. Now skips parent comparison when the parent ID can't be resolved, breaking the circular dependency between people matching and PairBond matching. 2. Add _augment_committed_id_map: resolves committed (positive) person ID mismatches when diagram_data has duplicate entries for the same person. 3. Add _augment_duplicate_person_id_map: when AI creates duplicate people (same name, different IDs), match_people only maps one (1:1). This adds many-to-one mappings so PairBonds referencing unmapped duplicates resolve correctly. Also adds PairBond completeness checklist to DATA_FULL_EXTRACTION_CONTEXT prompt, addressing the #1 extraction failure mode (under-extraction of PairBonds — disc 36 had 0 bonds extracted). Results (disc 36/37/39/48): - PairBond F1: 0.37 → 0.52 (target ≥ 0.50) - People F1: 0.72 → 0.79 - All 451 tests pass (308 training + 143 schema/personal) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Address PR #65 review: remove PII from logs, add relationshipTriangles, improve markdown - Remove person names from debug log messages in _augment_committed_id_map and _augment_duplicate_person_id_map to prevent PII leakage - Add relationshipTriangles to committed ID collection loop (was only collecting relationshipTargets, missing triangle person IDs) - Improve MVP_DASHBOARD.md table cell readability with <br/> tags Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert PairBond checklist from public prompts — already in fdserver The PairBond completeness checklist is already in fdserver/prompts/private_prompts.py, so the public btcopilot prompts don't need it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Hurin <hurin@openclaw.ai> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

- Defer production prompt loading to avoid import-time FileNotFoundError in Docker CI where fdserver is not co-located - Remove dead _BASE_SARF_REVIEW variable (review comment #1) - Extract error message to PROMPTS_UNAVAILABLE_ERROR constant shared between litreview.py and compare.py (review comment #2) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Defer production prompt loading to avoid import-time FileNotFoundError in Docker CI where fdserver is not co-located - Remove dead _BASE_SARF_REVIEW variable (review comment #1) - Extract error message to PROMPTS_UNAVAILABLE_ERROR constant shared between litreview.py and compare.py (review comment #2) Co-authored-by: Hurin <hurin@openclaw.ai> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

- apply_local_changes: refactored to two-phase form. Local edits now beat server-side deletes per item-level last-write-wins (per docstring). Added regression test `test_local_edit_beats_server_delete`. (Gemini #1) - reserve_id_block: hoisted `import PyQt5.sip` out of retry loop. Marked `# noqa: F401` since the import is for SIP registration side effects, not a name reference. (Gemini #2) - merge_scene_collection: removed entirely. Pre-release feature with no production callers — deprecation was over-cautious. Deleted the associated test file. (Patrick review comment) All apply_local_changes tests pass (16, incl. new regression case). Full schema + pro suites green, no regressions.

#1 _remap_person_refs no longer remaps PairBond person.parents nor the mixed-entity deltas.delete list with a person-id-only map. #2 _drop_committed_dup_pair_bonds remaps a dropped duplicate bond's child onto the surviving committed pair_bond instead of nulling it, preserving the parent-child link (_committed_dyads now dyad→pb-id). #4 _windowed_conversation: empty tail keeps the cursor marker/nonce so the model emits nothing rather than re-extracting the full thread. Tests updated to the new contracts; 59 personal/isolation tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* FD-319: per-discussion re-extraction cursor + commit-pdp Adds discussions.extracted_through_order / pending_extracted_through_order, the _windowed_conversation cursor windowing with an unforgeable per-call nonce marker, the POST /personal/discussions/<id>/commit-pdp accept endpoint, the cursor Alembic migration, and prompt-idempotency work from the committed-duplicate induction runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * FD-319: re-extraction cursor repairs + committed-data integrity - Empty-PDP "Accept all" is a valid full accept and advances the cursor (empty item_ids allowed only with full_accept). - fix_committed_person_duplicates iterated to a fixed point (fixes intermittent extraction 500 on real diagrams). - Validator: event person/spouse/child positive refs must be committed. - fix_unresolved_person_refs drops events/pair_bonds with unresolvable refs and clears orphaned parents; commit_pdp_items skips orphaned pair bonds; inferred bonds require resolvable people. - child_of (parent/offspring) F1 metric + dashboard; structural-completion pass evaluated and rejected (logged). - Tests: cursor + pdp/commit/isolation green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * FD-319: address PR #119 review (pdp ref remap + empty-tail cursor) #1 _remap_person_refs no longer remaps PairBond person.parents nor the mixed-entity deltas.delete list with a person-id-only map. #2 _drop_committed_dup_pair_bonds remaps a dropped duplicate bond's child onto the surviving committed pair_bond instead of nulling it, preserving the parent-child link (_committed_dyads now dyad→pb-id). #4 _windowed_conversation: empty tail keeps the cursor marker/nonce so the model emits nothing rather than re-extracting the full thread. Tests updated to the new contracts; 59 personal/isolation tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Patrick Stinson <patrick@alaskafamilysystems.com>

patrickkidd added 30 commits February 8, 2025 10:04

Initial commit

90b4216

Initial commit

34e416d

Added sources submodule

430e612

Update sources head

f9b4cce

readme

09bc64c

readme

e058344

Rm sources submodule

04c526b

re-added sources submodule

eb9928b

sources tip

cff1ef2

reply w/ source refs using chroma, groq

e610e8e

response with source snippets and names, chroma ingestion, hf embeddi…

349aa1a

…ngs, ollama / mistrel

First working REST service

3c65dd2

Added automated tests for model

2c8aa9d

quiz formatting

c2cd659

add gilbert

94978f3

back to mistral

bbb687b

Removed reset service in favor if python API only

809d744

adjust paths for relative import

92cf7d1

moved to gpt-4o-mini

e2d8ea4

Added app manual and fd metadata

958923d

Added github workflow

f930cc8

Added pyproject.toml

6227123

ubuntu-24.04

4c14ae0

don't sub,odule sources

cee801f

added deps to pyproject

4676edf

pyproject.toml

b799786

pyproject.toml

2545a13

tests/__init__.py

6326ab6

MANIFEST.in

fc6c3d7

Added timeseries data

437ab48

patrickkidd added 6 commits September 22, 2025 20:18

renamed therapist to personal

f8a33c5

lots of tests pass

19a9869

Fixed test

1c09820

more tests pass

f507f70

init_celery

03fae2b

Fix training login page

b344b71

patrickkidd added 12 commits September 23, 2025 03:34

cd debug

1e33919

tmate

a0a9526

fix test failures

168afa4

no tamte

4e7a1a1

homogenated deps

a97bd5d

deps correction

cfd4ee5

/health

e0635ed

README

8c4453c

vedana

b6375c9

uncomment fd-flask

a87f567

Rename VariableShift -> Shift

a9ddda1

deprecated bugsnag

936403a

patrickkidd closed this Oct 23, 2025

patrickkidd force-pushed the FD-238-open-source-3 branch from 65c6d52 to 936403a Compare October 23, 2025 01:26

patrickkidd-hurin mentioned this pull request Mar 4, 2026

Raise PairBond F1 from 0.37 to 0.52 #65

Merged

4 tasks

patrickkidd mentioned this pull request May 18, 2026

FD-319: per-discussion re-extraction cursor + commit-pdp #119

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FD-238: Open source ML auditing system in btcopilot #1

FD-238: Open source ML auditing system in btcopilot #1
patrickkidd wants to merge 81 commits into
masterfrom
FD-238-open-source-3

patrickkidd commented Sep 23, 2025

Uh oh!

gitguardian Bot commented Sep 23, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

patrickkidd commented Sep 23, 2025

Uh oh!

gitguardian Bot commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gitguardian Bot commented Sep 23, 2025 •

edited

Loading