Integrate R740 hardening stack → main (RH-003…RH-007) - #44
Merged
Conversation
_SUBDIRS omitted GEN and no session table was generated, so BPE aborted early — the exact stall hit during the NAMRIA training week. Closes readiness gap #2. - Add GEN to _SUBDIRS. - campaign_builder: generate_sessions_ses() returns the stock daily '???0' session table (whole UTC day, verbatim from $X/SUPGUI/PAN/SESSIONS.SES); stage_sessions_ses() writes it into campaign GEN/, copying an explicit install template when given and never clobbering a hand-tuned existing SESSIONS.SES. - prepare_campaign() writes SESSIONS.SES unconditionally (independent of CampaignConfig) since BPE needs it regardless; new sessions_template= arg. - test_campaign_builder.py +5. 80 pass, ruff clean. Developed in worktree .trees/rh-003-gen-sessions off branch feat/rh-003-gen-sessions.
…04 core) OPT/*.INP panels authored on Windows and copied to the Linux BPE carry backslash separators, absolute C:\Bernese drive paths, and hardcoded session/date literals (all seen verbatim in the training-week PGN_WK panels). Addresses gaps #8/#14 / readiness section 2.5. panel_sanitizer.py: - sanitize_panel_text(): converts MIXED Bernese/Windows separators (${P}/SOB\GEN -> ${P}/SOB/GEN) but FLAGS, never rewrites, foreign drive-letter paths and hardcoded session/date literals — silently 'fixing' C:\Bernese\... would produce a still-broken C:/Bernese path and hide the real remap. Structured PanelWarning list; ok == no warnings. - find_dangling_waits(): WAIT referencing an undefined PID (the stray WAIT=522 class that hangs the BPE forever). - INP-only by design: NOT run on SCRIPT/*.pl, where a backslash is a Perl escape and blanket conversion would corrupt the script. Verified on the real PGN_WK/ADDNEQ2.INP (flags 4 drive paths, 4 dates, 5 session stamps). test_panel_sanitizer.py +11 with verbatim panel-line fixtures. 86 pass, ruff + mypy clean. RH-004 remainder (render-path wiring, gold-standard provisioning, MAXPAR into ADDNEQ2 panel = task B) tracked in the backlog. Developed in worktree .trees/rh-004-panel-sanitizer.
Completes the RH-004 code mechanisms on top of the sanitizer core. - set_addneq2_maxpar(text, value): rewrites the ADDNEQ2 MAXPAR value line (leaves MSG_MAXPAR help text). Wires readiness task B to RH-002's compute_maxpar() so the combined-NEQ parameter ceiling scales with the network instead of the panel's frozen '5000'. - provision_opt_dir(src, dest, *, n_stations, strict): the sanitizer's applied layer. Sanitizes every *.INP on the way to $U/OPT, sizes MAXPAR on ADDNEQ2.INP when n_stations is given, and copies non-INP files (*.pl scripts) VERBATIM. Strict (default) refuses to write any panel still carrying an unresolved hazard (foreign drive path / hardcoded session) — so a dirty panel can never reach $U; it must be hand-remapped first. test_panel_sanitizer.py +6. 92 pass, ruff + mypy clean. Remaining RH-004 piece is authoring the gold-standard panel CONTENT (hand-remap C:\Bernese paths to Bernese vars) — a data/ops task the strict provisioner now enforces. Developed in worktree .trees/rh-004-panel-sanitizer.
CODSPP (BPE 301/302) reports a per-station RMS OF UNIT WEIGHT and a NEW- A PRIORI coordinate correction. Those two numbers separate the two failure modes staff triaged by hand in the training week (gap #9): high RMS + large coordinate shift = bad a priori (seed .CRD wrong, free auto-fix by re-seeding from CODSPP NEW coords); high RMS + small shift = bad observations (data fault, needs a human). codspp_qc.py: - parse_codspp_output(): station code, RMS OF UNIT WEIGHT, BAD/USED OBSERVATIONS, X/Y/Z NEW- A PRIORI deltas -> coord_shift_m. Missing fields degrade to None, never raise. - classify_codspp(): ok / bad_apriori / bad_obs / unknown, with tunable rms + shift thresholds (defaults 3 m / 1 m; CODSPP code solutions sit ~1 m, tune on a real network). - parse_codxtr_summary(): combined CODXTR MAX. RMS / MAX. BAD worst-station. Verified against the real CUSV 0840 SPP output. test_codspp_qc.py +9, ruff + mypy clean (run via the rh-004 worktree venv to avoid a uv sync while the BPE final solve was running). RH-005 remainder: the re-seed ACTION (gap #9) and the PID-322 tropo quarantine (gap #11) — tracked in the backlog. Developed in worktree .trees/rh-005-codspp-tropo.
Strict mode wrote clean panels inside the walk loop, then raised on the first dirty panel — leaving $U/OPT half-updated and losing the ProvisionReport. Now two-pass: sanitize and collect all warnings first, raise before ANY write if strict + warnings, then commit the planned writes. Adds an atomicity test (clean panel ordered before a dirty one is not written on abort).
…FTP_DWLD (RH-007) The Option-B decision (pre-download IGS products, skip in-BPE FTP_DWLD) was only half-built: the template still shipped 000 FTP_DWLD, and the downloader path was defined but never called. Closes gaps #4/#6/#7. - basic_processing.pcf.j2: strip 000 FTP_DWLD + its WAIT ref; 001 R2S_COP is now the head PID. - campaign_builder.verify_igs_products(): pre-flight existence check that reuses igs_downloader's _build_long_filename + on-disk layout (ORB/YYYY/DDD/<decompressed>) as the single source of truth, so it matches exactly what ProductDownloader writes and what V_ORB=COD0OPSFIN expects. Returns the missing set; IGS20 long-name era (pre-2022 raises). - prepare_campaign(prefetch_products=True, product_ac=...): pre-downloads ORB/CLK then verifies, raising BEFORE the BPE run if incomplete. Opt-in (default off) so existing callers are unchanged. test_igs_predownload.py +8; test_orchestrator FTP_DWLD assertion inverted. 83 pass, ruff clean. Verified via the rh-004 worktree venv (no uv sync — BPE final solve running). Developed in worktree .trees/rh-007-igs-predownload.
…006) Plumbing for the 502 GPSCLU_P single-cluster bottleneck (gap #13). The 40-min single-core final solve came from V_CLUFIN=A auto-clustering the whole network into ONE dense inversion; the fix is final-solution clustering + USER.CPU maxjobs across cores. That win is CONFIG, and its empirical value needs the R740 (BRN-001) — this ships the mechanisms. - cpu_config.compute_maxjobs(): maxjobs from PHYSICAL cores (sub-solves are FPU-bound, hyperthreads don't help), RAM-capped and reserve-aware. - cpu_config.set_user_cpu_maxjobs(): rewrites the localhost maxjobs field in USER.CPU, leaving command/speed/other CPUs/MSG lines intact. - PCFContext exposes v_clu (10) + v_clufin ('A', a MODE flag N/A per the real PCF — corrected from the readiness doc's 'set to a number' framing); the template now templates both (V_CLUFIN was absent). test_cpu_config.py +13. 88 pass, ruff + mypy clean. Verified via the rh-004 worktree venv (no uv sync — BPE running). Worktree .trees/rh-006-clustering. RH-006 stays PARTIAL: the actual V_CLUFIN split value is an R740 tuning task.
feat(bernese): USER.CPU maxjobs sizing + V_CLUFIN/V_CLU plumbing (RH-006)
feat(bernese): Option-B IGS pre-download + pre-flight verify, retire FTP_DWLD (RH-007)
…itizer # Conflicts: # docs/project_documentation/ticket_backlog.md
feat(bernese): panel sanitizer — separators/paths/sessions/WAIT (RH-004 core)
# Conflicts: # docs/project_documentation/ticket_backlog.md
feat(bernese): CODSPP-QC parse + a-priori/obs triage (RH-005 core)
# Conflicts: # docs/project_documentation/ticket_backlog.md
…o feat/rh-003-gen-sessions # Conflicts: # services/bernese-workflow/src/bernese_workflow/backends.py
feat(bernese): stage GEN/SESSIONS.SES in prepare_campaign (RH-003)
|
Warning Review limit reached
Next review available in: 5 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (14)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrate R740 hardening stack → main (RH-003…RH-007)
Brings the RH-003/004/005/006/007 work — merged onto
docs/bernese-training-notesvia #39/#40/#41/#42/#43 — intomain. (#38 already landed RH-001/RH-002 + docs; the stacked PRs targeted the docs branch, so this final merge promotes them to main.)Contents on top of #38:
Integrated + verified: full
bernese-workflowsuite 128 passed, ruff + mypy clean. The RH-003×RH-007prepare_campaignoverlap (SESSIONS.SES + IGS prefetch) was hand-merged so both features coexist.