gps3: LUZON 5.4 reprocessing, archive transfers, and §16-17 - #66
Merged
Conversation
Section 15.4 said the work was "still open" and that origin/main "was and is at 1d1082e". PR #64 merged as 9623395 shortly afterwards, so the present tense had to go. Worth recording how it was confirmed rather than just that it happened. scripts/merge_pr.sh returned exit 0 and printed nothing at all — no confirmation line, no error. On a gated operation that is precisely the shape a swallowed failure takes, and this section already carries a correction about claiming main had advanced when it had not. So the merge was verified at the git level instead: origin/main moved from 1d1082e to 9623395, and each of the eleven commits was checked with git merge-base --is-ancestor. gh pr view's own JSON template came back empty on the same PR, which is a second reason not to have trusted the tooling's word for it. Branch retired locally and on the remote per Rule 1 — feature branches are working space, never storage.
Adds scripts/sudo/processed_transfer.sh and records the day's transfers. 229,881 files and 177.26 GiB copied from the DOSTB drive and verified by independent census on both sides: 162,328 files of RECOVERED_* legacy archive and 67,553 of the Bernese 5.2 LUZON processed set. Every directory matched on file count and byte total. The legacy archive now exists in two places on independent hardware, which closes the project's Tier 0 item. Two things the measurements corrected. The handover's RECOVERED_* table listed three directories and omitted RECOVERED_SEAGATE_W2A0W9T2_DATA0, which turned out to be 131 G and 86% of the archive by file count. The total then matched the continuity audit's 157 G exactly, and the 162,328 files corroborate its independent ~155k figure. The omission happened because three names were typed by hand; both scripts glob instead. The LUZON SOL directory was described as a comparison target naming two recent GPS weeks. It holds 725 weekly and 166 monthly combines spanning 2010-02-28 to 2026-04-12 — sixteen years of results in 421 MB, against one month of raw input. Those solutions cannot be regenerated from this tree, so the new script copies that subtree first and alone before the other 21 GB, and writes a PROVENANCE.txt saying so. The new script also fixes a defect the first one revealed in use: archive_transfer.sh tees rsync's --info=progress2 output, producing an 11.2 MB log for the 157 GB run with the structured census buried in carriage-return spam. Progress now goes to the terminal, structured lines to the log. Neither copy has fixity. Both scripts decline to say "backed up" and point at the sha256 manifest instead.
The drive handover recorded the goal, the I14/I20 model warning and a priority order, but no procedure. A procedure also does not belong in a document scoped to "the drive is currently attached" — the drive comes off today, the runbook gets consulted for months. It now lives in docs/ with README §3 pointing at it. Surveying the copied set to write it turned up two findings that change the exercise, both of which would have cost days if met mid-run. The raw data and the reference solutions do not overlap. DATAPOOL holds RINEX for 2025 DOY 121-151; the F1_ daily solutions cover 2025 DOY 029-033 and 2026 DOY 106-110, and there are none at all for DOY 121-151. The RINEX on that drive was never processed into the solutions on that drive, so reprocessing it yields numbers with nothing to compare against. The campaign OBS directory covers exactly the ten days that do have solutions, which makes it the viable comparison set. PHIVOL_REL.PCF references eight scripts that do not exist in this 5.4 install: ADD_MON, ADD_WK, FTP_DWLD, ORBGENH, POLUPDH, PRETAB, RNXSMT_H, RXOBV3_H. 41 of 49 are present. It cannot run unmodified. ADD_WK and ADD_MON are not needed for a daily comparison; the H-suffixed four may have non-H equivalents in 5.4, which is untested; FTP_DWLD and PRETAB should be unnecessary when running from already-staged products. Also records a parsing trap that produced a wrong answer first time round: the PCF contains three tables with different column meanings, and reading column 3 from every line beginning with three digits yields twelve OPT directories that do not exist because they are not directories. Parse lines 2-64 only. Section 5 lists the four questions that inspection cannot settle, chief among them whether 5.4 reads a 5.2 campaign's OBS files directly. If it does not, the comparison gets substantially harder, because RINEX exists only for the days with no reference solution.
…ions do not overlap Cherry-picked from 6399c9a, written by the T420 session. Recovered and re-landed here because it was stranded: it was authored 2026-08-05 08:10 on parent bd0929f, the PR #65 head *before* the hazard-count correction, and PR #65 had already merged at 40f0409 the previous evening with the branch deleted. The T420 recreated the branch and pushed there, leaving two divergent corrections to the same file -- main had the six-to-fifty hazard fix without the overlap fix, and the recreated branch had the reverse. They merge cleanly; both are now present. Original message follows. PROVENANCE.md called SOL/ "the comparison target" next to the DATAPOOL RINEX, implying they cover the same days. They do not: RINEX in DATAPOOL/LUZON/ 2025 DOY 121-151 (31 days) Converted OBS/ and F1_ finals 2025 DOY 029-033, 2026 DOY 106-110 Solutions for DOY 121-151 none The raw data on that drive was never processed into the solutions on that drive, so "reproduce her SOL/ numbers" cannot be run against the RINEX we have. The viable set is the ten days where OBS/ and F1_ coincide. Worth recording how the error was made. Both facts were observed correctly and written down separately -- the DOY range in one inventory, the F1_ filenames in another -- and never compared against each other. Two accurate observations, one wrong conclusion, produced by holding them apart. (The AI co-author trailer on the original was dropped: CLAUDE.md forbids Claude or AI references in commit messages. Authorship is unchanged.)
Corrects both the runbook and the PROVENANCE correction landed an hour ago, and lands the two T420 reply files. SAVEDISK/2025/SOL holds all 365 daily finals for 2025, gzipped, including every day of DOY 121-151. Verified on gps3: 730 files, 365 distinct DOYs, 31 of 31 days in the window. CAMPAIGN/LUZON/SOL looked empty of them because PHIVOL_REL.PCF archives to SAVEDISK with 902 R2S_SAV and then cleans the campaign with 903 R2S_DEL. A campaign holding only the most recent days is normal operation. The usable window is therefore 31 contiguous days from raw RINEX, not ten from converted observations. That is materially better: it exercises RXOBV3, station matching and QC, so this exercise can produce BRN-001 acceptance evidence, which the OBS path could not. New finding neither session had: seven IGS fiducials in her DOY 121 solution have no RINEX locally. F1_251210.SNX carries 30 stations, the DOY 121 RINEX carries 24, and AIRA ALIC BASC DAEJ DARW MCIL PNGM are absent. DATAPOOL_IGS holds only orbit products. A run from local RINEX alone would process 24 stations against a 30-station reference and measure the wrong thing. This is the same gap as one of the eight missing scripts: FTP_DWLD is exactly what fetched them. Three passes over one question, each more careful and each wrong. The DOY ranges were catalogued but never compared; then compared, but only within one directory; then answered by asking where the software writes finished solutions -- which the PCF had stated plainly, and which both sessions had read while checking something else. Also recovers commit 6399c9a from the T420, which was stranded: authored on the pre-correction PR #65 head after that PR had merged and its branch was deleted. The branch was recreated, leaving two divergent corrections to one file. They merge cleanly and both are now present, with the AI co-author trailer dropped per CLAUDE.md.
Second correction today, and it removes the last data blocker. The seven "missing" fiducials were never missing. AIRA ALIC BASC CLAV DAEJ DARW MCIL PIMO PNGM are in GPSDATA/DATAPOOL/RINEX3 as RINEX 3 long-name Hatanaka. DATAPOOL has fourteen subdirectories and both sessions had searched only LUZON/. Coverage over DOY 121-151 is 31/31 for eight of the nine; PNGM is 26/31. The decisive check is the station reconciliation against her DOY 121 solution: she used nothing we lack. We hold two she did not use, PIMO and TGDN. So the network is complete and nothing needs downloading or requesting from anyone. My previous claim that BASC was a Philippine station absent from every archive and needing a request to Abegail was wrong on both counts. FTP_DWLD therefore does not gate the run either. Every product for the window is local -- orbits for weeks 2364-2368, ERP, clocks, ionosphere, DCB, the I14 ATX. Four of the eight missing scripts are now confirmed unnecessary when running from staged products; only the four H variants remain genuinely open. Three caveats that do bite, all newly recorded. PNGM is short five days that her solutions include, so those days solve a different network than the reference and must not be read as a 5.4-vs-5.2 effect. BASC and CLAV exist in both naming conventions, so staging both would duplicate observations for one station -- a rule must be chosen before the first run, and RINEX 3 is suggested since its coverage is complete where RINEX 2's is not. Local coverage is 741 of 775 station-days, so a per-day count belongs in pre-flight rather than being discovered inside RXOBV3. Adds section 1.5 naming the class this belongs to. Five false absences this week, one shape: searching with one convention and reading zero results as absence -- .gz/.Z suffixes, MARKER NAME vs MARKER NUMBER, GPS-week vs year-DOY in the ION files, SAVEDISK vs the campaign directory, and now RINEX 3 long names vs RINEX 2 short names. The remedy was identical every time: look at actual filenames before writing the pattern. Also takes the T420's navigational signpost into the LUZON PROVENANCE so a reader hitting the superseded 08-04 block is sent to the 08-05 one before acting on it, and deletes feat/bernese-capture-pagenet-pcf after verifying its content was redundant rather than assuming it. Per Alfie's decision that file is now single-writer on this side.
…t blockers Systematic survey of what actually stands between the copied 5.2 LUZON set and a 5.4 run. Five of the seven concerns dissolved on inspection; one real gap emerged that nobody had noticed. PHI_WK and PHI_MO are referenced only by PIDs 530 and 531, ADD_WK and ADD_MON. Stopping the PCF at 514 means they never load, which removes the entire panel-remediation blocker -- 72 hazardous panels that do not need remediating because the daily path does not touch them. The R2S_* panels it does use already ship with 5.4. The PCF turns out to anticipate two things we thought we would have to solve. V_RNXDIR and V_RX3DIR are separate variables, so mixed RINEX 2 and RINEX 3 staging is designed in rather than needing new globbing. And V_PCV=I14 with V_MYATX=I14.ATX means the model discipline that section 1.4 insists on is already encoded, not something an operator has to remember. All eight missing scripts resolve. Three are dropped as unnecessary when running from staged products. Four map onto 5.4 names -- POLUPDH to POLUPD, ORBGENH to ORBGEN, RXOBV3_H to RXOBV3, RNXSMT_H to RNXSMT_P -- and the H suffix is on the script only, so panel lookups are unaffected. The eighth, PRETAB, does not exist in 5.4 in any form; 5.4's own stock RNX2SNX runs ORBMRG then ORBGEN where 5.2 ran PRETAB then ORBGENH, so follow that chain rather than inventing a replacement. The real gap: PHIVOL_REL.PCF wants V_REFINF=IGS14 and V_REFPSD=IGS14, and 5.4's REF54 ships IGS20 only. The five I14 frame files are in the 5.2 capture and must be staged. Without them the run either fails or falls back to IGS20 silently -- which is the I14/I20 confound arriving through the back door, in an exercise whose entire purpose is to avoid that confound. Also records what was nearly a seventh false absence in the same session: a find piped through head -8 returned no orbits for GPS weeks 2364-2368, and the truncation was briefly read as absence. There are 76. The T420's claim was correct and mine was the artefact.
… to transform frames
Two additions from checking published IGS14-to-IGS20 guidance against
what this exercise actually needs.
The antenna table is named differently between versions. 5.2 resolves
{V_PCVINF}_{V_PCV} to PCV_I14.PCV; 5.4 to ANTENNA_I20.PCV. Porting the
PCF therefore needs V_PCVINF changed from PCV to ANTENNA, at which point
V_PCV=I14 resolves to ANTENNA_I14.PCV -- which 5.4 already installs.
Left unchanged, Bernese looks for a filename that does not exist in the
5.4 tree. Section 4.3 had missed this.
Also notes that 002 ATX2PCV runs in both PCFs, so the table is built at
runtime from the ATX and V_MYATX must agree with V_PCV, or the table is
built from one model and selected by another. REF54 holds four antenna
tables side by side and V_PCV is the only thing keeping them apart, so
the loaded table should be confirmed from BPE output rather than assumed
from the variable.
On the transformation approach: the published method -- 7-parameter
Helmert, epoch propagation to a common epoch, consistent ATX per dataset
-- is correct for reconciling two finished products in different frames.
It is the wrong method for this exercise, because it would leave
transform residuals entangled with the PCF effect being measured.
Running both sides under I14 removes the difference rather than
correcting for it, and costs nothing since 5.4 ships ANTENNA_I14.PCV.
The method is recorded as applying to the later LUZON-versus-PAGENET
comparison, which is genuinely cross-frame.
The external sources do corroborate the magnitude section 1.4 asserts:
antenna calibration changes produce centimetre-level offsets up to ~3 cm
concentrated in the Up component, independent of the frame translation,
which is sub-centimetre globally. That figure previously rested on a
single assessment.
Everything needed to reproduce the 5.4 LUZON environment from the copied 5.2 production set, as scripts rather than as hand-edits nobody can re-derive. stage_luzon_campaign.sh copies the 31-day window into the 5.4 tree: 741 RINEX 2 and 274 RINEX 3 observations into the separate directories the PCF addresses via V_RNXDIR and V_RX3DIR, 85 orbit and 31 ION products for GPS weeks 2364-2368, the eight LUZON station files, I14.ATX, and the five IGS14 reference-frame files that 5.4 does not ship. That last item is the one real gap: without them the run either fails or falls back to IGS20 silently, which is the I14/I20 confound arriving through the back door. Its first dry run reported every file missing and every count zero. LOADGPS.setvar exports its own $SRC and had overwritten the variable, so every source path resolved under the Bernese tree. Renamed LUZON_SRC, with a comment: Bernese exports a large set of short names and none of them are safe to reuse. adapt_luzon_pcf.py derives LUZON_DLY.PCF from PHIVOL_REL.PCF and is verified to reproduce the deployed file byte-for-byte. It drops seven PIDs, renames four scripts and repairs the four WAIT lists the drops break, then refuses to write if any dangling WAIT remains -- a WAIT on an undefined PID does not fail loudly, it makes the BPE wait forever. One correction to the plan recorded in the runbook: PRETAB is deleted, not substituted. 5.2 chained ORBMRG(111) to PRETAB(112) to ORBGENH(113); 5.4's own stock RNX2SNX chains ORBMRG(111) to ORBGEN(112). ORBMRG was already present at 111, so the substitution the runbook suggested would have created a duplicate. ORBGEN now waits on 101 and 111 directly. R2S_SAV, R2S_DEL and BPE_CLN are dropped deliberately rather than incidentally. That pair is precisely why the DOY 121-151 solutions appeared to be missing on 2026-08-04 -- they archive to SAVEDISK and then wipe the campaign. For a first run, leaving results and BPE logs where they can be inspected is worth more than archiving them. Also adds the LUZON driver, campaign GEN/ with SESSIONS.SES (readiness gap #2, the stall hit during the training week) and ANTENNA_I14.PCV -- I14, where the EXAMPLE campaign carries I20.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (16)
📝 WalkthroughWalkthroughThis PR adds LUZON reprocessing utilities, transfer automation, and validator updates. It also revises the runbook, provenance, reply notes, and session log to reflect corrected data availability, PCF compatibility limits, staging requirements, archive-transfer results, and current Bernese 5.4 blockers. ChangesLUZON reprocessing and archive operations
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
… recorded
The plan to adapt PHIVOLCS' 5.2 PCF was wrong in its premise. The PCF
file format changed between 5.2 and 5.4 -- fixed-column with a ruler
line versus free-form KEY=VALUE -- and handing 5.4 a 5.2 PCF segfaults
the menu program rather than reporting a parse error, so the
incompatibility does not announce itself. No amount of script renaming
or WAIT repair bridges that.
derive_luzon_pcf.py starts from 5.4's own RNX2SNX.PCF instead, which is
the same workflow and whose R2S_* option directories ship with 5.4. It
applies fourteen variable overrides and nothing structural, and refuses
to write if any process row lacks CPU= so the format error cannot recur
silently. PAGENET_DLY.PCF would also have worked but needs the PGN_*
option directories, which are still only on the T420.
fetch_igs_products.sh downloads long-name IGS products from BKG. This
corrects section 1.1c, which claimed FTP_DWLD could be dropped because
every product was already local. They are local in 5.2-era legacy naming
and 5.4 reads long-name, so they were present and unusable at the same
time. Presence had been verified; usability had not. The ERP is weekly
rather than daily, so fetching one for a given day means locating the
week that contains it.
The fifth attempt reached PID 001 and stopped on one missing file.
Station files copied into the campaign, ANTENNA_I14.PCV confirmed in
use, orbit and ERP copied. R2S_COP generates IAR_*.OSB by running
BIA2OSB over a bias SINEX, and that input -- IGS0OPSFIN_2025121*_OSB.BIA
-- does not exist at BKG, whose IGS final set carries only CLK, SP3, ERP
and SUM. AIUB is unreachable from this machine and CDDIS needs an
Earthdata login. Blanking V_OSBFIL does not help; the requirement is in
the script's key list. Her 5.2 run used DCB files rather than OSB, so
this is a 5.4 requirement and not inherited from her configuration.
Three tool defects the attempts exposed, all fixed and tested.
find_dangling_waits knew only the WAIT= dialect, so on a 5.2 PCF it
parsed zero PIDs and zero WAITs and reported zero dangling -- signing
off a file with four broken WAIT lists. The REWAIT regex captured four
fields where the row has three. And BPE_CAMPAIGN must be '${P}/LUZON'
single-quoted, because startBPE tests it with -d relative to the current
directory, which makes the stock drivers silently CWD-dependent.
Records the two-machine corrections, the PR merges, and five attempts at a first Bernese 5.4 run. Two conclusions this machine reached were wrong and are corrected. The LUZON reference solutions were never missing -- SAVEDISK holds all 365 daily finals for 2025, and the campaign directory looked empty of them only because the PCF archives there and then cleans the campaign. Nor were seven fiducials missing: all nine are in DATAPOOL/RINEX3 as RINEX 3 long names, and the advice to request BASC from PHIVOLCS as a station absent from every archive was wrong on both counts. DATAPOOL has fourteen subdirectories and both sessions had searched one. Both are the same defect, which by day's end had seven instances: searching with one convention and reading zero results as absence. Also records that a T420 commit was nearly lost to a merge race, having been authored on a PR head that had already merged and been deleted; recovered by cherry-pick with both corrections now present. The 5.4 attempts failed on a wrong premise rather than wrong details. The PCF file format changed between 5.2 and 5.4, and 5.4 answers a 5.2 PCF with a segmentation fault rather than a parse error, so four attempts passed before the incompatibility surfaced. The run now reaches PID 001 and stops on one satellite bias product that BKG does not carry, AIUB cannot be reached for, and CDDIS keeps behind a login. Three tool defects surfaced by use, all fixed: find_dangling_waits knew only one of the two PCF dialects and so reported a clean bill having parsed nothing; a regex captured four fields where the row has three; and BPE_CAMPAIGN as a bare name is resolved relative to the working directory, which makes the stock Bernese drivers silently CWD-dependent. 198 tests pass, from 128 at the week's start.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Session log §17, the LUZON 5.4 reprocessing runbook, and the tooling built
around it. Docs and scripts; the one library change is a bug fix with tests.
Two conclusions this machine reached were wrong
The LUZON reference solutions were never missing. §14 concluded none covered
the raw RINEX window.
SAVEDISK/2025/SOL/holds all 365 daily finals for2025 — 31 of 31 days in the window.
CAMPAIGN/LUZON/SOL/looked empty becausePHIVOL_REL.PCFarchives there (902 R2S_SAV) then cleans the campaign(
903 R2S_DEL). Normal operation.Nor were seven fiducials missing. All nine are in
DATAPOOL/RINEX3/asRINEX 3 long names. The advice to request BASC from PHIVOLCS as "a station
absent from every archive" was wrong on both counts.
DATAPOOL/has fourteensubdirectories; both sessions had searched one.
Once framed correctly the check took one command:
Both are the week's dominant defect — searching with one convention and
reading zero as absence — now at seven instances. §1.5 of the runbook
tabulates them.
Five attempts at a first 5.4 run
BPE_CAMPAIGN="LUZON"— bare names resolve relative to CWD"${P}/LUZON"— Perl interpolated a nonexistent variableThe premise was wrong, not the details. The PCF file format changed
between versions — fixed-column with a ruler versus free-form
KEY=VALUE;— and5.4 answers a 5.2 PCF with a segmentation fault, not a parse error, so four
attempts passed before it surfaced.
derive_luzon_pcf.pynow starts from 5.4'sown
RNX2SNX.PCFand refuses any process row lackingCPU=.Attempt 5 copied the station files, confirmed
ANTENNA_I14.PCVin use, andstaged orbit and ERP before stopping on
IAR23644.OSB. Its input(
IGS0OPSFIN_2025121*_OSB.BIA) isn't at BKG, AIUB is unreachable from gps3, andCDDIS requires an Earthdata login. One credential or one file unblocks it.
Three tool defects, found by use
find_dangling_waits()knew only theWAIT=dialect. On a 5.2 PCF itparsed zero PIDs and zero WAITs, reported "0 dangling", and signed off a
file with four broken WAIT lists — the eighth instance of a check reporting
success without inspecting anything, in the function written to prevent it.
REWAITregex captured four fields where the row has three.BPE_CAMPAIGNas a bare name is resolved with-drelative to CWD, makingthe stock Bernese drivers silently directory-dependent.
Also in here
stage_luzon_campaign.sh,fetch_igs_products.sh,derive_luzon_pcf.pydeliverables_tracker.md,bernese_orchestration_explainer.md(its
${U}/CPU/path does not exist), andgnss_automation_roadmap.mdreclassified as a design backlog
Verification
Summary by CodeRabbit
New Features
Bug Fixes
Documentation