Improve GTFS input logging with optional CSV-aware previews after archive preparation

[Copilot]

prepare_only.sh previously had no direct visibility into the unpacked GTFS inputs in workflow logs. This update adds an optional post-prepare log step that inspects GTFS .txt (CSV) files and prints a structured preview for both current and next datasets.

• Workflow placement

  • Added a new step immediately after MT select archive (./prepare_only.sh) in shared-overwrite/.github/workflows/mt-sync-code-data.yml.
  • Targets:
    • agency-parser/input/gtfs/
    • agency-parser/input/gtfs_next/ (when present)

• CSV-focused log output

  • For each .txt file found, logs:
    • total line count
    • inferred column count (from header)
    • header row
    • first 100 lines with line numbers
    • explicit truncation note when file length exceeds 100 lines
  • Uses grouped logs (::group:: ... ::endgroup::) per file for scanability.

• Non-blocking behavior

  • Step is explicitly optional via continue-on-error: true.
  • Missing directories/files are logged as informational; workflow proceeds.

- name: MT log GTFS CSV previews
  continue-on-error: true
  run: |
    for DIR in "agency-parser/input/gtfs/" "agency-parser/input/gtfs_next/"; do
      # ...
      TOTAL_LINES=$(wc -l < "$FILE")
      COLUMN_COUNT=$(awk -F',' 'NR==1{print NF; exit}' "$FILE")
      HEADER=$(head -n 1 "$FILE")
      nl -ba "$FILE" | head -n 100
    done

[gemini-code-assist]

Code Review

This pull request adds a new step to the GitHub Actions workflow to log previews of GTFS CSV files. Feedback was provided to address a potential pipeline failure caused by using nl -ba "$FILE" | head -n 100 on files with more than 100 lines, which triggers a SIGPIPE error under pipefail. Reversing the command to head -n 100 "$FILE" | nl -ba is recommended to ensure the workflow runs successfully.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[mmathieum]

@copilot what about this #805 (comment) ?

[Copilot]

@copilot what about this #805 (comment) ?

Good catch. I addressed that thread by switching the preview pipeline to head -n 100 "$FILE" | nl -ba, so files over 100 lines no longer trip pipefail and stop processing subsequent files. Fixed in commit 8579ca6.

[mmathieum]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds a new step to the GitHub Actions workflow to log previews of GTFS CSV/TXT files, including their headers, line counts, and column counts. The feedback suggests stripping trailing carriage returns (\r) from headers to prevent rendering issues with CRLF line endings, and sorting the files alphabetically to ensure deterministic log output.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[mmathieum]

@copilot address those #805 (comment) #805 (comment)

[Copilot]

@copilot address those #805 (comment) #805 (comment)

Addressed in commit 7c0b375.

• stripped trailing CR from HEADER to avoid CRLF log rendering issues
• sorted find ... -print0 output with sort -z for deterministic .txt preview order

[mmathieum]

Tested 😍 :

• https://github.com/mtransitapps/ca-dawson-creek-transit-system-bus-android/actions/runs/28675875817/job/85049049298