Home

NCEPLIBS-BUFR_PR673_Analysis.md

Error Catching and Recovery Mechanism for BUFR Library

Report Generated: October 14, 2025 Author: Anthropic Claude Sonnet 4.5 with direction from human analyst Terrence McGuinness, NOAA

Global Workflow Wiki

Welcome to the NOAA Global Workflow technical wiki. This knowledge base documents solutions, configurations, and insights for operating and developing the Global Forecast System workflow.

---

🧪 Background information of Cases used in the CTest Framework

The CTest framework provides self-contained test cases for validating individual workflow components. Each test creates an isolated environment with staged inputs from nightly stable baseline runs, enabling independent testing and validation.

C48 Fixed Atmosphere-Only Tests (ATM)

• C48_ATM-gfs_fcst_seg0

  • 120-hour deterministic forecast validation (209 output files)
  • 13 input files (atmosphere initial conditions)
  • 18 output files (forecast history files)
  • C48_ATM-gfs_fcst_seg0.yaml

• C48_ATM-gfs_atmos_prod_f000-f002

  • Atmosphere product generation test (f000, f001, f002)
  • 5 input files (forecast history from f000, f001, f002)
  • 12 output files (post-processed products)
  • C48_ATM-gfs_atmos_prod_f000-f002.yaml

C48 Coupled System Tests (S2SW)

• C48_S2SW-gfs_fcst_seg0

  • 48-hour coupled atmosphere-ocean-ice-wave forecast
  • Fixed coupled forecast test with proper restart staging
  • 17 input files (13 atmosphere ICs + 3 restarts + 1 wave prep)
  • 24 output files (18 atmos + 2 ocean + 2 ice + 2 wave)
  • Key fix: Added H_offset = '-6H' for staging restart files from previous cycle
  • C48_S2SW-gfs_fcst_seg0.yaml

• C48_S2SW-gfs_ocean_prod_f006 - Ocean product generation at forecast hour 6

  • 2 input files (ocean forecast at f006)
  • 2 output files (ocean products)
  • C48_S2SW-gfs_ocean_prod_f006.yaml

• C48_S2SW-gfs_ice_prod_f006

  • Ice product generation at forecast hour 6
  • 2 input files (ice forecast at f006)
  • 2 output files (ice products)
  • C48_S2SW-gfs_ice_prod_f006.yaml

C48 Ensemble Tests (S2SW_gfs)

• C48_S2SWA_gefs-gefs_fcst_mem001_seg0
• GEFS ensemble member 001 coupled forecast (48-hour segment)
  • Implemented GEFS ensemble member 001 forecast test
  • 17 input files with unique two-cycle pattern:
    • 13 atmosphere ICs from current cycle (12Z)
    • 3 restart files from previous cycle (06Z)
    • 1 wave prep file from current cycle (12Z)
  • 24 output files (ensemble forecast outputs)
  • GEFS requires different source cycles for ICs vs restarts
  • Special handling for mem001/ subdirectory structure
• C48_S2SWA_gefs-gefs_fcst_mem001_seg0.yaml

Framework Features:

• Self-contained test environments with isolated EXPDIR
• Input staging from STAGED_CTESTS (stable nightly runs)
• Consistent naming convention: CASE-JOB.yaml
• Comprehensive validation with input/output file verification

---

🔧 CI/CD & DevOps

GitLab CI/CD Pipeline

• GitLab-Pipeline-MultiHost-Architecture-for-Global‐Workflow
• Managing-Multiple-GitLab-Pipeline-Scripts
• Monitoring-your-HPC-CI-CD-infrastructure-with-GitLab
• Running-.after_script-Commands-on-a-Remote-Host:-Challenges-and-Solutions

Jenkins Integration

• Configuring-the-Jenkins-Controller's-Node-default-configurations
• Increasing-the-number-of-executors-on-Jenkins-Master-(built‐in)-Node
• Scaling-Number-of-agents-on-a-Jenkins-Node
• Spreading-out-executors-on-RDHPCS-head-nodes
• SCM-Timeout-Fix

GitHub & Jenkins Integration

• Using-a-GitHub-Action-with-WebHooks-to-Dispatch-a-Jenkins-Job-for-a-Specific-PR-and-pass-it-Parameters
• Using-WebHooks-and-PR-Comments-to-Launch-a-Multi‐Branch-Jenkins-Job
• Setting-up-a-mirror-of-a-GitHub-Repo-using-GitLab-Community-Edition-Server
• Proposal-for-Creating-Dedicated-emc‐bot-Project-in-NWS-EMC-GitLab

---

🤖 AI/ML & Intelligent Tools

Model Context Protocol (MCP)

• GitHub-MCP-Tools-installed-for-global‐workflow-software-development-and-how-they-work
• Global‐workflow-RAG-added-to-MCP-server
• RAG-enhanced-MCP-server-configured-and-all-8-tools-are-now-available
• MCP-RAG-Development-Status
• Differences-and-Similarities-between-MCP-(Model-Context-Protocol)-and-RAG-(Retrieval‐Augmented-Generation)-in-agentic-LLM-pipeline
• MASSIVE-IMPROVMENT
• Opps-‐-no-RAG-no-go

AI Development Tools

• Anthropic-Claude-Sonnet-4-has-the-big-picture-ahead-of-me
• Customizing-GitHub's-built‐in-Copilot's-PR-Feature
• Appreciated-Collaboration
• Thanks-Devin!!
• Thanks-Devin

---

🌊 Workflow Management Systems

Rocoto Workflow Engine

• Rocoto-Example
• Tracking-UNKNOWN-states-in-Rocoto-and-suggested-updates
• "Resource-temporarily-unavailable"-when-using-rocotorun
• Fixing-the-Resource-Fork-Error-in-Rocoto-Check-Python-Script
• PID-created-and-release-on-each-call-to-rocotostat

CROW & EcFlow

• CROW-workflow-definition-explained
• Polymorphism-examples-in-CROW
• Undocumented-support-of-EcFlow-in-public-release-of-fv3gfs
• AQM‐Workflow-and-Global-Workflow-(Community-‐-Operations-version)-for-Generating-EcFlow

🌐 Weather Modeling & Configuration

Model Configuration

• How-are-IC-specified-for-Free-Forecast-Jobs
• Detailed-Analysis-of-HOMEDIR
• Prate_ave-in-atmospheric-sfc-history-files-accumulates-while-cycling
• Lingering-restart-files-can-cause-re‐ran-cases-to-fail

---

💻 HPC System Administration

System Configuration

• Getting-bash-for-GitLab-Runners-to-be-a-login-shell
• Why-Admins-Configure-Role-Accounts-with-login_shell=off
• Is-VAST-like-PFS?

---

🔬 Research & Theory

Scientific Computing

• Discrete-Memetic-Operators-advancing-electrodynamic-field-theory

---

🐛 Development & Debugging

Bug Fixes & Solutions

• Deleted-suggestion-got-back-make-it-a-bug-...

Development Process

• Development-Processes-and-Ideas

---

📚 Quick Reference

Most Viewed Topics:

• CI/CD Pipeline Architecture
• Rocoto Workflow Management
• MCP/RAG Integration
• Jenkins Configuration
• HPC System Setup

Latest Updates:

• MCP Server RAG Enhancement
• GitLab Multi-Host Architecture
• AI-Assisted Development Tools

---

This wiki is actively maintained. Last organized: October 2025