Skip to content

Compare prop and rel charts with DVOACAP results#101

Merged
skyelaird merged 1 commit intomainfrom
claude/compare-charts-dvoacap-01WxwqmqukTNnbevGJ9DaCCS
Nov 17, 2025
Merged

Compare prop and rel charts with DVOACAP results#101
skyelaird merged 1 commit intomainfrom
claude/compare-charts-dvoacap-01WxwqmqukTNnbevGJ9DaCCS

Conversation

@skyelaird
Copy link
Copy Markdown
Owner

CRITICAL FINDING:
DVOACAP predicts 0-1.2% reliability when VOACAP shows 20-100% reliability, despite both showing similar strong SNR (28-45 dB). This is a 50-100x error.

Analysis Details:

  • Compared DVOACAP predictions to VOACAP reference charts (REL and SDBW)
  • Test scenario: FN74ui 14.100 MHz, 1800 UTC, SSN 81
  • SNR predictions: GOOD (within ±5 dB)
  • Path loss: GOOD (137-156 dB vs expected 135-160 dB)
  • Reliability: BROKEN (0-1% vs expected 20-100%)

The system predicts "unlikely propagation" when it should predict "excellent propagation" - making DVOACAP unusable for operational planning.

Files added:

  • VOACAP_COMPARISON_ANALYSIS.md - Full technical analysis
  • compare_voacap_charts.py - Comparison test script
  • debug_14mhz_issue.py - Debug script for 14 MHz

Related to known issues in VALIDATION_IMPROVEMENT_ANALYSIS.md

Description

A clear and concise description of what this PR does.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Performance improvement
  • Code refactoring
  • Test coverage improvement

Related Issues

Closes #(issue number)
Related to #(issue number)

Changes Made

  • Change 1
  • Change 2
  • Change 3

Testing

Describe the tests you've run to verify your changes:

Test Configuration

  • Python version:
  • Operating System:
  • NumPy/SciPy versions:

Test Results

# Command used to test
python -m pytest tests/

# Or validation command
python test_voacap_reference.py
  • All existing tests pass
  • New tests added for new functionality
  • Validation pass rate maintained or improved
  • No performance regression

Validation Checklist

  • Code follows PEP 8 style guidelines
  • Docstrings added/updated for public functions
  • Type hints added where appropriate
  • Comments added for complex logic
  • Documentation updated (README, USAGE, etc.)
  • CHANGELOG.md updated
  • No new warnings or errors introduced

Performance Impact

  • No performance impact
  • Performance improved (provide benchmarks)
  • Minor performance regression (justified because...)
  • Not applicable

Benchmarks (if applicable)

Before: X seconds
After: Y seconds
Change: Z% improvement/regression

Breaking Changes

If this PR introduces breaking changes, describe:

  1. What breaks
  2. Migration path for users
  3. Why this change is necessary

Screenshots/Output (if applicable)

For dashboard changes or output format changes, include before/after screenshots or sample output.

Additional Notes

Any additional information that reviewers should know:

  • Assumptions made
  • Trade-offs considered
  • Future work needed
  • Dependencies added/removed

Reviewer Checklist

For maintainers reviewing this PR:

  • Code quality is acceptable
  • Tests are comprehensive
  • Documentation is clear
  • Validation results are acceptable
  • No security issues introduced
  • Backward compatibility maintained (or breaking changes justified)

CRITICAL FINDING:
DVOACAP predicts 0-1.2% reliability when VOACAP shows 20-100% reliability,
despite both showing similar strong SNR (28-45 dB). This is a 50-100x error.

Analysis Details:
- Compared DVOACAP predictions to VOACAP reference charts (REL and SDBW)
- Test scenario: FN74ui 14.100 MHz, 1800 UTC, SSN 81
- SNR predictions: GOOD (within ±5 dB)
- Path loss: GOOD (137-156 dB vs expected 135-160 dB)
- Reliability: BROKEN (0-1% vs expected 20-100%)

The system predicts "unlikely propagation" when it should predict "excellent
propagation" - making DVOACAP unusable for operational planning.

Files added:
- VOACAP_COMPARISON_ANALYSIS.md - Full technical analysis
- compare_voacap_charts.py - Comparison test script
- debug_14mhz_issue.py - Debug script for 14 MHz

Related to known issues in VALIDATION_IMPROVEMENT_ANALYSIS.md
@skyelaird skyelaird merged commit 905a783 into main Nov 17, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants