test: fix all ignored integration tests

[sanity]

Problem

Multiple critical integration tests are ignored/disabled, creating blind spots in our test coverage. Recent bugs (#2018, etc.) were not caught because the tests that would expose them are disabled.

Ignored Tests Inventory

apps/freenet-ping/app/tests/

test_small_network_get_issue.rs:

• test_small_network_get_failure - IGNORED
• Reason: "Test has reliability issues in CI - PUT operations timeout and gateway crashes"
• Ignored by: @sanity (feccbde, June 3, 2025)
• Tracked in: test: fix PUT timeout and gateway crashes in small network test #2023

run_app_partially_connected_network.rs:

• test_ping_partially_connected_network - IGNORED
• Reason: "Test has never worked - nodes fail on startup with channel closed errors"
• Ignored by: @sanity (feccbde, June 3, 2025)
• Tracked in: test: fix 'channel closed' startup errors in partially connected network tests #2022

run_app.rs:

• test_ping_multi_node - IGNORED

• test_ping_application_loop - IGNORED

• Reason: "this test currently fails and we are workign on fixing it" [sic]

• Ignored by: @netsirius (0131b61, June 25, 2025)

• Tracked in: test: fix multi-node ping test failures #2024

• test_ping_partially_connected_network - IGNORED (duplicate)

• Reason: "Test has never worked - gateway nodes fail on startup with channel closed errors"

• Ignored by: @sanity (feccbde, June 3, 2025)

• Tracked in: test: fix 'channel closed' startup errors in partially connected network tests #2022

run_app_blocked_peers.rs:

• test_ping_blocked_peers - IGNORED
• test_ping_blocked_peers_simple - IGNORED
• test_ping_blocked_peers_solution - IGNORED
• Various reasons including "fix me"
• Tracked in: test: fix blocked peers test suite (update propagation through gateway) #2025

test_50_node_operations.rs:

• test_50_node_operations - IGNORED
• Reason: "large scale test - run manually"
• Tracked in: test: evaluate enabling large scale test (50 nodes) in CI #2026

Root Causes

The ignore reasons reveal systemic issues:

1. "Test has never worked" - Tests were committed broken (test: fix 'channel closed' startup errors in partially connected network tests #2022)
2. "Gateway crashes" / "channel closed errors" - Real bugs in core functionality (test: fix 'channel closed' startup errors in partially connected network tests #2022, test: fix PUT timeout and gateway crashes in small network test #2023)
3. "PUT operations timeout" - Network layer issues (test: fix PUT timeout and gateway crashes in small network test #2023)
4. "Reliability issues in CI" - Flaky tests masking real problems (test: fix PUT timeout and gateway crashes in small network test #2023)
5. No enforcement - Nothing prevents ignoring tests without tracking (test: establish policy and enforcement to prevent future ignored tests #2027)

The Vicious Cycle

Real bug exists → Test exposes it → Test fails
→ Test ignored ("flaky") → Bug stays unfixed
→ More bugs accumulate → Tests stay ignored forever

Impact

• GET operation fails when caching contracts with version-based update validation #2018: GET operation failing wasn't caught
• Unknown number of other bugs lurking
• False confidence from "all tests passing"
• Technical debt compounding

Fix Strategy

This issue has been broken down into 6 focused subissues (see below). Each subissue has detailed investigation tasks, success criteria, and proper labels.

Phase 1: Triage and Reproduce (Weeks 1-2)

• test: fix 'channel closed' startup errors in partially connected network tests #2022: Channel closed startup errors
• test: fix PUT timeout and gateway crashes in small network test #2023: PUT timeouts and gateway crashes
• test: fix multi-node ping test failures #2024: Multi-node ping failures
• test: fix blocked peers test suite (update propagation through gateway) #2025: Blocked peers test suite

Phase 2: Fix Underlying Bugs (Weeks 3-5)

Work on subissues above based on investigation findings.

Phase 3: Prevention and Cleanup (Week 6)

• test: establish policy and enforcement to prevent future ignored tests #2027: Implement CI enforcement to prevent future ignored tests
• test: evaluate enabling large scale test (50 nodes) in CI #2026: Evaluate large scale test automation
• Un-ignore tests as fixes are verified

Success Criteria

• All subissues below are resolved
• Zero ignored tests without explicit tracking issues
• All "never worked" tests either work or are deleted
• CI test suite actually catches regressions
• Enforcement in place to prevent recurrence

Related Issues

• GET operation fails when caching contracts with version-based update validation #2018 - GET operation bug not caught by tests
• Likely many more undiscovered

[AI-assisted debugging and comment]

[github-actions]

🤖 Auto-labeled

Applied labels:

• T-tracking (98% confidence)
• E-hard (85% confidence)
• A-developer-xp (95% confidence)
• A-networking (80% confidence)
• S-needs-reproduction (90% confidence)

Reasoning: This is a meta issue proposing a coordinated triage and fix effort for many ignored/disabled integration tests, so T-tracking is appropriate. The work spans developer experience/CI policy (A-developer-xp) and core networking/gateway bugs exposed by the tests (A-networking). Addressing all ignored tests and underlying bugs is complex and cross-cutting, so E-hard is a reasonable effort estimate. The next actionable step is to run and reproduce each ignored test failure, so S-needs-reproduction applies.

Previous labels: none

If these labels are incorrect, please update them. This helps improve the auto-labeling system.