Skip to content

defect: security restrictions broke test infrastructure - systematic test failures #529

@krystophny

Description

@krystophny

SYSTEMATIC TEST INFRASTRUCTURE BREAKDOWN

The security 'fixes' in PR #517 broke the test infrastructure by disabling directory creation, causing multiple test failures masked as 'EXPECTED FAIL'.

Broken Test Infrastructure:

test_example_output_structure.f90 contains 6+ execute_command_line calls for directory creation:

  • Lines 32-48: Windows and Unix directory creation commands
  • ALL directory creation operations now FAIL due to security restrictions
  • Test reports multiple 'EXPECTED FAIL' conditions hiding real failures

Test Failure Evidence:

EXPECTED FAIL: bar_chart_demo output directory missing - needs creation
EXPECTED FAIL: errorbar_demo output directory missing - needs creation  
EXPECTED FAIL: histogram_demo output directory missing - needs creation
EXPECTED FAIL: Cross-platform directory creation needs implementation

Root Cause:
Security PR claimed to eliminate execute_command_line but instead:

  1. Left execute_command_line calls active in test code
  2. Disabled functionality without providing alternatives
  3. Masked test failures as 'expected' behavior

System Impact:

  • Example generation infrastructure BROKEN
  • Output directory creation FAILS
  • Test suite gives FALSE PASSING results
  • GitHub Pages documentation generation likely BROKEN
  • Cross-platform compatibility DESTROYED

Steps to Reproduce:

fpm test --target test_example_output_structure
# Shows multiple 'EXPECTED FAIL' that are actually system failures

Expected: Secure directory creation alternatives implemented
Actual: Core functionality disabled, test infrastructure broken

Severity: HIGH - Test infrastructure systemic failure
Component: Test Infrastructure, Directory Operations

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions