Architecture and Design

Architecture and Design -- Apotropaios Firewall Manager

System Architecture Overview

Apotropaios follows a strict 5-layer dependency architecture. Each layer may only import from layers below it. There are zero circular imports across the entire 35-file, 14,545-line codebase. This is not just convention -- it is verified by mypy --strict on every commit.

The architecture is designed around three principles: (1) defense-in-depth input validation at every trust boundary, (2) backend-agnostic rule management that works identically across all 5 firewall backends, and (3) fail-safe operations where errors are caught, logged, and propagated without leaving the system in an inconsistent state.

Layer Dependency Diagram

graph TD
    subgraph "Layer 5 -- User Interface (1,107 + 790 + 470 + 57 = 2,574 lines)"
        CLI["cli.py -- 18 commands, argparse<br/>Progressive help, global option pre-extraction"]
        MAIN["__main__.py -- Entry point<br/>Python version check, delegates to cli.main()"]
        MENU["menu/main.py -- 8-option interactive menu<br/>ExpiryMonitor daemon thread, cancel-aware input"]
        HELP["menu/help_system.py -- Tier 2 help<br/>Decorator-registered per-command help functions"]
    end

    subgraph "Layer 4 -- Engine (555 + 360 + 277 + 267 + 294 + 240 + 216 + 252 = 2,520 lines)"
        ENGINE["rules/engine.py -- Lifecycle orchestrator<br/>validate → UUID → backend → index → state"]
        INDEX["rules/index.py -- Persistent pipe-delimited index<br/>27-field records, atomic write, thread-safe"]
        STATE["rules/state.py -- TTL tracking<br/>StateEntry dataclass, expiry calculation"]
        IMPEXP["rules/import_export.py -- Bulk operations<br/>KV file parsing, dry-run, checksum verification"]
        BACKUP["backup/backup.py -- Archive creation<br/>tar.gz + SHA-256 sidecar, retention enforcement"]
        RESTORE["backup/restore.py -- Recovery<br/>Pre-restore safety backup, path validation"]
        IMMUTABLE["backup/immutable.py -- Snapshots<br/>chattr +i, SHA-256 integrity verification"]
        INSTALL["install/installer.py -- Package management<br/>apt/dnf/pacman dispatch"]
    end

    subgraph "Layer 3 -- Firewall Backends (190 + 279 + 695 + 402 + 353 + 327 + 386 = 2,944 lines)"
        BASE["firewall/base.py -- ABC with 12 abstract methods<br/>TypeError on missing implementation"]
        COMMON["firewall/common.py -- Registry + dispatch<br/>Auto-registration at import, set_backend/require_backend"]
        IPT["firewall/iptables.py -- 725 lines<br/>Compound actions: separate LOG + terminal rules<br/>_build_match_args core builder"]
        NFT["firewall/nftables.py -- 478 lines<br/>Single expression for compound actions<br/>Auto-create tables/chains"]
        FWD["firewall/firewalld.py -- 440 lines<br/>Zone-aware rich rules, --permanent + --reload<br/>Reset iterates ALL zones"]
        UFW["firewall/ufw.py -- 400 lines<br/>Simplified syntax, --force on all commands"]
        IPS["firewall/ipset.py -- 432 lines<br/>Set management + iptables cross-references<br/>Entry validation per set type"]
    end

    subgraph "Layer 2 -- Detection (433 + 519 = 952 lines)"
        OSDET["detection/os_detect.py -- 4 fallback methods<br/>/etc/os-release → lsb_release → redhat-release → uname"]
        FWDET["detection/fw_detect.py -- 5 backend probes<br/>Binary check → version parse → systemctl state"]
    end

    subgraph "Layer 1 -- Core (738 + 879 + 1247 + 727 + 675 + 504 = 4,809 lines)"
        CONST["core/constants.py (746 lines<br/>27 ErrorCodes, 8 StrEnums, 18 compiled regex<br/>frozensets, NamedTuples, all config values"]
        ERRORS["core/errors.py -- 897 lines<br/>25 exception classes, CleanupStack (LIFO)<br/>SignalHandler, retry/fallback utilities"]
        VALID["core/validation.py -- 1,256 lines<br/>27 whitelist validators + sanitizer<br/>Compound action parsing, path traversal detection"]
        LOG["core/logging.py -- 729 lines<br/>4-family credential masking (LogSanitizer)<br/>Dual output, correlation IDs, secure rotation"]
        SEC["core/security.py -- 678 lines<br/>FileLock (fcntl.flock + stale detection)<br/>SHA-256 checksums, UUID gen, sensitive scrubbing"]
        UTIL["core/utils.py -- 503 lines<br/>UTC timestamps, human formatting<br/>KV file I/O, parallel_exec, command check"]
    end

    CLI --> ENGINE
    CLI --> COMMON
    CLI --> BACKUP
    CLI --> INSTALL
    MAIN --> CLI
    MENU --> ENGINE
    MENU --> COMMON
    HELP -.-> |no upward deps| HELP

    ENGINE --> INDEX
    ENGINE --> STATE
    ENGINE --> COMMON
    ENGINE --> VALID
    INDEX --> SEC
    BACKUP --> SEC
    RESTORE --> BACKUP
    IMMUTABLE --> BACKUP
    IMPEXP --> ENGINE

    COMMON --> BASE
    IPT --> BASE
    NFT --> BASE
    FWD --> BASE
    UFW --> BASE
    IPS --> BASE

    OSDET --> CONST
    FWDET --> CONST

    ERRORS --> CONST
    VALID --> CONST
    VALID --> ERRORS
    LOG --> CONST
    SEC --> CONST
    SEC --> ERRORS
    UTIL --> CONST

Loading

Rule Lifecycle -- Complete Data Flow

This diagram shows the exact call chain when a user creates a rule. Every function call, every validation check, every file write is shown.

flowchart TD
    A["User: sudo python3 apotropaios.py add-rule<br/>--protocol tcp --dst-port 443 --action accept"] --> B["cli.py: main()"]
    B --> C["Pre-extract --log-level, --backend<br/>from argv (position-independent)"]
    C --> D["argparse: parse remaining argv<br/>→ args.command = 'add-rule'"]
    D --> E["_initialize(): logging → errors → security<br/>→ OS detect → FW detect → backends<br/>→ rule_index.init() → backup init"]
    E --> F["_dispatch() → _cmd_add_rule()"]

    F --> G["Build rule_params dict from args<br/>Filter out None values"]
    G --> H["rules/engine.py: rule_create(params)"]

    H --> I["Validate direction: validate_rule_direction()"]
    I --> J["Validate action: validate_rule_action()<br/>Handles compound: 'log,drop' → split, verify<br/>each in ALL_ACTIONS, reject double-terminal"]
    J --> K["Validate duration_type, TTL if temporary"]
    K --> L["Validate optional: protocol, IPs, ports<br/>conn_state, log_prefix, rate_limit, description"]
    L --> M["Generate UUID: security.generate_uuid()<br/>→ uuid.uuid4() → 'a1b2c3d4-e5f6-...'"]
    M --> N["Set tracking comment:<br/>params['comment'] = 'apotropaios:a1b2c3d4-e5f6-...'"]

    N --> O["firewall/common.py: fw_add_rule(params)"]
    O --> P["require_backend() → IptablesBackend"]
    P --> Q["iptables.py: add_rule(params)"]

    Q --> R["_parse_compound_action('accept')<br/>→ non_terminals=[], terminal='accept'"]
    R --> S["_build_match_args(params)<br/>→ chain='INPUT', args=['-p','tcp','--dport','443',<br/>'-m','comment','--comment','apotropaios:a1b2...']"]
    S --> T["subprocess.run(['iptables','-A','INPUT',<br/>'-p','tcp','--dport','443',...,'-j','ACCEPT'],<br/>shell=False, timeout=30, capture_output=True)"]

    T --> U{Backend success?}
    U -->|No| V["RuleApplyError → propagates to CLI<br/>→ stderr: 'Error: ...'<br/>→ exit(ErrorCode.RULE_APPLY_FAIL = 23)"]
    U -->|Yes| W["Build 27-field record dict<br/>rule_id, backend, direction, action, protocol,<br/>src_ip, dst_ip, ports, interface, chain, table,<br/>table_family, zone, set_name, conn_state,<br/>log_prefix, log_level, limit, limit_burst,<br/>duration_type, ttl, description, state,<br/>created_at, activated_at, expires_at"]

    W --> X["rule_index.add(record)<br/>→ threading.Lock acquire<br/>→ add to _rules dict + _order list<br/>→ save() → write tmp file → os.replace() atomic<br/>→ chmod 0o600"]
    X --> Y["rule_state.set(rule_id, 'active', 'permanent', 0)<br/>→ create StateEntry<br/>→ save to rule_state.dat (atomic)"]
    Y --> Z["Return UUID to _cmd_add_rule()"]
    Z --> AA["stderr: 'Rule created: a1b2c3d4-e5f6-...'<br/>→ exit(ErrorCode.SUCCESS = 0)"]

Loading

Detection Pipeline

flowchart LR
    subgraph "OS Detection -- 4 Fallbacks"
        OR["/etc/os-release<br/>Parses ID, NAME, VERSION_ID, ID_LIKE<br/>File size capped at 64KB"]
        OR -->|fail| LSB["/etc/lsb-release<br/>Ubuntu/Debian fallback<br/>DISTRIB_ID, DISTRIB_RELEASE"]
        LSB -->|fail| RH["/etc/redhat-release<br/>RHEL family fallback<br/>Single-line regex parse"]
        RH -->|fail| UN["platform.uname()<br/>Always available<br/>Minimal detail"]
    end

    subgraph "FW Detection -- Per Backend"
        direction TB
        BP1["1. shutil.which(binary)<br/>O(1) PATH lookup"]
        BP2["2. binary --version<br/>Backend-specific regex<br/>5-second timeout"]
        BP3["3. systemctl is-active service<br/>Returns 'active' or 'inactive'"]
        BP4["4. systemctl is-enabled service<br/>Returns 'enabled' or 'disabled'"]
        BP1 --> BP2 --> BP3 --> BP4
    end

    subgraph "Family Resolution"
        FM["OS_FAMILY_MAP lookup<br/>os_id → family<br/>ID_LIKE fallback for derivatives"]
    end

Loading

Backup and Recovery Data Flow

flowchart TD
    subgraph "Backup Creation"
        BC1["Create staging dir in temp<br/>(secure_dir, 0o700)"]
        BC2["For each installed FW backend:<br/>fw.save(staging/backend.conf)<br/>Parallel via parallel_exec()"]
        BC3["Copy rule_index.dat<br/>Copy rule_state.dat"]
        BC4["Write manifest.json:<br/>timestamp, version, label,<br/>backends list, file checksums"]
        BC5["tarfile.open(path, 'w:gz')<br/>Add all staging files"]
        BC6["SHA-256 sidecar:<br/>sha256sum archive > archive.sha256"]
        BC7["Retention: list backups sorted by mtime<br/>Delete oldest beyond MAX_RETAINED (20)"]
        BC1 --> BC2 --> BC3 --> BC4 --> BC5 --> BC6 --> BC7
    end

    subgraph "Restore Process"
        RS1["Verify SHA-256 checksum<br/>hmac.compare_digest()"]
        RS2["Create pre-restore backup<br/>(safety net for rollback)"]
        RS3["Extract to temp dir<br/>tarfile.open(path, 'r:gz')"]
        RS4["Validate ALL paths in archive:<br/>Reject any containing '..'<br/>Reject absolute paths"]
        RS5["Read manifest.json<br/>Determine which backends to restore"]
        RS6["For each backend in manifest:<br/>fw.load(extracted_config)<br/>(iptables-restore, nft -f, etc.)"]
        RS7["Copy rule_index.dat + rule_state.dat<br/>back to rules directory"]
        RS1 --> RS2 --> RS3 --> RS4 --> RS5 --> RS6 --> RS7
    end

    subgraph "Immutable Snapshots"
        IM1["Create normal backup"]
        IM2["Copy archive to immutable/ subdir"]
        IM3["Generate .integrity file<br/>Contains SHA-256 hash"]
        IM4["subprocess.run(['chattr','+i',file])<br/>Sets Linux immutable attribute<br/>(ext2/3/4/btrfs only)"]
        IM1 --> IM2 --> IM3 --> IM4
    end

Loading

Input Validation Pipeline

flowchart LR
    INPUT["Raw User Input<br/>(CLI args or menu prompts)"] --> SANITIZE["sanitize_input()<br/>1. Reject None/non-string<br/>2. Strip whitespace<br/>3. Remove HTML tags<br/>4. Whitelist regex filter<br/>5. Truncate to 4096 chars"]
    SANITIZE --> VALIDATE{"Type-specific<br/>validator"}

    VALIDATE -->|IP| VIP["validate_ip()<br/>→ validate_ipv4() or validate_ipv6()<br/>Regex format + octet range check"]
    VALIDATE -->|Port| VP["validate_port()<br/>Parse to int, 1-65535"]
    VALIDATE -->|Protocol| VPROT["validate_protocol()<br/>Whitelist: tcp/udp/icmp/icmpv6/sctp/all"]
    VALIDATE -->|Path| VPATH["validate_file_path()<br/>1. Null byte rejection<br/>2. Path traversal (..) rejection<br/>3. Shell metachar rejection<br/>4. Regex whitelist"]
    VALIDATE -->|Action| VACT["validate_rule_action()<br/>Single or compound<br/>Split on comma, classify<br/>terminal vs non-terminal<br/>Reject double-terminal"]
    VALIDATE -->|UUID| VUUID["validate_rule_id()<br/>UUID v4 regex: 8-4-4-4-12 hex"]

    VIP --> SAFE["Validated parameter<br/>→ engine → backend<br/>→ subprocess list-form"]
    VP --> SAFE
    VPROT --> SAFE
    VPATH --> SAFE
    VACT --> SAFE
    VUUID --> SAFE

Loading

Error Handling Architecture

flowchart TD
    subgraph "Exception Hierarchy -- 22 Classes"
        AE["ApotropaiosError<br/>code: ErrorCode<br/>message: str<br/>context: dict"]
        AE --> VE["ValidationError (40)"]
        AE --> SE["SanitizationError (41)"]
        AE --> RE["RuleError"]
        AE --> FE["FirewallError"]
        AE --> BE["BackupError (30)"]
        AE --> LE["LockError (60)"]
        AE --> IE["IntegrityError (70)"]
        AE --> SIG["SignalReceivedError (80)"]

        RE --> RNF["RuleNotFoundError (22)"]
        RE --> RAE["RuleApplyError (23)"]
        RE --> RRE["RuleRemoveError (24)"]
        RE --> RIE["RuleInvalidError (20)"]
        RE --> REE["RuleExistsError (21)"]
        RE --> RIM["RuleImportError (25)"]

        FE --> FNFE["FirewallNotFoundError (11)"]
        FE --> FNRE["FirewallNotRunningError (12)"]
        FE --> FIE["FirewallInstallError (13)"]

        AE --> RSE["RestoreError (31)"]
        AE --> BNFE["BackupNotFoundError (32)"]

        LE --> LTE["LockTimeoutError (61)"]
    end

    subgraph "CleanupStack -- LIFO Execution"
        CS1["register(func, description)"]
        CS2["Signal received<br/>or normal exit<br/>or explicit call"]
        CS3["execute_all():<br/>1. Acquire lock<br/>2. Check recursion guard<br/>3. Execute in LIFO order<br/>4. Continue on handler failure<br/>5. Log each step"]
        CS1 --> CS2 --> CS3
    end

    subgraph "Signal Handler"
        SH1["SIGTERM (15) → exit(143)"]
        SH2["SIGINT (2) → abort operation (interactive) / exit(130) (headless)"]
        SH3["SIGHUP (1) → exit(129)"]
        SH1 --> CS2
        SH2 --> CS2
        SH3 --> CS2
    end

Loading

SIGINT handling is scope-aware. SignalHandler.interruptible() is a nestable context scope entered for the duration of an interactive menu session: while at least one scope is active, SIGINT raises KeyboardInterrupt into the running frame, the current operation (for example a package install) is aborted, and the menu layer catches the exception and returns control to the menu -- the Install & Update submenu recovers in place, and any other operation recovers to the main menu. When no scope is active (headless CLI execution), SIGINT executes the cleanup stack and exits with status 130. SIGTERM and SIGHUP terminate through the cleanup stack regardless of scope.

Design Decision Rationale

Why stdlib-only runtime?

A firewall management tool runs as root on production servers. Every external dependency is an attack surface. Supply chain attacks on PyPI packages could compromise the firewall manager itself. By using only the Python standard library, the framework has zero external supply chain risk.

Why pipe-delimited flat files for the rule index?

Simplicity and auditability. The index can be inspected with cat, grepped with standard tools, and repaired with a text editor. No database, no ORM, no migration tooling, no daemon. The index is small (dozens to hundreds of rules) so O(n) iteration is negligible.

Why separate LOG + terminal rules in iptables?

iptables LOG is a non-terminating target -- it logs and continues. To "log and drop," two kernel rules with identical match criteria are needed. nftables handles this in a single expression (log drop). The difference is abstracted by the backend layer.

Why ABC for the backend interface?

If a developer adds a new backend and forgets to implement block_all(), Python raises TypeError at class definition time (when the module is imported), not at runtime when a user tries to block traffic. This moves the error from production to development.

Why correlation IDs in logs?

When multiple Apotropaios processes run concurrently (e.g., a cron job creating a backup while an admin adds a rule), the correlation ID in each log entry disambiguates which entries belong to which execution. Format: 8-byte hex string (16 characters), generated via secrets.token_hex(8).

Why default console log level WARNING?

Most users don't want to see framework initialization noise. The default WARNING level shows only operational warnings (e.g., "Blocking ALL traffic via iptables") and errors. Users who need diagnostics use --log-level info or --log-level debug. The log file always captures everything regardless of console level.

Why thread-safe singletons for rule_index and rule_state?

The ExpiryMonitor daemon thread checks expired rules every 30 seconds, reading from rule_index and rule_state. The main thread processes user commands that write to these same structures. Without threading.Lock, concurrent reads and writes would cause data races.