Troubleshooting and Selftest

Troubleshooting: Smoke & Selftest

Type: Operator Reference (Troubleshooting) Scope: nftban smoke and nftban selftest behavior, SKIP/PASS/FAIL semantics Since: v1.95.0 (smoke/selftest split) · Last verified: v1.218.7 Verified: RHEL-family and Debian-family hosts

---

Smoke vs Selftest

As of v1.95.0, NFTBan has two distinct validation commands:

Command	Purpose	Modifies state?	CI-safe?
nftban smoke	Non-destructive system verification	NO	YES
nftban selftest	Extended validation with controlled state changes	YES (ban/unban/whitelist)	NO

nftban smoke reads system state and reports PASS/SKIP/FAIL. It never bans, unbans, modifies whitelist, or changes firewall rules.

nftban selftest performs controlled mutations (ban test IP, verify count changes, unban) using RFC 5737 documentation-range IPs that do not route on the internet.

---

Result Semantics

Status	Meaning	Action needed
PASS	Test executed and succeeded	None
SKIP	Prerequisite not met — test not executed	Check if prerequisite should be present
FAIL	Test executed and failed	Investigate

SKIP is not an error. A result of PASS (6/10 PASS, 4 SKIP) means the system passed all tests it could run. The 4 skipped tests had prerequisites that were not satisfied (e.g., binary not installed, module not enabled).

The overall result is PASS if zero tests FAIL. SKIPs do not affect the overall result.

---

1. Truth tests SKIP — validator binary missing

Symptom:

nftban smoke

  truth:
    skip validator JSON (prerequisite not met: binary:nftban validate)
    skip health JSON (prerequisite not met: validator_present:nftban validate)
    ok status

Cause:

The nftban validate binary is not installed. Smoke tests T1 and T2 require it for validator JSON and health JSON validation.

Verify:

test -x /usr/lib/nftban/bin/nftban-validate
# If empty or "not found" → binary is missing

Fix:

Reinstall or upgrade the NFTBan package. The nftban validate binary should be included at /usr/lib/nftban/bin/nftban-validate.

Important: This is correct SKIP behavior, not a smoke bug. Smoke detects the missing prerequisite and skips instead of failing. All production systems should have nftban validate installed for full truth validation coverage.

---

2. Module test SKIP but module is enabled

Symptom:

Smoke reports:

botguard → SKIP (prerequisite not met: module_enabled:botguard)
loginmon → SKIP (prerequisite not met: module_enabled:login)

But health reports the module as enabled:

nftban health --json | jq '.modules.botguard'
# {"config": "enabled", "structural": "present", ...}

Cause:

Mismatch between smoke's prerequisite detection (module_enabled:* checks config files for ENABLED=true) and the actual module state as reported by the validator.

The smoke prerequisite checker reads specific config file paths:

• /etc/nftban/conf.d/<module>/main.conf.local
• /etc/nftban/conf.d/<module>/main.conf
• /etc/nftban/modules/<module>.conf.local
• /etc/nftban/modules/<module>.conf

If the module is enabled through a different config path or mechanism, smoke may not detect it.

Verify:

# What validator says
nftban health --json | jq '.modules'

# What smoke prereqs check
cat /etc/nftban/conf.d/botguard/main.conf 2>/dev/null | grep ENABLED
cat /etc/nftban/modules/botguard.conf 2>/dev/null | grep ENABLED

Expected behavior:

Condition	Smoke result
Module disabled in config	SKIP (correct)
Module enabled and working	PASS (correct)
Module enabled but smoke can't detect it	SKIP (bug — report it)
Module enabled but broken at runtime	FAIL (correct)

If mismatch occurs:

This is a bug in smoke prerequisite detection, not user misconfiguration. Report the issue with the output of nftban health --json and the contents of the module config files listed above.

---

3. nftban smoke does not run lifecycle tests

Symptom:

nftban smoke lifecycle

# Output:
NOTE: 'nftban smoke lifecycle' was moved to 'nftban selftest' in v1.95.

  nftban smoke    = non-destructive system verification (CI-safe)
  nftban selftest = extended validation with controlled state changes

Run instead:  nftban selftest lifecycle

Explanation:

In v1.95.0, lifecycle tests (ban/unban, whitelist add/remove) were moved from smoke to selftest because they modify system state. The smoke command is now exclusively non-destructive.

The old subcommands (run, test, all, lifecycle) print a migration message and exit with code 1.

Fix:

# Old (pre-v1.95)          →  New (v1.95+)
nftban smoke run            →  nftban selftest run
nftban smoke test           →  nftban selftest test
nftban smoke all            →  nftban selftest all
nftban smoke lifecycle      →  nftban selftest lifecycle
nftban smoke quick          →  nftban selftest quick
nftban smoke verify         →  nftban selftest verify

---

4. Verify smoke is non-destructive

To confirm smoke does not modify firewall state:

# Count sets BEFORE
echo "=== BEFORE ==="
nft list set ip nftban blacklist_manual_ipv4 2>/dev/null | wc -l
nft list set ip6 nftban blacklist_manual_ipv6 2>/dev/null | wc -l
nft list set ip nftban whitelist_ipv4 2>/dev/null | wc -l
nft list set ip6 nftban whitelist_ipv6 2>/dev/null | wc -l

# Run smoke
nftban smoke

# Count sets AFTER
echo "=== AFTER ==="
nft list set ip nftban blacklist_manual_ipv4 2>/dev/null | wc -l
nft list set ip6 nftban blacklist_manual_ipv6 2>/dev/null | wc -l
nft list set ip nftban whitelist_ipv4 2>/dev/null | wc -l
nft list set ip6 nftban whitelist_ipv6 2>/dev/null | wc -l

Expected: All counts identical before and after.

Verified on RHEL-family and Debian-family hosts — set counts did not change.

If counts change after smoke: This is a critical bug. Report immediately.

---

5. nftban-core not in PATH

Symptom:

command -v nftban-core
# → empty (not found)

But nftban smoke runs successfully.

Cause:

nftban-core is installed at /usr/lib/nftban/bin/nftban-core, not in /usr/sbin/ or /usr/bin/. The smoke shell shim resolves the binary via ${NFTBAN_LIB_DIR}/bin/nftban-core, which works without PATH.

Impact:

Not a runtime issue. Only affects operators who want to run nftban-core directly (e.g., nftban-core smoke --json).

Optional fix:

ln -s /usr/lib/nftban/bin/nftban-core /usr/sbin/nftban-core

---

6. When to use smoke vs selftest

Use nftban smoke for:

• CI/CD pipeline checks
• Production health verification
• Routine monitoring (cron, fleet checks)
• Verifying system integrity after updates

nftban smoke              # all tests
nftban smoke --json       # machine-readable output
nftban smoke --group=truth   # validator/health checks only
nftban smoke --group=daemon  # daemon service check only

Use nftban selftest for:

• Post-installation verification
• Debugging ban/unban behavior
• Verifying kernel enforcement lifecycle
• Lab/staging environment validation

nftban selftest quick       # 3 core commands (version, help, status)
nftban selftest lifecycle   # ban/unban + whitelist add/remove (IPv4+IPv6)
nftban selftest run         # ~32 CLI commands + lifecycle
nftban selftest verify      # smart count validation (ban count +1/-1)

---

7. Minimal health checks

Fast (2 seconds):

nftban smoke --group=daemon

Checks: daemon is active.

Standard (5 seconds):

nftban smoke --group=truth

Checks: validator JSON, health JSON, status command.

Full (10 seconds):

nftban smoke

Checks: truth + daemon + config + metrics + enabled modules.

Deep (15 seconds):

nftban smoke --deep

Checks: all above + deep assertions.

---

8. Reporting failures

When reporting smoke or selftest failures, attach:

nftban smoke --json | jq .
nftban health --json | jq .
nftban version --json 2>/dev/null | grep -v '^#'
uname -r
cat /etc/os-release | head -3

This provides: test results, health state, version, kernel, and distro — enough context to diagnose most issues without further questions.

---

Known v1.95 Behaviors

These are expected behaviors, not bugs:

• T1/T2 SKIP when nftban validate is missing. Install the binary for full truth validation. Smoke correctly skips rather than failing.

• Module SKIP on some hosts despite being enabled. Known edge case in prerequisite detection. Being tracked for fix.

• nftban smoke lifecycle prints migration message. Correct. Use nftban selftest lifecycle instead.

• Smoke is intentionally minimal. It checks system integrity, not feature completeness. Use selftest for deeper validation.

• Selftest uses RFC 5737 test IPs. Ban/unban lifecycle tests use 198.51.100.99 (IPv4) and 2001:db8::dead:beef (IPv6). These are documentation-range addresses that do not route on the internet.