Skip to content
Playbook for system hardening maintained by the #! security research team.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.nojekyll
README.md
_config.yml
index.html

README.md

Hardening Playbook

Abstract

An opinionated minimal-compromises guide to configuring a maximally secure server for high stakes use cases where privacy and security are favored over compatibility, cost, or effeciency.

This intends to be largely a showcase of the work of others and act as a starting point for researching this space.

Threat Profile

  • Target protects:
    • Automated air/ground transportation
    • Nuclear weapons
    • Electric grid
    • Medical implants
    • Dive computers
    • Secrets that could end any entity
    • Access to unlimited financial gain
    • Human lives
  • Attacker has
    • No ethics
    • unlimited funding
    • decades of patience
    • Knowledge deeper than yours of every component
    • 0-days of any currently known class
  • Attacker can
    • compromise any single point in the supply chain
    • compromise any single system
    • compromise any single individual
  • Attacker wants
    • Theft (cryptocurrency, bank accounts, stock tips, blackmail, databases)
    • Sabotage (to a company or country for any reason)
    • Chaos (May not be rational)

Design

  • Favor security and privacy over efficiency
  • Every system:
    • is treated as a single purpose immutable appliance
    • replaced not updated
  • Every component must be:
    • auditable by anyone
    • reproducible deterministically by anyone
    • audited by multiple reputable third parties.
    • fail on any unathorized physical tampering attempt
    • handle cryptographic operations in constant time
    • maintain secret keys physically separate from networks
    • have the bare minimum resources to complete its intended function

Implementation

Code Review (WIP)

Overview

Recommendations

  • Decentralized
  • Multisig
  • Coding practices

Dependency Management (WIP)

Overview

Recommendations

  • review practices
  • Signed reproducible builds must be possible
  • Code must be signed with a well-known key of author and ideally reviewer(s)
  • Consider reviews by any distribution channel maintainers
  • Always assume force pushes and tag clobbers: pin hashes
  • Assume upstreams will vanish without warning: mirror everything yourself

Background

Release Management (WIP)

Overview

Recommendations

  • Reproducible builds
  • Signing

Background

Static Application Security Testing (WIP)

Overview

Static analysis analyzes source code or compiled binaries for security flaws. A critical part of a security focused software development is using tools to help catch common human errors before code hits production.

Memory bugs are most common and where possible one should favor memory safe languages designed for security: Rust, Go, OCaml, Zig

Recommendations

Language Agnostic
  • Google CodeSearchDiggity
  • Graudit
  • LGTM
  • SonarQube
  • VisualCodeGrepper
Bash
  • shellcheck
Node.js
  • synode
C/C++
  • BOON
  • CQual
  • xg++
  • Eau Claire tool
  • MOPS
  • Split
  • Flawfinder
  • PreFast
C#
  • Puma Scan
  • .Net Security Guard
Python
  • Bandit
Ruby
  • Brakeman
  • Codesake Dawn
Java
  • SpotBugs
  • PMD
PHP
  • progpilot
  • RIPS
  • pgpcs-security-audit

Background

Sandboxing (WIP)

Overview

Recommendations

SELinux
Apparmor
Seccomp
cgroups
namespacing

Filesystem (WIP)

Everything on unix is a file, and as such filesystem mount options and permissions are one of the most effective ways to restrict what can or can't be done in a given directory.

Everything should be either a read-only filesystem like quashfs or a tmpfs. Never allow writes to root filesystem.

This is all managed via /etc/fstab

Overview

Recommendations

Mount options
Restrict /proc so users can only see their own processes
  • Usage: proc /proc proc defaults,hidepid=2 0 0
Disable suid binaries in /dev
  • Usage: udev /dev devtmpfs defaults,nosuid,noexec,noatime 0 0
Force mode 0666 in /dev/pts
  • Usage: devpts /dev/pts devpts defaults,newinstance,ptmxmode=0666 0 0
Use tmpfs for /dev/shm and restrict suid, exec, and dev
  • Usage: tmpfs /dev/shm tmpfs defaults,nodev,nosuid,noexec 0 0
Use tmpfs for /tmp and disable devices, suid binaries, and exec
  • Usage: tmpfs /tmp tmpfs nodev,nosuid,noexec,size=2G 0 0
Bind /var/tmp to /tmp and restrict suid, exec, and dev
  • Usage: /tmp /var/tmp none rw,noexec,nosuid,nodev,bind 0 0
Encryption
  • luks
  • luks + sgx
Resources
  • Cold boot attacks

Toolchain

Overview

Recommendations

GCC/Binutils Options
Indirect Branching
Stack Canaries
Position Independent Executables (PIE)
Position Independant Code (PIC)
Stack Clash Protection
Data Execution Prevention (DEP)
  • Usage: -Wl,-z,noexecstack -Wl,-z,noexecheap
  • Intention:
    • Buffer overflows tend to put code in programs stack and jump to it
    • If all writable addresses are non-executable, the attack is prevented
    • Don't mark memory as executable when it is not required
    • ELF headers are marked with PT_GNU_STACK and PT_GNU_HEAP
    • Set stacks/heaps to be executable only if segment flag calls for it
  • Resources:
Source Fortification
Run-time bounds checking for C++ strings/containers
Hardening Quality Control
Zero Caller Saved Registers
Control-Flow Enforcement Technology (CET)
Format Security
Reject missing function prototypes
Reject undefined symbols
RElocation Read-Only ELF Hardening
C/POSIX Standard Library Implementation
musl
  • Notes:
    • Small binaries (~13k hello world vs ~600k with glibc)
    • Small .a/.so footprint (~1MB vs ~10MB for glibc)
    • Small codebase designed to be easy to audit
    • Supported by security focused Linux distros (linuxkit, gentoo, alpine)
    • Low-memory or resource exhaustion conditions are never fatal
    • Wider support for microcontrollers and embedded targets than glibc
    • Rapid termination on security violation to limit attacks on error codepaths
    • Can build itself with ASLR to catching internal stack-smashing
    • Double-Free protection (as possible)
    • Moderate heap-overflow detection
    • Use PIE together with static-linking
    • Limited machine-specific code minimizing chances of minority arch breakage
    • Limit buggy translations via forced literal strings without format strings
    • Block all LD_* for suid/sgid binaries limiting runtime behavior overrides
    • Non-use of arbitrary-size VLA/alloca, minimal dynamic allocation
    • Avoid subtle race condition and async-signal safety issues found in glibc
    • Attempts to remove all undefined behavior. Less code == less bugs
    • Safe, fully-standards-conforming UTF-8 (source of many security bugs)
    • Consistent results even under transient errors without silent fallbacks
    • Lazy/Late allocations that would abort on failure are unsupported
  • Resources:

Background

Kernel

Overview

While Linux is certianly not designed for out-of-the-box high security it is the most portable for the widest range of use cases and has the largest number of deployments so advice in this section will assume it.

If your application does not require a Linux kernel it is suggested the reader carefully consider security-focused alternatives like OpenBSD, FreeBSD, FreeRTOS, or seL4.

Some of these features don't ship with any published binary kernels for any major distribution so it is assumed the reader will compile their own kernel with a hardened toolchain following the advice in the Toolchain section of this document.

Recommendations

Boot Options (WIP)
slub_debug=FZP

2:

page_poison=1
  • Platforms: x86_64, arm64
  • Intention: *
  • Resources:
slab_nomerge
  • Platforms: x86_64, arm64
  • Intention: *
  • Resources:
pti=on
  • Platforms: x86_64, arm64
  • Intention: *
  • Resources:
Sysctl Options
Avoid kernel address exposures in /proc files (kallsyms, modules, etc).
Restrict Kernel Syslog Access
Restrict Performance Event Access
Explicitly Disable Kexec
Avoid non-ancestor ptrace access to running processes and their credentials.
Disable User Namespaces
Disable Unprivileged eBPF
Turn on BPF JIT hardening, if the JIT is enabled.
Config Flags (WIP)
CONFIG_GCC_PLUGINS=y

2: https://www.kernel.org/doc/Documentation/gcc-plugins.txt)

CONFIG_GCC_PLUGIN_STACKINIT=y
  • Platforms: x86_64, arm64
  • Intention:
    • Exploits often take advantage of uninitalized variables
    • Force unconditional initialization of all stack variables
  • Resources:
CONFIG_GCC_PLUGIN_STACKLEAK=y
CONFIG_GCC_PLUGIN_STRUCTLEAK=y
CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL=y
CONFIG_GCC_PLUGIN_LATENT_ENTROPY=y
  • Platforms: x86_64, arm64
  • Intention:
    • Some systems have known issues generating hardware entropy in early boot
    • insert local variable in loop counts, cases, branching, etc
    • Permuate a global variable based on value changes to marked functions
    • Use global variable to help seed early boot entropy pool
  • Notes:
    • Many of the sources of entropy here are deterministic.
    • It is not advised to rely on this plugin alone. Use a TRNG where possible.
  • Resources:
CONFIG_GCC_PLUGIN_RANDSTRUCT=y
CONFIG_RETPOLINE=y
  • Platforms: x86_64, arm64
  • Intention:
    • Infinite loops prevent CPUs from speculating the target of an indirect jump
    • Attach infinite loop to every return call as a "return trampoline"
    • Never actually execute this infinite loop
    • Mitigates kernel or cross-process memory disclosure attacks like Spectre
  • Notes:
    • Use in combination with -mindirect-branch=thunk-extern in GCC8
    • Edge case: When an RSB empties, Skylake+ uses vulnerable BTB prediction
    • Also apply vendor firmware mitigations where possible
  • Resources:
CONFIG_HARDENED_USERCOPY=y
CONFIG_CC_STACKPROTECTOR_STRONG=y
  • Platforms: x86_64, arm64
  • Intention: *
  • Resources:
CONFIG_STRICT_KERNEL_RWX=y
  • Platforms: x86_64, arm64
  • Intention: *
  • Resources:
CONFIG_DEBUG_RODATA=y
  • Platforms: x86_64, arm64
  • Intention: *
  • Resources:
CONFIG_DEFAULT_MMAP_MIN_ADDR=65536
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_RANDOMIZE_BASE=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_RANDOMIZE_MEMORY=y
  • Platforms: x86_64
  • Intention: *
CONFIG_LEGACY_VSYSCALL_NONE=y

2:

CONFIG_PAGE_TABLE_ISOLATION=y
  • Platforms: x86_64
  • Intention: *
CONFIG_IA32_EMULATION=n
  • Platforms: x86_64
  • Intention:
    • Disable 32 bit program emulation and all related attack classes.
CONFIG_X86_X32=n
  • Platforms: x86_64
  • Intention: *
CONFIG_MODIFY_LDT_SYSCALL=n
  • Platforms: x86_64
  • Intention: *
CONFIG_ARM64_SW_TTBR0_PAN=y
  • Platforms: arm64
  • Intention: *
CONFIG_UNMAP_KERNEL_AT_EL0=y
  • Platforms: arm64
  • Intention:
    • Kernel Page Table Isolation
    • Remove an entire class of cache timing side-channels.
CONFIG_BUG=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_STRICT_KERNEL_RWX=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_DEBUG_WX=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_STRICT_DEVMEM=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_IO_STRICT_DEVMEM=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SYN_COOKIES=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_DEBUG_CREDENTIALS=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_DEBUG_NOTIFIERS=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_DEBUG_LIST=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_DEBUG_SG=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_BUG_ON_DATA_CORRUPTION=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SCHED_STACK_END_CHECK=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SECCOMP=y
CONFIG_SECCOMP_FILTER=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SECURITY=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SECURITY_YAMA=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_HARDENED_USERCOPY=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SLAB_FREELIST_RANDOM=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SLAB_FREELIST_HARDENED=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SLUB_DEBUG=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_PAGE_POISONING=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_PAGE_POISONING_NO_SANITY=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_PAGE_POISONING_ZERO=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_VMAP_STACK=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_REFCOUNT_FULL=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_FORTIFY_SOURCE=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_ACPI_CUSTOM_METHOD=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_COMPAT_BRK=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_DEVKMEM=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_PROC_KCORE=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_COMPAT_VDSO=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_KEXEC=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_HIBERNATION=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_BINFMT_MISC=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_LEGACY_PTYS=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_SECURITY_SELINUX_DISABLE=n
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_PANIC_ON_OOPS=y
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_PANIC_TIMEOUT=-1
  • Platforms: x86_64, arm64
  • Intention: *
CONFIG_MODULES=n
  • Platforms: x86_64, arm64
  • Intention: *

Background

You can’t perform that action at this time.