Skip to content

henryvps/my-links

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

84 Commits
README.md
README.md
 
 

Repository files navigation

My Links

IT News, Resources, and Community Sites

Table of Contents

IT News

Linux News and Articles

Message Boards and Forums

Asset Management

  • Clusto - Inventory tracking and "cluster management" system
  • Collins - Infrastructure source of truth, created by Tumblr
  • Fusion Inventory - Mulit-lingual
  • GestioIP - IP address management (IPAM), web based, supports discovery
  • GLPI - Also provides license management, software auditing, and ticketing
  • HostDB - New tool to help manage data center inventory
  • IP Plan - IPAM written in PHP
  • IT ITems DataBase (ITDB) - Includes purchase order management, floor plans, and ISO20000-like features
  • Network UPS Tools - Manage power devices from over 100 manufacturers using a single web interface
  • OCS Inventory NG (OCS-NG) - Automated inventory, deployment system, can sync with GLPI
  • openDCIM - Data center infrastructure management
  • phpIPAM - Written in perl. No, Ruby. No, Perl. Probably PHP
  • RackMonkey - No longer under development as of 2013
  • RackTables - Data center asset management, highly recommended
  • racktables-contribs - RackTables user-contributed plugins
  • Ralph - DCIM and CMDB, supports auto-discovery
  • rho - Scans for and gathers information about Linux and Unix servers
  • Snipe IT - Uses Bootstrap, web based, supports mobile
  • Tracmor - Web-based inventory and asset tracking system written in PHP 5 utilizing the Qcodo Framework

Backups

  • Amanda - The classic
  • Attic - Written in Python
  • Backupninja - Centralize way to configure and schedule many different backup utilities
  • BackUp and Restore Program (BURP) - Reduces network traffic and the amount of space required by using librsync
  • BackupPC - Dedupe and a web GUI for restores
  • Bacula - Open source backup tool, lots of downloads so I guess it's good I don't know
  • bup - Uses the git packfile format, supports global dedupe, can use "par2" redundancy
  • Duplicati - Supports encryption and dedupe
  • Duplicity - Encrypted bandwidth-efficient backup using the rsync algorithm
  • FSArchiver - Save the contents of a file system to a compressed archive, if one of the checksums doesn't match the file is lost, not the whole backup
  • lsyncd - Watches a local directory trees event monitor interface (inotify or fsevents) and kicks off an rsync when things change
  • Mondo Rescue - Disaster recovery, supports tapes, disks, network and CD/DVD as backup media, multiple filesystems, LVM, software and hardware RAID
  • Obnam - Snapshots, dedupe, encryption
  • rdiff-backup - Combines the best features of a mirror and an incremental backup in a bandwidth efficient manner
  • Redo Backup and Recovery - Simple bare metal backup and restore
  • Relax and Recover (REAR) - BMR, simple
  • rsnapshot - Uses rsync and hard links, can keep multiple full backups available while using very little disk space
  • SafeKeep - Superb project, uses LVM snapshots
  • SnapRAID - Backup program that also stores RAID parity information

Bootable USB Tools

Cluster Management and Schedulers

  • Aurora - Mesos scheduler for "always on" jobs, similar to Marathon
  • Bamboo - HAProxy auto configuration and auto service discovery for Marathon
  • Chronos - Distributed, fault tolerant cron scheduler that runs on top of Apache Mesos
  • crypt - Store and retrieve encrypted configs from etcd or consul
  • consul - Service discovery and configuration via DNS or HTTP, great for auto-scaling
  • Curator - ZooKeeper client wrapper and rich ZooKeeper framework, by Netflix
  • etcd - Highly-available key value store for shared configuration and service discovery
  • eureka - AWS Service registry for resilient mid-tier load balancing and failover, by Netflix
  • kubernetes - Container cluster manager
  • Marathon - Mesos scheduler for "always on" jobs (eg: web services)
  • Mesos - Cluster manager and scheduler, similar to Google's Borg and Omega projects
  • Mesosphere - Apache Mesos schedulers for almost anything
  • Myriad - YARN framework/scheduler
  • Serf - Decentralized solution for service discovery and orchestration that is lightweight, highly available, and fault tolerant
  • SmartStack - Automated service discovery and registration framework, includes Nerve and Synapse
  • vulcand - HTTP proxy that uses etcd as a configuration backend
  • Zookeeper - Distributed configuration service, synchronization service, and naming registry

Configuration Management and Orchestration

  • Ansible - CM and orchestration, also can do provisioning
  • Ansible Galaxy - Community site for finding, reusing, and sharing Ansible content
  • Awesome Chef - A curated list of amazingly awesome Chef resources
  • BOSH - Cloud orchestration, supports Docker containers too
  • Capistrano - Run scripts/commands and push software using SSH, a RubyGem, uses a Rake DSL
  • Chef - Well known configuration management tool
  • chef-book - Open source book about Chef
  • chef-handlers - Log failures to IRC and send metrics to Graphite (including deploys)
  • ChefScripts - Quickly parse the Chef run log to get the resources updated recently, in chronological order
  • Chef Development Kit (ChefDK) - Includes Berkshelf, Test Kitchen, ChefSpec, Foodcritic, Knife, Chef Zero, Ohai, and more
  • Chef is Easy - Chef tutorial
  • Chef Solo - Allows using cookbooks on nodes without access to a Chef server, designed for testing
  • Chef Supermarket - Massive collection of cookbooks
  • Chef Wiki - The official Opscode wiki
  • Cooking Infrastructure by Chef - Free Chef ebook
  • ElasticBox - Create your own catalog of applications and infrastructure components then build, deploy, and go
  • Fabric - Python library, uses SSH
  • Getting Started with Chef - Chef tutorial
  • grocery-delivery - Utility for managing cookbook uploads to distributed Chef backends
  • Juju - Ubuntu focused but works on RHEL / CentOS now
  • knife-backup - Knife plugin to help backup and restore a Chef server
  • Knife Community Plugins -
  • Learn Chef - Chef tutorial by Opscode
  • mcollective - Orchestration, can be used independently of Puppet
  • motherbrain - Use knife to spin up one node? Nah, use this to create a whole environment
  • omnibus - Create full-stack installers for your project, Chef
  • Open Linux Management Infrastructure (OpenLMI) - Manage, monitor, and configure servers via API calls (instead of SSH), included in RHEL 7
  • Opscode - Chef / Opscode on Github, tons of great tools
  • Puppet - Well known CM tool
  • puppet-lint - Check that your Puppet manifest conform to the style guide
  • Puppet Forge - Community repository of Puppet modules
  • puppet-module-skeleton - Skeleton modules for Puppet
  • Really Awesome New Cisco config Differ (RANCID) - Designed to monitor network hardware
  • Rudder - Automation and compliance, uses CFengine
  • Salt - Cloud orchestration, server provisioning, and configuration management
  • ServerSpec - Write RSpec tests for checking that your servers are configured correctly
  • spiceweasel - Quickly automates the creation of whole Chef environments (batch loading)
  • Supermarket - Community cookbook repository for Chef, can be hosted internally
  • Using Chef - Chef weekly newsletter
  • VirtKick - Simple panel for orchestrating VMs and containers
  • zero-user - Zero hassle Linux user management, requires Ansible

Containers

  • cadvisor - Analyzes resource usage and performance characteristics of running containers
  • Control Center - App management and orchestration, can manage Docker applications
  • crane - Docker orchestration
  • Docker - Designed to applications in a container, but not a whole OS instance
  • Docker-Registry - Source code to Docker Registry, allows you to create private registries
  • Docker Nano - The smallest possible Docker images
  • dockerfiles - Great little collection of pre-made Docker files
  • Docker Hub Registry - Official Docker images for many projects
  • Dokku - Docker powered mini-Heroku in around 100 lines of Bash
  • Dokku Alternative - Dokku fork, supports plugins
  • Ferry - Big data development environment on your local machine using Docker
  • fig - Fast, isolated development environments using Docker
  • Flocker - Docker container and data management
  • geard - CLI tool for installing Docker images as containers onto a systemd-enabled Linux system
  • libcontainer - Used by Docker 1.0+ so it doesn't have to use LXC/libvirt/systemd-nspawn
  • libswarm - Toolkit to compose network services, by Docker
  • Linux Containers (LXC) - Runs an OS instance (sshd, cron, daemons, init, etc) within a container
  • lmctfy - Open source version of Google’s container stack, which provides Linux application containers
  • Longshoreman - Heroku-like CLI tool for automated Docker deployments
  • nsenter - Attach to a Docker container
  • Panamax - Containerized app creator with an open-source app marketplace hosted in GitHub
  • pipework - SDN for Linux Containers
  • ShutIt - Manage your build process in a way that is both structured and flexible
  • Weave - Virtual network that connects Docker containers deployed across multiple hosts

File Sync and Sharing

  • BitTorrent Sync (btsync) - Sync on the local network or over the internet -- closed source
  • CloudBerry Explorer - Windows client for accessing AWS S3 buckets -- freeware, closed source
  • Cloud Explorer - Access storage on multiple clouds, Windows -- freeware, closed source
  • ownCloud - The most popular file sync and share program
  • Pulse - Uses an ID rather than an IP address, share your ID with friends and go. Formerly Syncthing
  • Pydio - Formerly AjaXplorer, AGPL license
  • Seafile - Also offers a paid professional edition with more features
  • SparkleShare - Uses git under the hood
  • Syncany - Supports tons of different protocols for the transfer
  • Unison - Multiplatform, feature-rich file sync

Git Tutorials

High Availability Clustering Tools

High Performance Computing Tools

HTTP Load Generation Tools

  • Gatling - HTTP, JMS, and WebSocket load generator
  • FunkLoad - Another load generator
  • httperf - HTTP load generator, originally created by HP, now open source
  • httping - Simple program that "pings" a URL and shows response time
  • JChav - Visualize JMeter output
  • JMeter - Supports many different services and protocols, written in Java
  • Locust - Written in Python, allows you to define user behavior
  • Siege - CLI based, supports basic authentication, cookies, and HTTPS
  • Tsung - Used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and Jabber/XMPP servers
  • Web Performance Today - News site dedicated to web / HTTP performance
  • wrk - Multi-threaded CLI tool

IO Load Generation Tools

Logging and Log Management

  • Adiscon LogAnalyzer - Slick web interface for syslog messages
  • Bigdesk - Live charts and statistics for elasticsearch cluster
  • ElasticDump - Import / export tools for Elasticsearch
  • Elasticsearch - Component of the ELK stack, used for super fast searches
  • elasticsearch-river-email - E-mail river plugin for Elasticsearch
  • Elasticsearch Copier (escp) - Tool to copy elasticsearch indexes, great for copying from prod to dev
  • Elasticsearch Exporter - Small script to export data from one Elasticsearch cluster into another
  • FileSystem River (fsriver) - Elasticsearch river plugin helps to index documents from your local file system and using SSH
  • Fluentd - Unified logging layer
  • Flume - Distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data
  • GoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal
  • Graylog2 - Log capture and analysis
  • InfluxDB - Distributed time series database with no external dependencies
  • kairosdb - Time series DB written on top of Cassandra
  • Kibana - Visualization component of the ELK stack
  • Logplex - Distributed syslog log router, uses Redis
  • logstalgia - Web site access log visualization tool, aka Apache Pong
  • logstash - Collect, parse, and store logs. A component of the popular ELK stack
  • logstash-forwarder - Formerly Lumberjack, forwards logs to centralized server(s), very lightweight
  • multilog - Reads a sequence of lines from stdin and appends selected lines to any number of logs
  • NXLOG - Universal log collector and forwarder, supports many formats/platforms/sources including Windows
  • OpenTSDB - Store and serve massive amounts of time series data without losing granularity
  • Piwik - Web analytics platform with a killer UI
  • Webalizer - Web and FTP log analyzer

Monitoring and Alerting

  • Adagios - Web based Nagios configuration interface
  • Awesome Dashboard - A collection of dashboards
  • Bosun - By Stack Exchange
  • Cabot - Self-hosted, easily-deployable monitoring and alerts service - like a lightweight PagerDuty
  • Cacti - Web-based network monitoring and graphing tool designed as a front-end to RRDtool
  • check_mk - Extensions / plugins for Nagios
  • collectd - Collects system performance statistics
  • collectd Related Sites - Great tools that integrate with collectd
  • Dashing - Dashboard framework
  • Dashing Widgets - Massive collection of widgets for Dashing
  • Freeboard - Ridiculously simple dashboards for your devices
  • Fully Automated Nagios - Easy to install, pre-packaged Nagios that includes the most-used tools and plugins
  • Ganglia - Monitoring software that's focused on HPC / distributed clusters
  • Grafana - Modern dashboard for Graphite
  • Graphite - Store numeric time-series data and render graphs of the data on demand
  • Graphite-Sankey - Turns Graphite targets into a D3.js Sankey visualization
  • Graphite Tools - Tools that work with Graphite
  • Icinga - Nagios fork, updated frequently
  • jmxtrans - Connector between speaking to a JVM via JMX on one end and whatever logging / monitoring / graphing package
  • LibreNMS - GPL fork of Observium
  • linux-dash - A simple web dashboard to monitor your Linux server
  • Monit - Includes tools to automatically take action when certain conditions are met
  • Monitorix - So lightweight that it can be used in mobile devices, aka Mikaku
  • Multi Router Traffic Grapher (MRTG) - Still being updated!
  • Naemon - Modular Nagios fork
  • Nagdash - Dashboard / NOC screen for Nagios
  • Nagios - One of the most widely used OSS monitoring programs
  • Nagiosgraph - Another visualization tool for Nagios data
  • NagVis - Visualization suite for Nagios
  • NeDi - Network discovery and monitoring
  • Network Weathermap - Network visualization tool, create a "weather map" just like big ISPs
  • OpenNMS - Network monitoring, also supports configuration/asset management
  • Open Monitoring Distribution (OMD) - Nagios bundled with the most popular plugins and tools
  • OpsView Core - GPL version of OpsView, not as feature-rich as the commercial product
  • php-weathermap-graphite - Allows Network Weathermap to source data from Graphite
  • pmacct - IP and network traffic accounting / monitoring
  • PNP4Nagios - Nagios visualization tool
  • riemann - Aggregates events from your servers and applications with a powerful stream processing language.
  • RRDtool - High performance data logging and graphing system for time series data
  • Sensu - Open source monitoring framework, cloud-focused
  • Seyren - Alerting dashboard for Graphite
  • Shinken - Compatible with Nagios
  • Skyline - Designed to be used wherever there are a large quantity of high-resolution timeseries which need constant monitoring
  • StatsD - Monitoring in node.js
  • Tattle - Self service alerting and dashboard frontend for Graphite and Ganglia
  • Thruk - Web based, can create SLA reports, has a mobile client
  • Uchiwa - Dashboard for Sensu
  • Xymon - Fork of Big Brother
  • Zabbix - Stores monitoring data in a DB, has agents for almost every OS
  • Zenoss - Supports Nagios plug-in format

Network Load Generation Tools

  • Flowgrind - Distributed TCP traffic generator
  • iperf3 - Supports tuning of various parameters related to timing, protocols, and buffers
  • netperf - The web page looks ancient but the software's still being updated!
  • netperf-wrapper - Python wrapper to run multiple simultaneous netperf instances and aggregate the results
  • ostinato - "WireShark in reverse"

Networking Tools

  • Argus - Generates network activity reports
  • bbFTP - Uses it's own transfer protocol, optimized for large file transfers
  • BIRD - (Almost) fully functional IP routing daemon for Linux, supports tons of standard routing protocols
  • Charles Web Debugging Proxy - Highly recommended, includes support for decoding HTTPS traffic and throttling -- commercial software
  • CoDel - Network scheduling algorithm designed to overcome bufferbloat
  • Comcast - Simulate crappy network connections
  • DNSsec - Reddit posters in r/netsec explain DNSSec
  • firewalld - Replaces iptables, included in RHEL/CentOS 7
  • ipcalc - CLI tool to calculate subnets, netmasks, IP ranges, broadcast addresses, and more
  • GridFTP - Supports parallel streams
  • lftp - Supports many protocols (FTPS, HTTPS, SFTP), scheduling, bandwidth throttling, scripting, and more
  • localtunnel - Share a web service on your local development machine without messing with DNS and firewall settings
  • mitmproxy - Intercept, modify, replay and save HTTP/S traffic, edit flows on the fly
  • moloch - Large scale IPv4 full PCAP capturing, indexing and database system
  • mtr (My traceroute) - Combines ping and traceroute into a single program
  • namebench - Hunts down the fastest DNS servers for your computer to use
  • ncat - Improved netcat, written by the Nmap team
  • netbrake - Limit the bandwidth used by a process
  • nfdump - Captures network flows
  • NfSen - GUI for nfdump
  • ngrep - Network grep
  • nping - Create custom network packets
  • ntopng - New version of the popular ntop tool
  • Paris Traceroute - Shows proper network topology when load balancers and load-balanced routers are used
  • PageKite - Makes web servers or SSH publicly available on any server
  • prettyping.sh - bash wrapper for ping to create pretty output
  • Server Name Indication (SNI) - Think vhosts for SSL
  • SMB Multichannel - Multiple SMB streams over a single TCP connection, killer feature
  • socat - netcat on steroids, supports serial devices
  • sslh - Protocol multiplexer, forwards packets based on the first packet
  • Stanford Linear Accelerator Center - Network Monitoring Tools - MASSIVE list of network monitoring tools
  • tc - Built-in Linux kernel traffic control
  • tcpflow - TCP demultiplexer, each flow is stored in it's own file
  • TCPWatch - TCP monitoring and logging tool
  • trickle - Userspace bandwidth shaper
  • WireShark - The classic network analyzer
  • WireShark Tools - Superb list of network tools from the WireShark wiki
  • ZeroTier - Cloud / provider-agnostic private network backplanes

Packages and Repositories

  • apt-dater - ncurses frontend for package management via SSH, also supports yum
  • aptly - Debian-centric, includes repo support and the ability to take snapshots for easy rollback
  • CentOS Errata for Spacewalk (CEFS) - Import errata information from CentOS-announce into Spacewalk
  • CentOS Repositories - Large list of both official and unofficial CentOS software repositories
  • CentOS Software Collections (SCL) - Use multiple versions of software on a system without disturbing the system default version
  • Copr - Automatic build system providing a package repository as its output
  • DNF - Package management system that will replace Yum in upcoming Fedora releases
  • ELRepo - Repo that focuses on hardware related packages, supports RHEL and CentOS
  • Extra Packages for Enterprise Linux (EPEL) - Supports CentOS, RHEL, Scientific Linux, and Oracle Linux
  • Extra Packaging Guidelines and Policies for EPEL - Packaging guidelines, great even if not creating EPEL stuff
  • Fedora Packaging Guidelines - Excellent information that can be applied to CentOS & RHEL
  • Fucking Package Management (fpm) - Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity
  • IUS SafeRepo Initiative - Create "safe" 3rd party repos
  • Koji - Software that builds packages for Fedora, can be used for other stuff too, uses mock
  • Mock - Build packages in a simple chroot so you don't blow up your system
  • mrepo - Yum & APT repository mirroring, formerly Yam
  • OStree - Tool for managing bootable, immutable, versioned filesystem trees (not really a package manager but...)
  • Package Repository Manager (PRM) - OS-independent package repository tool
  • Pulp - Next generation repository management, a component of Red Hat Satellite 6
  • Red Hat Software Collections (SCL) - Use multiple versions of software on a system without disturbing the system default version
  • reposync - Synchronize yum repositories to a local directory
  • rpm-ostree - Take RPM packages and commit them to an OSTree repository
  • RPM Fusion - Provides software that the Fedora Project or Red Hat doesn't want to ship
  • Spacewalk - Provides package management (among other things)
  • yum-presto - yum plugin that provides support for downloading package deltas instead of whole packages
  • yum-security - Plugin to only install security updates

Password Management

  • 1Password - Password manager for OS X, Windows, iOS, and Android -- closed source
  • ccrypt - Replacement for the standard Unix "crypt" utility
  • Diceware - Neat passphrase generator
  • GNU Privacy Guard (GPG) - The original! Store em in text files and encrypt
  • GRC Password Generator - Generates 63 bit passwords, web based
  • KeePass - Lightweight, easy to use password manager, runs on most OSes, requires MONO for non-Windows systems
  • Keepass2Android - Compatible with KeePass 2.x, requires MONO
  • KeePassDroid - Compatible with KeePass 1.x
  • KeePassX - Alternative to KeePass, requires QT
  • kpcli - Command line interface for KeePass
  • LastPass - Fremium, can sync to other devices -- closed source
  • lastpass-cli - CLI for LastPass
  • msktutil - Keytab client for a Microsoft Active Directory environment
  • One-Time Secret - Create links that self-destruct after a single viewing, great for sharing passwords
  • pass - Uses GPG at it's core, supports tracking password changes in git
  • Password Gorilla - Written in Tcl/Tk
  • Password Pusher - Links to passwords expire after a certain number of views and/or time has passed, RoR app
  • Password Safe - Designed by Bruce Schneier himself
  • Pleasant Password Server - Multi-user management tool for KeePass, great for businesses -- commercial software
  • pwgen - Password generated, included in most Linux distros
  • RatticDB - Web-based password management, includes an API, great for teams or companies
  • scrypt - Super secure against hardware brute-force attacks, is also a part of/module for most programming languages
  • TeamPass - Collaborative password management using the LAMP stack
  • webpasswordsafe - Very powerful, designed "for the enterprise"
  • zxcvbn - Password strength estimator, written by Dropbox, operates in a browser window

Performance Monitoring

  • apachetop - top for Apache
  • atop - Supports both real-time and historical monitoring
  • bandwidthd - Monitors and graphs bandwidth usage
  • blktrace - Article on blktrace, includes tldr setup steps in the comments
  • bmon - Console based network monitor
  • bwm-ng - Small and simple console-based live network and disk I/O bandwidth monitor
  • collectl - sar on steroids
  • Conky - Lightweight system monitor for X windows
  • Diamond - Python daemon that collects system metrics and publishes them to Graphite (or similar), has an API
  • dim_STAT - Collects almost everything and stores it in a MySQL database, produces reports too
  • dstat - Versatile replacement for vmstat, iostat, netstat and ifstat
  • ESnet Fasterdata Knowledge Base - Provides proven, operationally sound methods for troubleshooting and solving performance issues
  • findswappers - perl script that identifies swapped processes
  • Glances - Real-time monitoring, written in curses and Python
  • htop - top replacement, has a few additional features
  • iftop - top for network interfaces
  • ioping - Monitor I/O latency in real time
  • iotop - top for I/O requests, displays information on a per-process basis
  • iptraf - A top'esque program for network traffic, has multiple display modes, written in ncurses
  • ksar - Creates pretty graphs from sar output
  • Linux Performance Observability Tools - Awesome graphic that shows you which tool to use, by Brendan Gregg
  • mem_logger.sh - Monitor a processes' memory usage over time
  • Munin - Historical performance monitoring to help determine when you server became "slow"
  • nbwmon - Ncurses bandwidth monitor
  • netatop - Kernel module for atop to watch and report on network packets
  • NetHogs - Displays per-process bandwidth usage
  • ngxtop - top for nginx
  • nicstat - vmstat for network interfaces
  • nload - Console application that monitors network traffic and bandwidth usage in real time, neat ASCII graphs
  • nmon - Provides both real-time and historical monitoring
  • PageSpeed Module - Reduce page load time without modify your existing content or workflow
  • perf-tools - By Brendan Gregg, uses perf and ftrace, includes iosnoop, KILLER TOOLS
  • Performance Co-Pilot (PCP) - Included in RHEL / CentOS 7
  • pidstat - vmstat type output for CPU, disk I/O, page faults, and more on a per-process basis
  • pmap - Shell scripts for tracking memory usage using "pmap"
  • PowerTOP - Real-time power consumption on a per-process & per-thread basis
  • ps_mem - Accurately reports core memory usage for a process
  • recap - Collects info from various standard utilities (free, sar, vmstat, etc) at specified intervals
  • slabtop - Tutorial on how to use slabtop, useful when you have no idea on how your RAM is being used
  • smem - Reports memory usage based on proportional set size (PSS) instead of the usual resident set size (RSS)
  • sysdig - "Think of it as strace + tcpdump + lsof + awesome sauce.", recommended
  • SmokePing - Network latency visualizer, written by the MRTG and RRDtool guy
  • vnStat - Console based traffic monitor, supports statistic collecting
  • wtop - Another top for Apache

Provisioning and Lifecycle Management

  • cloud-init - Configures settings the first time a system spins up (SSH keys, hostname, almost anything else)
  • Cobbler - Provisioning system used in Red Hat's Satellite
  • Crowbar - Multi-purpose provisioning tool, originally created by Dell
  • edeploy - provision and update systems using trees of files instead of packages or VM images
  • Foreman - Provisioning and life cycle management
  • gPXE - GPL'ed version of PXE
  • iPXE - GPL'ed version of PXE, replacement for gPXE
  • Katello - Candlepin + Pulp + Foreman, the core of Red Hat Satellite 6
  • Kickstart - The classic Red Hat tool
  • Metal as a Service (MaaS) - Ubuntu provisioning system
  • Packer - Automates VM creation for multiple platforms (VMware, AWS, etc), can use a "golden image" approach
  • Razor - Puppet provisioning software that handles bare metal hardware and virtual server provisioning
  • Sandstorm.io - Download, install, and run apps in a container
  • Terraform - Infrastructure provisioning using existing tools
  • Vagrant - "Hypervisor wrapper" that leverages CM software to quickly spin up environments
  • veewee - Automate all the steps for building Vagrant base boxes

Python Tutorials

RDBMS Load Generation and Performance Tools

Regular Expressions

Security Tools

  • Advanced Intrusion Detection Environment (AIDE) - File integrity checker, alternative to Tripwire
  • Angry IP Scanner - Supports plugins, also gathers NetBIOS information
  • Automatic Server Hardening - Scripts and cookbooks/manifests for Puppet and Chef
  • BlackArch Linux - Penetration testing Linux distro, based off of Arch Linux
  • BoringSSL - Google's fork of OpenSSL, does not guarantee API and ABI compatibility
  • Brakeman - Static analysis security vulnerability scanner for Ruby on Rails applications
  • bro - Framework for network analysis and security monitoring
  • chkrootkit - Rootkit checker, best used from a live CD
  • cipherscan - Find out which SSL ciphersuites are supported by a target
  • CIS-CAT Benchmark Tool - Center for Internet Security's benchmark tool, evaluates systems based on their hardening recommendations
  • CIS Linux Benchmarks - Linux OS hardening guides, superb!
  • Computer Aided INvestigative Environment (CAINE) - Live CD that focuses on forensics
  • ConfigServer Security and Firewall (CSF) - Stateful Packet Inspection (SPI) firewall, Login/Intrusion Detection and Security application for Linux servers
  • Cryptkeeper - System tray applet that manages EncFS encrypted folders
  • dnscrypt-wrapper - Add dnscrypt support to any resolver
  • diskscrub - Supports many disk-wiping standards including military / government grade wipes
  • duraconf - A collection of hardened configuration files for SSL/TLS services by Jacob Appelbaum
  • EncFS - Encrypted file system in user space via FUSE
  • fail2ban - Watches log files to ban IPs based on rules (too many failed logins, exploit attempts, brute force attacks, etc)
  • fail2ban Dashboard - Written with Flask
  • Forensics Wiki - Digital forensics wiki
  • FTimes - System baselining and evidence collection tool
  • fwknop (FireWall KNock OPerator) - Single Packet Authorization (SPA) aka next-generation port knocking
  • HoneyDrive - Honeypot appliance
  • hping - Create custom TCP/IP packets, very flexible, see also hping3
  • HTTP Strict Transport Secuirity (HSTS) - Forces browsers to interact with a site by only using HTTPS
  • Information Security Cheat Sheets and Checklists - An assortment of IS checklists and cheat sheets
  • Kali Linux - Penetration testing Linux distro
  • Kippo - SSH honeypot
  • knockd - Port knocking server for Linux
  • LibreSSL - OpenBSD's fork of OpenSSL, Linux compatible
  • Lynis - Auditing and hardening tool, supports most Unix-like operating systems
  • Metasploit - Classic exploit framework
  • mtree - Can be used as a simple IDS
  • Network Security Toolkit (NST) - Live CD that includes most tools in insecure.org's top 125 tools list
  • nmap - Classic port scanner
  • nmap Scripting Engine (NSE) - Tons of scripts for nmap
  • nogotofail - Spot and fix weak TLS/SSL connections and sensitive cleartext traffic
  • OpenSCAP - Standardized approach to maintaining the security of systems, evaluates machine compliance with a given profile
  • OpenVAS - Vulnerability scanner, forked from the now closed-source Nessus scanner
  • OpenVPN - Well known open source VPN
  • Open Source Tripwire - File integrity checker and monitor, replacement for the now closed-source Tripwire
  • Open Web Application Security Project (OWASP) - Focus on application security
  • Oracle Database Attacking Tool (ODAT) - Open source penetration testing tool that tests the security of Oracle databases remotely
  • OSSEC - Host based intrusion detection system (HIDS), supports most Unix-like OSes
  • pfsense - The best firewall software, supports appliances and Live CDs
  • pyCAmanager - GUI for managing your certificate authority, written in Python
  • Qualys SSL Server Test - Evaluates and provides recommendations for the SSL settings of any web site
  • r/netsec Tools List - r/netsec creates a list of security related tools
  • Red October - Go server for two-man rule style file encryption and decryption
  • Rootkit Hunter - Compares hashes of important files with known good hashes that are stored in online databases
  • Samhain - HIDS, file integrity checker, rootkit detection, log file monitoring, and more
  • ScoutBot - Automated network scanner for Raspberry Pi
  • Security Engineering - Free online book
  • Security Onion - Linux distro for IDS, NSM, and log management
  • Snoopy Logger - Logs commands that are executed and saves the information to syslog
  • Snort - The classic network intrusion prevention system (NIPS)
  • SoftEther - Maybe the best VPN software out there
  • SSH Guard - Think fail2ban for SSH
  • sslconfig - CloudFlare's Internet facing SSL cipher configuration
  • sslsniff - A tool for automated MITM attacks on SSL connections
  • SSLsplit - Transparent and scalable SSL/TLS interception
  • sslyze - Fast and full-featured SSL scanner, written in Python
  • sudosh2 - Login shell that records all keystrokes and output, supports session playback
  • tinc - Simple, multi-platform VPN
  • Tripwire - File integrity checker and monitor, open source version
  • VirusTotal - Allows you to upload a file and have it scanned by tons of virus scanners
  • ZMap - Scan the entire Internet (or a specified IPv4 range)
  • zuluCrypt - Simple CLI and GUI frontend to cryptsetup and tcplay

Shell Scripting

SSH Tools

  • autossh - Automatically restart SSH sessions that stop passing traffic
  • Balancing Load Across Systems (Ballast) - Balance user load across SSH servers based on configured policies
  • Bitvise SSH Client - Feature-rich SSH & SFTP client for Windows, free for individual use -- closed source
  • Byobu - An enhanced version of the "screen" utility
  • cloud-ssh - SSH wrapper that supports hostname tab-completion based on name, tag, or security group
  • ClusterSSH - Make a change on many servers at the same time
  • Corkscrew - Tunnel SSH through HTTP proxies
  • csshX - Cluster SSH for OS X
  • Dancer's Shell (dsh) - Wrapper for executing multiple remote shell (rsh/remsh/ssh) commands
  • Guacamole - Clientless (in-browser) remote desktop gateway, supports VNC, RDP, and SSH
  • HPN-SSH - High performance ssh/scp (auth handshake is encrypted, data transfer is not)
  • KeyBox - Web-based SSH console that executes commands on multiple shells
  • Keychain - frontend to ssh-agent and ssh-add, reduces the number of times you have to type in your passphrase
  • KiTTY - PuTTY fork with additional features
  • Mass Parallel SSH (mpssh) - Simple parallel SSH
  • Match - Creates a conditional block, great for controlling actions on a per-user and/or per-host basis
  • MobaXterm - Tabbed SSH, VNC, and RDP client for Windows, free for personal use -- closed source
  • Mobile Shell (Mosh) - Remote shell that supports roaming (client IP address changes) and intermittent connectivity
  • mRemoteNG - Open source, tabbed, multi-protocol, remote connections manager
  • MTPuTTY (Multi-Tabbed PuTTY) - Multiple PuTTY sessions in a single window
  • MUltihost SSH Wrapper (Mussh) - SSH wrapper
  • NoVNC - Client-less VNC in a web browser, uses HTML5 and WebSockets
  • parallel-ssh (pssh) - Parallel version of OpenSSH tools, comes with prsync too
  • Parallel Distributed Shell (pdsh) - Kick off many SSH sessions in parallel
  • paramiko - SSH2 protocol library for Python, provides both client and server
  • Parcel - High performance scp replacement, WARNING: uses UDP
  • PuTTY - The classic SSH client for Windows
  • rssh - Restricted shell for SSH, only allows scp, sftp, rdist, rsync, and cvs
  • scponly - Shell that only allows incoming SCP, supports chroot
  • SSH Power Tool (sshpt) - Execute commands and upload files to many servers simultaneously via SSH without using pre-shared keys
  • SSHFS - Mount remote file systems using a SSH tunnel
  • sshrc - Sources the ~/.sshrc on your local computer after logging in remotely
  • sshuttle - Transparent proxy server that works as a poor man's VPN, forwards over SSH, supports DNS tunneling
  • SuperPutty - Allows the PuTTY SSH client to be opened in tabs
  • tmux - An alternative to GNU screen
  • tmux-cssh - Cluster SSH for tmux
  • WinSCP - The well-known SCP client for Windows
  • Xming - Windows Xwindows server, supports PuTTY

Storage Tools

  • ATA Secure Erase - Send a signal to an ATA drive to perform a hardware-based erase
  • bcache - Block layer cache, allows SSDs to be used as caches for slower drivers
  • EnhanceIO - SSD caching, in the mainline Linux kernel, a fork of Facebook's Flashcache project
  • Flash-Friendly File System (F2FS) - File system designed specifically for SSDs
  • fs-cache - NFS client caching
  • IPOS Calculator - Created by Marek Wolynko
  • Linux-IO Target (LIO) - Create and share iSCSI, FC, FCoE, and other storage targets on Linux
  • Linux Journal - Linux Swap Space - Superb article about Linux swap, includes some tuning parameters
  • lvcache - Wrapper to make lvmcache easier
  • OpenDedupe - Open source dedupe software, works on both block storage and file systems
  • Parted Magic - Resize, grow, shrink, clone, recovery, wiping, benchmarking, and more. Supports Linux and Windows file systems.
  • RAID LVMs - As of RHEL 6.3 LVM supports software RAID without using mdraid
  • Red Hat LVM RAID Calculator - Login required
  • snapper - CLI tool to manage Btrfs snapshots, snapshot timelines, and more
  • System Storage Manager (SSM) - Generic CLI for managing all types of storage (DM, LVM, multipath), added in RHEL 7
  • zram - A compressed, in-memory block device to reduce paging
  • zswap - A compressed, write-back cache cache for swapped pages

Tracing Debugging Profiling

Two Factor Authentication

  • Authy - Provides both personal and enterprise versions
  • Barada Aint Respecting Any Deceptive Adversaries (BARADA) - PAM module for TFA support
  • Crypto Stick - OTP and more -- commercial products
  • Duo Security - Cloud-based, supports many methods of authentication (hard token, smartphone application, etc) -- commercial product
  • FreeOTP - Open source fork of Google Authenticator for iOS and Android
  • Google Authenticator - Two factor authentication; supports Android, iOS, and even has a PAM module -- closed source
  • Two Factor Auth (2FA) - Massive list of web sites that support and don't support two factor authentication
  • YubiKey - Acts as a USB keyboard to automatically input a strong, one time password -- commercial product
  • Yubico PAM - PAM module for use with YubiKey devices

Virtualization

Misc Tools and Utilities

  • ack - grep-like tool designed to search source code
  • Adminer - GUI for database management in a single PHP
  • appscale - Deploy and host Google App Engine applications almost anywhere
  • AutoHotKey - Automate input (desktop and web forms, data entry, keybinds, etc)
  • autojump - Small database of directories that you visited in the past, used to quickly navigate complex directory structures
  • Awesome Node.js - A curated list of Node.js packages
  • Awesome PHP - Curated list of PHP libraries and resources
  • Awesome SQLAlchemy - A curated list of awesome tools for SQLAlchemy
  • Awesome Sysadmin - A curated list of open source resources
  • blockdiag - Generate simple block/sequence/activity/network diagrams from text files
  • boto - Python interface to AWS
  • Checkpoint/Restore in Userspace (CRIU) - Freeze a process, save it to disk, then resume it later
  • comm - Display lines that two files have in common (eg: the opposite of diff)
  • conserver - Virtual console server with many features
  • curlish - Improved curl, includes cookie management and OAuth support
  • cwrap - Wrappers for creating test scenarios and faking behavior, super useful
  • Cygwin - GNU shell and tools for Windows
  • devcheatsheet.com - Cheat sheets and quick references for almost every programming language
  • disown - Disowns a processes from the shell, useful if you forgot to use screen or nohup
  • dotfiles - Store your .bashrc and others on GitHub
  • dsync - No software yet but a hell of a theory
  • eatmemory - Allocates X amount of memory, great for testing
  • etckeeper - Stores /etc in version control
  • Etherpad - Enter, save, and share text/code in a web browser
  • f.lux - Changes your screen from blue light to yellow light when the sun sets to tell your brain it's night time -- closed source
  • FastGlacier - Amazon Glacier client for Windows -- closed source
  • FileUtils - File utilities for distributed computing
  • Filewatcher - Written in Ruby, monitors changes in the filesystem by polling
  • ForgeRock - Enterprise identification management and access (IAM)
  • FreeIPA - Identity, policy, and audit suite, think Active Directory for Linux
  • gcalcli - CLI for Google Calendar
  • Geocities-izer - Make any web page look like a Geocities page
  • Gource - Software version control visualization tool
  • grabmem - Allocates memory until the machine runs out or you quit it with ctrl-c
  • Gravit - Web-based vector image editor / design tool, source code is available on GitHub
  • gt5 - A "du browser", handy for finding large files
  • HasteBin - Open source pastebin alternative for sharing code, can be installed locally / on-site
  • HeidiSQL - GUI SQL DB browser and editor
  • httpie - curl replacement with many new features
  • incrond - Fires off a job when a file system event happens
  • Inkscape - Open source vector image editor
  • IRCcat - Sent log messages and alerts to an IRC channel, can also use IRC to send commands
  • ISO 8601 - ISO standard for dates, the only way to fly
  • JSON Formatter - Automatically format JSON code, 100% client side
  • Inconsolation - MASSIVE list of lightweight and (legit) obscure tools
  • inotify - Watch file(s) for changes, take an action when the change occurs
  • inxi - "a full featured system information script" (hardware info, etc)
  • Keybase - Upload your public key and find other people's public key via their social media user name(s)
  • LargeFile - vim plugin that automatically disables certain things so you can edit large files faster
  • lz4 - Crazy fast compression algorithm
  • maim - Linux desktop screenshot tool
  • Mergeatron - Kicks off Jenkins when it sees a PR, gives a thumbs up/down reply
  • mockaroo - Generate up to 100,000 rows of realistic test data in CSV, JSON, SQL, and Excel formats
  • moreutils - "Collection of the unix tools that nobody thought to write long ago when unix was young"
  • multicast remote sync (mrsync) - Transfers from a master to many remote machines using Unix multicast sockets
  • multitail - Monitor multiple log files in a single terminal window
  • ncdu - ncurses version of "du"
  • Netflix OSS - Netflix OSS tools on GitHub
  • Notepad++ - Killer GPL'ed text editor for Windows
  • observr - Monitor a directory and kick off an action when a file in the directory changes
  • OpenResty - Turn nginx into a non-blocking API server
  • parallel - Execute jobs in parallel using one or more computers
  • percol - Interactive grep tool
  • pv - Shows the progress of data as it flows through a pipe
  • Rackerlabs - Rackspace's OSS tools, hosted on GitHub
  • rcron - cron redundancy and failover, ensures a job will only run on the "active" machine
  • Repose - REST proxy
  • Request Tracker - Bug tracking, help desk ticketing, customer service, workflow processes, change management and more
  • reset - Reset your terminal after you accidentally cat a binary file
  • Rundeck - Easy automation
  • ScreenCloud - Take a screenshot and either upload it to their site to share, or save it locally
  • script - Standard utility to record commands, input, and output
  • Silver Searcher, The - Source code searching tool, similar to "ack" but faster
  • Simple Event Correlator (SEC) - Umm, no idea where to put this one, it's super multi-purpose
  • Snappy - Compression that aims for very high speed and resonable compression, by Google
  • SQL Fiddle - Write and test SQL
  • Sticky Notes - On-site / local pastebin
  • Sublime Text - Ok it's closed source and costs money but it's probably the best one out there
  • Synergy - Share a single keyboard and mouse with multiple physical computers
  • sysadmin-util - A collection of small but useful utilities
  • tac - Reverse "cat" (eg: last line displayed first)
  • tengine - A distribution of Nginx with some advanced features
  • TermRecord - Record and playback terminal sessions
  • Tools of the Trade - Massive list of SaaS, via Hacker News
  • tup - Super fast and efficient build system
  • UDR - UDT wrapper for rsync that improves throughput of large data transfers over long distances
  • Unix Rosetta Stone - The one and only resource for a mixed Linux/Unix environment
  • Varnish - Caching HTTP accelerator
  • watch - Run a command repeatedly at a specified interval and display the output
  • watch-make - Monitor Makefile dependencies and kick off make when one changes
  • watchman - Watch files and take action when they change (eg: kick off the CI system)
  • xCAT - Complete management solution for distributed systems
  • Your Own URL Shortener (YOURLS) - Lets you run your own URL shortener a'la TinyURL or bit.ly
  • watchr - Monitor a directory tree and kick off a script when a file changes, written in Ruby
  • Webjob - Downloads a program or script from a remote WebJob server, executes it, and sends back output to the server
  • zgrep - grep the contents of a gzip'ed file without uncompressing and piping
  • zopfli - Much slower than zlib but (almost) always compresses better

About

A collection of OSS tools and handy links

Resources

Readme
Activity

Stars

3 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published