How to survive^W live with Linux

How to survive live with Linux

↑Title for Windows users↑

Table of Contents

Introduction

Motivation

• Many questions about tools/tips/scripting/etc in Linux
• Many mentions about my life with Linux
• You should know, what you are losing (Linux vs Windows)
• Sharing experience and knowledge

Background

First time in 2006 (Windows Vista vs Mandriva: какую ОС выбрать в 2007 году?)

• Mandriva
• Ubuntu
• Runtu
• Fedora
• Debian
• Slackware
• SUSE
• MeeGo
• NixOS
• GuixSD
• Xubuntu

• CentOS
• Calculate
• Gentoo
• BolgenOS
• OpenWRT
• webOS
• ChromiumOS
• AstraLinux
• AltLinux
• Mageia
• Moblin

• Linux Mint
• Kubunutu
• Knoppix
• Damn Vulnerable Linux
• Backtrack
• Mythbuntu
• SteamOS
• Mandrake
• many others

Arch Linux since 2013

Open source community member (maintainer, developer)

Disclaimer

I’m the software evangelist I use

Not only Linux, but Open Source

Principles

KISS

keep it simple stupid

TUI

Text-based user interface

CLI

Command-line interface

OS

*It’s my opinion, for my workflow

ArchLinux

• KISS (no bloatware and unnecessary services)
• Stability & Reliability (rolling release)
• Arch Wiki
• pacman
• Arch User Repository
• Perfect learning base
• Arch Linux Community (no companies, no corporate ties)
• Flexibility & Customization

Why not others?

Ubuntu, Fedora, Mint, Manjaro, etc

Because these are very fat OSs:

• a lot of unused software
• many configuration files
• a lot of strange behavior

But for some reason these distributions are very useful

Gentoo, LFS

Fuss too much with them

NixOS, GuixSD

NixOS:

• DSL
• some my issues
• FHS (Filesystem Hierarchy Standard)

GuixSD:

• very unstable
• libre OS

Software

Mostly CLI and TUI

Internet

Network managers

• NetworkManager (nmcli, nmtui)
• systemd-networkd
• iwd

Tor and proxies

• SOCKS
  • tsocks, proxychains-ng
  • environment variables
• HTTP
  • httptunnel
  • proxytunnel
  • corkscrew
• torify (applications w/o proxy settings)

Web browsers

• w3m
• elinks

Firefox + extensions

Other VIM-based browsers are unstable and have less features (based on WebKit, didn’t have Rust Quantum engine):

• luakit
• surf
• qutebrowser

VIM plugins:

• Pendactyl (only Extended Support Release)
• Tridactyl
• Vimium
• Vimium C (current)

Hint mode

Console mode (Tridactyl)

youtube-dl

youtube-dl usage

Extractors list

tridactyl / etc + youtube-dl + mpv / mpd

=

video/audio ninja (you is godlike with i3)

Restrictions, advertising? What are you talking about?

curl — httpie

• simple using
• nice format output
• JSON parsing (curl + jq)
• etc

Example of default output

wget — aria2

• Faster (segmented downloading)
• BitTorrent extensions (DHT, PEX, Multi-Tracker, UDP tracker, local peer discovery, etc)
• More protocols (FTP, Metalink, etc)

Torrents

Transmission

• Client-server architecture
• Mobile client
• CLI client for scripts + TUI client — tremc

Torrent info

+peerflix+ WebTorrent

• Streaming torrent client
• Modules
• Media futures
• node.js & browser needed Rust/wasm version

Mail — (neo)mutt + OfflineIMAP + getmail + msmtp + etc

• Customization
  • Key bindings
  • Custom mail headers
  • Favorite editors
  • View, colors, sidebars
• Modules
  • Address books
  • IMAP, POP3, SMTP clients
  • Password managers
  • Everywhere in sh/bash

• GPG
• CLI
• No JS, HTML

Messengers

BitlBee - IRC gateway

• XMPP
• Hipchat
• Hangouts
• Twitter
• Skype
• Telegram
• Facebook
• VK

• Matrix
• Discord
• Steam
• Mastodon
• Slack
• ICQ
• etc

BitlBee interface

WeeChat - IRC

MCabber, profanity — XMPP

• PGP
• OTR
• Plugins/Modules
• Multi-User chat
• Command completion

profanity Pip-Boy

toxic — Tox

• P2P
• For secure conversation

tg-cli — Telegram, but telegram-desktop too

RSS, news

newsboat

As mutt, but for RSS

Telegram

Channels with bots, which sends RSS (Reddit, Twitter, HackerNews, Lobste.rs, etc) to channels.

Remote desktop

• Easy tasks

  ssh with X forwarding (use settings for compressing and encrypting traffic for speedup)

• Hard tasks
  • VNC
  • qemu with SPICE + VNC

Multimedia

Image viewers

Framebuffer (energy saving)

• fbi
• fbv

Don’t forget configure tty!

Simple

• feh
  • CLI and hotkey powerful
  • Very simple (so no screenshots)

• sxiv (more GUIably, than feh)
• vimiv (VIM image viewer)

ImageMagick

Very powerful image processing tool!

Throw out your GIMP. A joke of course. Throw out your Photoshop.

Music

mpd

• Server with clients:
  • mobile
  • GUI
  • TUI
  • CLI
• mpdscribble (Last.fm)

ncmpcpp (mpd client)

Video

Players

• mpv — most powerful video/audio player
• cvlc — CLI version of VLC

FFmpeg

Converting, hardware rendering, screencasting, input/output many formats (camera, screens, devices).

Metadata

Many interesting things you can know from metadata of video, images and data.

• ExifTool (audio, video, images, documents, etc)
• exiv2 (images)

Utilities

Terminal emulators

urxvt (rxvt-unicode)

• Old
• Universal
• Many patches and plugins developed in Perl
• Configurations in Xresources

Alacritty

• GPU rendering
• Rust
• Many features

Kitty

• GPU rendering
• Python
• More features (Unicode glyphs, many nice bindings, graphics, client-server SSH, multiplexer, etc)

Terminal multiplexer. tmux

Also byobu as more friendly and screen as older

File managers

ranger

• Many features
• Python
• Slow

ranger interface

vifm

• C
• VIM
• Diffing
• Filtering
• Searching
• Batch renaming

vifm interface

Git

You should learn it!

I use it everywhere:

• control version
• backups
• diffing
• merging
• review
• messaging
• file system
• etc

Gitea, GitLab, GitWeb as self-hosted Git-repository managers

QEMU (KVM)

A free and open-source emulator and virtualizer that can perform hardware virtualization.

• User-mode and system emulation
• The emulation of various architectures and periphery
• TUI, CLI
• Integration
• Open Source

WINE

Wine Is Not an Emulator is a free and open-source compatibility layer that aims to allow application software and computer games developed for Microsoft Windows to run on Unix-like operating systems.

• Debugging
• Backward compatibility
• Non-x86 architectures
• RE of Windows

Other versions and uses:

• CrossOver
• Cedega/WineX
• Darwine
• Pipelight
• Proton
• ReactOS
• Odin (OS/2)
• Parallels Desktop and VirtualBox
• WineD3D
• etc

Backups

restic

Encrypted snapshots, many backends + systemd

Text editing

Use doc{,x}, ppt{,x}, xls{,x} and others if you really needed it.

VIM

• Modern text editor (tabs, windows, buffers, syntax highlighting, spell checking, folding, etc)
• Modal editor (insert and normal modes + visual and line modes)
• Comprehensive help system
• Memory footprint is very low, faster than other
• Command centric. You can perform complex text related task with few commands
• Many plug-in available
• Native scripting (vimscript) and built-ins commands
• vimdiff, rview, rvim
• Ubiquitous

Emacs

• Client-server (share the buffer list, kill ring, undo and other)
• Pervasive help system with keybindings, functions and commands documented on the fly
• CLI, TUI, GUI
• Extensible and customizable Lisp programming language variant (Emacs Lisp)!..

• Ability to emulate vi and vim (using Evil, Viper or Vimpulse)
• A powerful and extensible file manager (dired), integrated debugger, and a large set of development and other tools
• Having every command be an Emacs Lisp function enables commands to DWIM (Do What I Mean). For example, a switch-or-split-window

“An OS inside an OS”

Rust ♥️ Emacs = remacs

Spacemacs & Doom

My choice:

Org-mode

Your Life in Plain Text

• Keeping notes
• Maintaining TODO lists
• Planning projects
• Authoring documents (à la Jupyter notebook, tablesheet, exporting to another formats)

This presentation created in org-mode and exported to HTML + reveal.js

Planning

Clocking

Agendas

Capturing

Tables

Exporting

Working with source code

LaTeX

\mode<presentation>
{
  \usetheme{dsec}
  \dsecset{progressbar = frametitle}
  \dsecset{subsectionpage = progressbar}
}
\title{Introduction to software-based
 microarchitectural side-channel attacks}
\subtitle{}

\author{Abc Xyz\\
@dura\_lex}
\titlegraphic{\includegraphics[height = 0.9cm]{logo.pdf}}

\date{}
\subject{Information security}

• Typesetting journal articles, technical reports, books, and slide presentations
• Control over large documents containing sectioning, cross-references, tables and figures
• Typesetting of complex mathematical formulas
• Advanced typesetting of mathematics with AMS-LaTeX
• Automatic generation of bibliographies and indexes
• Multi-lingual typesetting
• Inclusion of artwork, and process or spot colour

Zathura

Simple PDF reader

• VIM bindings
• Automatic reloading document
• Synctex
• Recoloring, reformatting
• CLI

Markdown

Use for work

AsciiDoc

Use for *real* work

• Including files
• Normal lists with complex content
• Anchors
• Formatted images
• Powerful code blocks
• Comments
• Tables for the people
• ToC
• Bibliography
• etc

Pandoc

Markup converter

• Markdown
• reStructuredText
• AsciiDoc
• Org-Mode
• Textile
• HTML
• CSV
• EPUB
• roff

• LaTeX
• DocBook
• OPML
• InDesign
• Wiki
• reveal.js
• Microsoft Word
• OpenOffice/LibreOffice
• etc

Spell checkers

Aspell, Hunspell + LanguageTool for grammars

Use my personal dict everywhere (editors, browser, shell, edit fields)

Tesseract OCR

• CLI
• API
• Multilanguages

Core utilities

• Standard utilities are old and have few features, but stable
• The list below is alternatives, not replace
• For scripts you should use first order applications for compatibility
• It’s normal, that more than half applications were developed in Rust language (r)

Bash — Zsh

• Featured aliases (suffix and global)
• Easy directory navigation
• Advanced completion
• Globbing
• Command line editing
• Spelling correction
• Themes and plugins (Oh My Zsh, Prezto, Powerlevel9k, etc)
• etc

Zsh features

ls — lsd (r)

Colorized and with icons ls

tree (ls) — exa (r)

More features and faster

cat + less, more — bat (r)

• Automatic paging
• Customization
• More features…

Syntax highlighting

Git integration

Show non-printable characters

cp, mv, rm — rsync

• Fast and extraordinarily versatile file copying
• Progress bar
• Delta-transfer
• Client-server architecture
• Can replace scp
• Can used as backup tool

du, df — ncdu

Interactive du and df

ncdu

tar, (un)zip — bsdtar, 7z

tar + (un)zip = bsdtar

• Automatic format detection
• Handles file flags, ACLs, arbitrary pathnames, etc
• Library. libarchive

find — fd (r)

• Convenient syntax: fd PATTERN instead of =find -iname ’PATTERN’=
• Colorized terminal output (similar to ls)
• It’s fast
• Smart case
• Ignores hidden directories and files, .gitignore by default
• Fast regular expressions
• Unicode-awareness
• The command name is 50% shorter* than find
• Parallel command execution with a syntax similar to GNU Parallel

diff — git-diff, colordiff, delta (r)

Use git-diff colordiff delta for diffing!

• Language syntax highlighting with color themes
• Within-line highlights based on a Levenshtein edit inference algorithm
• Git style strings (foreground color, background color, font attributes) are supported for >20 stylable elements
• Side-by-side view
• Line numbering

• diff-highlight and diff-so-fancy emulation modes
• Stylable box/line decorations to draw attention to commit, file and hunk header sections.
• n and N keybindings to move between files in large diffs, and between diffs in log -p views (--navigate)

delta with line numbers

delta with side-by-side

vimdiff (VIM), ediff (Emacs)

Powerful interactive diffing tools

3-way diff for merging

diffoscope

In-depth comparison of files, archives, and directories

• Compare many file formats (binary, archive)
• Fallback on hexdump comparison
• Fuzzy-matching to handle renamings

grep — ripgrep

• Fastest (Rust’s regex engine + paralleling)
• Unicode support by default
• Powerful regexp on Rust (but PCRE2 exist)
• Support searching files compressed (bzip2, gzip, lzma, xz, etc)
• Multiline searching
• Arbitrary input preprocessing filters

sed, {g,n,m}awk

If you are not using it yet, then it’s time

(u)mount — udisks2

Mounting with user’s permissions

pass

CLI pass manager. Powered by OpenPGP

fzf

A command-line fuzzy finder.

Live demo

systemd

Not Unix way, but very useful

• Journal: system logging
• Timers: reasonable alternative to cron
• systemd-boot: UEFI boot manager
• systemd-logind: session manager
• systemd-networkd: network configuration management

• systemd-nspawn: light-weight namespace container
• systemd-resolved: network name resolution
• systemd-timesyncd: system time synchronization across the network
• systemd-tmpfiles: temporary files
• etc

Simple and useful tools for RE and life

• man, apropos, mandb: throw out the Internet, you have manuals
• tee: read from standard input and write to standard output and files
• mktemp: create a temporary file or directory
• tr: translate or delete characters
• cut: remove sections from each line of files
• od, xxd, hexdump: custom hex view
• sort, uniq, comm: sort, delete duplicates, compare output

• head, tail
• wc: count of chars, lines, bytes
• strings
• iconv: convert text from one character encoding to another
• file: what is it?
• watch: execute a program periodically, showing output
• shred: burn after reading

• inotifywait: wait for changes to files using inotify
• ls{blk,pci,cpu,usb,mod}: block, PCI, CPU, USB devices, kernel modules
• sudo, but not su (sudoedit for editing files): I’m root
• pgrep, pidof: find process
• kill, pkill, killall: then kill it
• ps, htop: process list
• sleep: Zzz…

radare2

• rabin2
• radiff2
• rafind2
• ragg2
• rahash2
• rarun2
• rasign2
• rasm2
• rax2

r2

binwalk

• Disassembly scan
• Signature scan
• Extractions
• Entropy
• Binary diffing

Rust section

Install rustup: https://rustup.rs

$ cargo install package_name

and you on the horse!

• tokei: inspect source code
• jq: JSON prettifier
• xsv: very fast parsing of large CSV files
• fselect: searching on file system with SQL queries. find + ls + awk
• bingrep: color readelf, more powerful grep for ELF files

• ripgrepall: find everywhere: binary files, PDFs, images, music, video, etc
• stringsext: Better searching Unicode strings, but performance may be better
• tealdeer: tldr, documentation
• topgrade: upgrade your system
• choose: a human-friendly and fast alternative to cut and (sometimes) awk

Windows managers

i3

• Minimalism (i3 is fast)
• Screen real estate
• Keyboard-driven workflow
• Flexibility
• Workspaces

Live demo

Panel

• i3bar
• i3status
• i3status-rust
• i3blocks
• polybar
• conky
• j4status
• many many others

Live demo

dmenu rofi

Featured dynamic menu for X.

Live demo

dunst

Dunst is a highly configurable and lightweight notification daemon.

Live demo

Life + IT Workflow

Configurations

One configuration for everything: PC, Laptop, Work, Guest/Virtual PC, Smartphone, Router, Server etc?

• A custom shell scripts?
• A bare git repo?
• An existing dotfile manager: homeshick, rcm, GNU Stow, yadm, etc?

Chezmoi — manage your dotfiles across multiple machines, securely.

• Flexible: OS (Linux, MacOS, Windows, FreeBSD, Termux), Arch, Hostname
• Personal and secure (1Password, Bitwarden, gopass, KeePassXC, LastPass, pass, Vault, CLI tool and GnuPG)
• Transparent: dry run mode, one-to-one map of files and symlinks
• Declarative (templates, metadata in names) and robust (atomic operations)

My Dotfiles

File system cleanliness

• chezmoi unmanaged + fzf and ignored files
• Scripts for checking unneeded packages, installing packages
• Encrypted sensitive information and private repositories

Dark and light

This eye are looking at you like a Winduzyatnik

• Auto switching temperature of display (without light detector)
• Auto switching dark/light themes (no universal solution)

VIM/Emacs everywhere

• Don’t use a mouse, because it need more time, than a keyboard
• A home row of your keyboard is your friend: speedup and economy (keyboard with lights)
• Xdotool + Emacs = edit everything in Emacs
• VIM/Emacs keymaps and modes everything: browser, IDE, windows manager, multiplexer, music player, torrent manipulation programs, mail, everything… except Ghidra.

Presentations, reports, documents, messages

Only plaintext:

• diffing, manipulation with CLI tool
• using version control system
• emails without HTML and JS shits (Yandex.Mail, you place is in the hell)
• more informative, more convertible, more security

How I use it

I can convert org-mode (asciidoc) to everything: plaintext, static HTML, LaTeX, PDF, presentations (beamer, reveal.js). I can generate books and reports from plaintext.

I can share sources or make easy review and use parts of already wrote items.

I have many notes with ideas, presentations in structured plaintext format in version control systems.

I can sync all my documents (reports, books, presentations, ideas) between machines for working with it everywhere.

Smartphone

• AOSP or similar
• Open source utilities: F-Droid or GitHub + ADB
• Termux for PC’s work: SSH, running all Linux stuff (tmux, my Rust CLI programs), using Chezmoi configurations
• Sync with other devices: passwords, web history, music playlists, etc
• VPN for working with remote (home, work, laptops, other) devices

Subway workflow

• Reading issues, mailing lists, writing answers (GitLab, GitHub apps)
• Reviewing PR/MR, fixing small bugs (IDE or VIM if exist keyboard)
• Writing notes, drafts of presentation (including this) with orgzly (org-mode) and git for sync
• News from mail, RSS, RSS in Telegram, I don’t read news
• Reading books on jailbroken Amazon Kindle (all formats and many features)

Other devices (router, multimedia box, etc)

• All works on ArchLinux
• One configuration for everything
• Pi-hole, logger, route table, DNS server, proxy server with TLS, etc (replaced by Mikrotik)

Many firmwares are changed:

• Bookreader is jailbroken
• Printer is jailbroken
• Digital music player is jailbroken (Rockbox OS)
• Vacuum cleaner with Ubuntu but not jailbroken yet (but with Russian voice already)
• I bought a car…

Packages and open source

• Need a new software? Search in repositories and install it, that’s all.
• AUR or nix have many, many packages (Repology), including for Windows OS, BlackArch have packages for penetration testers

I have custom PKGBUILDs for all patched software (also cracks^Winitial settings for proprietary software).

I’m maintainer of some packages in AUR and I regulates other packages (asks to update, to delete, to change permissions of maintaining).

Automate everything

If you do the same thing a second time, then it needs to be automated.

• Analyzing the OS (packages, configurations, status, etc)
• Smart notifications (TODO, pomodoro, honeypots, China’a attacks, etc)
• Scraping of sites (buying tickets, news, generating API)
• All around RE/pentest
• Deleting complex ads (Yandex, for example)
• Specific subjects

JavaScript, PHP, Python, Rust, Bash

Statistics about everything

• Finance budget
• Books
• Movies, serials, TV shows
• Sports activity
• Board games
• Quotes and situations from life

Text vs GUI

I use text files for managing it all (filtering, creating reports, sharing information between machines), all is encrypted.

Summary

• Just a drop in the bucket
  • Master the command line
  • A book about CLI
• Technologies are evolving extremely rapidly
• Productivity = more free time
• It’s only my workflow