The ex command :g/re/p (grep) was the way they documented how to "globally" "print" lines containing a "regular expression" (re).
:... g is a prefix for "globally" applying a subsequent command to all lines which match a pattern (regular expression) while :... v applies such a command too all lines which do NOT match the given pattern ("v" from "conVerse").
Get machine type: arch; uname -a
Subshell hack:
$ cmd0 | ( cmd1;cmd2;cmd3) | cmd4
$ echo -e "Beijing Shanghai\n HongKong Chongqing" | ( while read wa wb; do echo "$wa to $wb"; done )
Beijing to Shanghai
HongKong to Chongqing
script and scriptreplay: record terminal sessions.
$ script -t 2> timing.log -a output.session
type commands;
...
...
exit
$ scriptreplay timing.log output.session
Bash process password
#!/bin/sh
#Filename: password.sh
echo -e "Enter password: "
stty -echo
read password
stty echo
echo
echo Password read.
Bash arrays
$ array_var=(1 2 3 4 5 6)
$ echo ${array_var[0]}
$ echo ${array_var[*]}
$ declare -A ass_array
$ ass_array=([index1]=val1 [index2]=val2)
$ declare -A fruits_value
$ fruits_value=([apple]='100dollars' [orange]='150 dollars')
$ echo "Apple costs ${fruits_value[apple]}"
Apple costs 100 dollars
$ declare -a MYRA
$ MYRA=($(ls -ld DIR))
find:
$ find . \( -name "*.txt" -o -name "*.pdf" \) -print
./text.pdf
./new.txt
$ find /home/users -path "*slynux*" -print
/home/users/list/slynux.txt
/home/users/slynux/eg.css
$ find . -iregex ".*\(\.py\|\.sh\)$"
./test.py
./new.PY
$ find . ! -name "*.txt" -print
.
./next.jpg
./test.py
./new.PY
$ find . -type f -user root -exec chown slynux {} \;
$ find devel/source_path \( -name ".git" -prune \) -o \( -type f -print \)
xargs:
$ cat cecho.sh
#!/bin/bash -
echo "$@ #"
$ echo "arg1 arg2 arg3" | xargs -n 2 ./cecho.sh
arg1 arg2 #
arg3 #
$ echo -e "arg1\n arg2\n arg3\n" | xargs -I {} ./cecho.sh -p {} -l
-p arg1 -l #
-p arg2 -l #
-p arg3 -l #
Hence we must use -print0 along with find to produce an output with delimited character null ('\0') whenever we use the find output as the xargs input.
$ find . -type f -name "*.txt" -print0 | xargs -0 rm -f
tput and stty are utilities that can be used for terminal manipulations.
tput sc store the cursor position
tput rc restore cursor position
tput ed clear rest of current line
{1..6} {a..z}
`#!/bin/bash –xv`
`:(){ :|:& };:`
A function can be exported like environment variables using export export -f fname
While using string comparison, it is best to use double square brackets since use of single
brackets can sometimes lead to errors. [[ $str1 = $str2 ]]
Ubuntu, hold down the alt key and click and drag anywhere in the window to move it so that all the buttons are visible
# echo "255" > /sys/devices/platform/i8042/serio1/sensitivity
# echo "250" > /sys/devices/platform/i8042/serio1/speed
The sgid permission for directory means that files created in it the directory will inherit its group affiliation from the directory, rather than inheriting it from user.
dmidecode # get Machine info
lsb_release -a # get Linux distribution info
hal
hdparm # get disks
Sometimes when i execute make command, i am getting Clock Skew detected error (your build may be incomplete).
This is a classic problem with VMs: it's hard to get an accurate clock when it's virtual. If this happens, run the following command from time to time:
/sbin/service ntpd restart
Use -h option, which requests that the system be either halted or powered off after it has been brought down, with the choice as to which left up to the system.
shutdown -h now
When compile new kernel from old version of CentOS, it is possible that the new kernel complains about no hard drives found. To solve this, enable the following two options in kernel config:
CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y
After make menuconfig, make, make modules_install, and make install, also update initramfs using
$sudo update-initramfs -c -k 3.4.7 # substitute 3.4.7 with your version
#lspci | grep -i eth
02:05.0 Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet
Controller (Copper) (rev 01)
02:05.1 Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet
Controller (Copper) (rev 01)
03:07.0 Ethernet controller: Intel Corporation 82541GI Gigabit Ethernet
Controller (rev 05)
List all PCI hardware info
# lspci -vv
http://kkovacs.eu/cool-but-obscure-unix-tools http://coolshell.cn/articles/7829.html
git://github.com/lzap/dancepill.git
Wajig is a simplified wrapper to Debian's package management tools including dpkg and APT. Wajig provides the functionality of apt-get, dpkg, dpkg-deb, apt-cache and other tools. These tools launch as a subprocess. Wajig also provides extra functionality beyond that of the stock apt and dpkg tools. For example, the command wajig sizes provides a listing of all installed packages and the amount of disk space they require, from smallest to largest.
$ cat /etc/*-release
$ lsb_release -a
$ hdparm -I
http://www.thegeekstuff.com/2010/07/logrotate-examples/
/etc/logrotate.conf
man logrotate
$ sudo apt-cache search manpages
$ sudo apt-get install manpages-dev
$ sudo apt-get install manpages-posix
dev_t typedef as an unsigned bit-or of major and minor dev no.
MAJORBITS
MAJOR
MINOR
$ echo "set bell-style none" >> ~/.inputrc
$ echo "set vb" >> ~/.vimrc
# bunzip2 file.bz2
blktrace
blkparse
btrecord
btreplay
if [[ $UID -ne 0 ]]; then
echo "$0 must run as root"
exit 1
fi
$ echo -e '#!/bin/bash -\n sleep 120' > sleep-1.sh
$ echo -e '#!/bin/bash -\n sleep 120' > sleep-2.sh
$ chmod +x ./sleep-*.sh
$ (./sleep-1.sh | ./sleep-2.sh) &
[4]
14794
$ echo $!
14794
$ ps -ef | grep slee-
mchen 14795 14794 0 17:49 pts/2 00:00:00 /bin/bash - ./sleep-1.sh
mchen 14796 14794 0 17:49 pts/2 00:00:00 /bin/bash - ./sleep-2.sh
mchen 14800 11348 0 17:49 pts/2 00:00:00 grep --color=auto sleep-
Check available settings:
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
conservative ondemand userspace powersave performance
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
2661000 2660000 2527000 2394000 2261000 2128000 1995000 1862000 1729000 1596000
Set governor and frequency
$ echo "conservative" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governors
$ echo "2661000" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_frequencies
As indicated in the online manual, the most common cause for that error is that lines are indented with whitespaces when make expects tab characters.
Correct
target:
\tcmd
where \t is TAB
Wrong
target:
cmd
ls | xargs echo
$ ipmitool -U root -H i-chevron8 chassis power reset
sudo apt-get install ncurses-term
apt-get install apt-file
apt-file search file_name
$sudo vim /etc/default/grub # change default index to the desired one
$sudo update-grub2
$ cat Makefile
OVERRIDING=default
run:
echo running with $(ARG) and $(OVERRIDING)
$ make run
echo running with and default
running with and default
$ make run ARG=a
echo running with a and default
running with a and default
$ make run ARG=a OVERRIDING="a little more"
echo running with a and a little more
running with a and a little more
Fail through of bash is only available after bash 4.0
case a in
1) echo ;&
2) echo ;;
esac
$ usermod -G groupname username
$ groups username # view groups belongs to
$ id username # view groups names and numeric values
$ cat /etc/group
Set group
$ usermod -G groupname username
$ echo 7 > /proc/sys/kernel/printk
cd ~
ls -ln
cat /etc/*release*
yum install perl
yum install perl-CPAN
/sbin/dumpe2fs /dev/sda | grep 'Block size'
blockdev --getsize /dev/sda
debug:
ssh -vvv user@host
On ssh server
echo "UseDNS no" >> /etc/ssh/sshd_config
echo "GSSAPIAuthentication=no" >> /etc/ssh/sshd_config
alias ghadoop='luit -encoding gbk ssh -o GSSAPIAuthentication=no ime@10.12.11.200'
useradd loginname
passwd loginname
usermod -d /search/chenming -m chenming
dig @8.8.8.8 hostname
nslookup -debug hostname 8.8.8.8
host hostname
cat /etc/resolv.conf
apt-cache search package_regex
yum search package_regex
sudo apt-get install alien
sudo alien -i /path/to/file.rpm
rpm --checksig packagename.rpm
rpm -K --nopgp packagename.rpm
The corresponding commands for deb packages are
dpkg -I | --info packagename.deb
dpkg -e | --control packagename.deb
./configure --help
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
--target=TARGET configure for building compilers for TARGET [HOST]
Find open ports
$ grep port /etc/services
$ grep 2432 /etc/services
Few quick methods/programs that we can use to find out running processes and associated ports
$ netstat -tulpn
fuser to view process occupying given port
$ fuser port_num/tcp_or_udp
$ fuser 50075/tcp
50075/tcp: 7535
To make usermod take effect, the user involved should logout and re-login
# groupadd -g 503 ime
# usermod -g 'init_login_gid' -G 'group1, group2' loginname
# usermod -g ime -G ime,root chenming
# chown -R chenming:ime /home/chenming
# id chenming
Bourne-type shells, such as bash , usually have $ in the prompt. The C shell
uses % (but tcsh users often use >).
The shell doesn't start a separate process to run internal commands (such as cd). External commands require the shell to fork and exec (such as ls) a new subprocess.
When you first log in to a Unix system, the login program performs various security measures. First, login checks to see if you are not the root user and whether the file /etc/nologin exists (it is created by the shutdown command to indicate that the system is being brought down for maintenance). If both of these conditions are true, the login program prints the contents of that file to the terminal, and the login fails.
Second, login checks to see if there are any special conditions on your login attempt (which may be defined in /etc/usertty or /etc/securetty), such as on which tty you're coming in. Linux systems do this, for example. Some systems (such as Darwin and other BSD-based systems) also check /etc/fbtab and may restrict your access to any devices listed in that file. These systems may also log failed login attempts to a file, such as /var/log/failedlogin, if it exists. login may also record the time of login in the file /var/log/lastlog, make an entry in the file /var/run/utmp, showing that you are successfully logged in (it is removed once you log out), and append the utmp entry to the file /var/log/wtmp , showing that you logged in. This wtmp record will be updated on logout, showing the duration of your login session.
If the file .hushlogin exists in the user's home directory, the login will be quiet; otherwise, the following sequence of events will occur. If the system has any special copyright information to display, it will be printed to the terminal, followed by the message of the day (usually stored in /etc/motd ), and the user's last login time and system of origin (from the wtmp file, as discussed in the previous paragraph).
Finally, if all other checks have passed and restrictions have been performed, login starts a shell for you. Which shell depends on what is set in your user database entry (/etc/passwd, NIS, or possibly NetInfo under Darwin). If the shell specified for you is not interactive (Section 3.4), you may well be denied a command line. This is common for POP and ftp-only user accounts, where /bin/true and /bin/false are often specified as shells to disallow shell logins from those accounts.
Don't have a terminal with multiple windows or another terminal close to your desk? You can get the same result by using rlogin, su or telnet to log in to your host again from the same terminal.
% vi .cshrc
...Make edits to the file...
% rlogin localhost
...Logs you in to your same account...
An error message
% logout
Connection closed.
% vi .cshrc
...Edit to fix mistake...
$ grep -v # reVersed, find items without pattern
$ grep -q # quiet, answer found or not by $?
$ grep -h # do not print filename
$ grep -l # just print filename
$ zgrep regex xxx.gz # search compressed file
$ zcat # cat compressed file
$ rm -i $(grep -l 'This file is obsolete' * )
/etc/hosts is for mapping IP addresses to hosts. To change the hostname, it depends on the distro you're running. it's usually /etc/hostname or /etc/HOSTNAME or /etc/sysconfig/network. hostname command can temporary change hostname.
$ mkisofs -V LABEL -r dir | gzip > cdrom.iso.gz
#Create cdrom image from contents of dir
$ mount -o loop cdrom.iso /mnt/dir
#Mount the cdrom image at /mnt/dir (read only)
$ gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom -
#Burn cdrom image (use dev=ATAPI -scanbus to confirm dev)
split(string, chars, "") # convert string into array of chars
split("", array) # empty array
split(string, fields, "[ ]") # use a single blank as separator
$ ls -u # access time, visit time
$ ls -c # modify time, time of inode metadata modified
$ ls -l # change time, time of file content changed
$ cp -p # preserve change time of file
$ chroot # prevent malicious escape to the parent directory
$ bash -n scriptname # check bash script syntax
Take the time to learn a bit about sudo itself and the /etc/sudoers file. You should always use visudo to edit your /etc/sudoers file. Like vipw, visudo locks the file so that only one person can edit it at a time, and it performs some syntax sanity checks before replacing the production file so that you don’t accidentally lock yourself out of your system.
ssh: mkdir -m 0700 -p ~/.ssh. Your ~/.ssh directory must be mode 0700 or OpenSSH will complain. It’s not a bad idea to use chmod 0600 ~/.ssh/authorized_keys as well.
Automatically log out. Set the $TMOUT environment variable in /etc/bashrc or ~/.bashrc to the number of seconds of inactivity before ending the session.
$ declare -r TMOUT=3600 # or
$ readonly TMOUT=3600
$ less /proc/cpuinfo
$ pstree
$ ps -u chenming
$ refdate="2007-09-03"
$ sec=`date -d "$refdate" +s`
$ let sec=$((sec - N*24*60*60))
$ date -d "1970-01-01 UTC $sec seconds" +'%Y%m%d'
$ date -d "$refdate $N day ago" +'%Y%m%d'
function prevdate() {
local ref="$1"
local num="${2:-1}"
local refdate=`date -d "${ref}" +'%Y-%m-%d'`
date -d "$refdate $num days ago" +'%Y%m%d'
}
$ sudo dpkg -i package.deb
$ top
$ cat /proc/meminfo
$ free
$ abpath=`readlink -m $1`
reducer_dist_LDFLAGS=-static-libtool-libs # Makefile.am
--enable-static --enable-profile --enable-debug --disable-optimize
autoconf copies the contents of 'configure.in' to 'configure', expanding macros as they occur in the input.
Make. Target rules start at the beginning of a line and are followed by a colon. Following the colon is a whitespace separated list of dependencies. A series of lines follow which contain shell commands to be run by a sub-shell. Each of these lines must be prefixed by a horizontal tab character.
These commands may be prefixed by an '@' character to prevent make from echoing the command line prior to executing it.
They may also optionally be prefixed by a '-' character to allow the rule to continue if the command returns a non-zero exit code.
$@ represent the names of the target
$< the first dependency
$ cat Makefile
all: dummy
@echo "$@ depends on dummy"
dummy:
touch $@
$ make
touch dummy
all depends on dummy
Makefile's suffix rule:
$.c.o:
$(CC) $(CFLAGS) -c $< -o $@
Unless a more specific rule matches the target being sought, this rule will match any target that ends in '.o'. These files are said to always be dependent on '.c'.
$ shopt -s extglob
$ mkdir ~/.vim/{plugin,doc,syntax} -p
$ help shopt # set shell option
$ arch # view machine arch
$ dd # convert and format data
$ od -An -j4 -N4 -t 4d # view binary data
$ hexdump -n 16 -s [byte_to_skip] -e '4/4 "%d\n"' bin
$ help compgen # complete, autocomplete bash, complete globbing
$ ldd *.so
$ nm *.obj
$ ranlib archive
$ ar archive
/etc/ld.so.cache
ldconfig
$ wc -L file
$ sort -T ~/tmp
$ export TMPDIR=~/tmp
sudo apt-get autoremove
Download and install openssl and openssh.
set "system" function showld never be used from a set-user-ID or a set-group-ID program
tty stands for teletypewriter, an early form of terminal.
The first text editors were "line-oriented", such as "ed" and later "ex". "Vi" was developed later, based on "ex", and was screen-oriented. It used the redrawable ability of cathode ray tube (CRT) displays to show text one screen at a time.
When call functions that changes errno in signal handler, we should save and restore error.
install your public key in a remote machine's authorized_keys
$ ssh-copy-id mchen@msl.cewit.stonybrook.edu
$ cat >> ~/.ssh/config
Host msl
HostName msl.cewit.stonybrook.edu
User mchen
Host chevron8
User mchen
ProxyCommand ssh msl -W %h:%p
$
$ ssh-copy-id chevron8
$ sudo aptitude install pdftk
$ pdftk largepdfile.pdf burst
$ pdftk *.pdf cat output onelargepdfile.pdf
$ zip archivefile1 doc1 doc2 doc3
By default, libevent install the shared library into /usr/local/lib
# echo "/usr/local/lib" >> /etc/ld.so.conf
# ldconfig
utmpdump /var/log/wtmp
http://stackoverflow.com/questions/2549214/interpreting-segfault-messages
net rpc shutdown -I ipAddressOfWindowsPC -U username%password
Messages produced by show_signal_msg in the kernel.
proxy.ganesha.n[22172]: segfault at 16388 ip 000000000042d46d sp 00007fff78a1c7f0 error 4 in proxy.ganesha.nfsd[400000+153000]
address - the location in memory the code is trying to access ip - instruction pointer, ie. where the code which is trying to do this lives sp - stack pointer error - is not an errno nor a signal number, it's a "page fault error code" -- see definition of enum x86_pf_error_code.
/*
* Page fault error code bits:
*
* bit 0 == 0: no page found 1: protection fault
* bit 1 == 0: read access 1: write access
* bit 2 == 0: kernel-mode access 1: user-mode access
* bit 3 == 1: use of reserved bit detected
* bit 4 == 1: fault was an intruction fetch
*/
enum x86_pf_error_code {
PF_PROT = 1 << 0,
PF_WRITE = 1 << 1,
PF_USER = 1 << 2,
PF_RSVD = 1 << 3,
PF_INSTR = 1 << 4,
};
[400000+153000] is starting address and size of virtual memory area
where offending object was mapped at the time of crash. Value of "ip" should
fit in this region. With this info in hand, it should be easy to find offending
code in gdb.
http://www.centos.org/modules/newbb/viewtopic.php?topic_id=35275&forum=55
# yum --nogpgcheck --enablerepo=debug install glibc-debuginfo
edit /etc/yum.repos.d/CentOS-Debuginfo.repo, set enabled=1
wget -r --no-parent -A.txt http://www.abc.org/folder/
$ convert rose.jpg -resize 50% rose.png # resize
$ convert rose.jpg -rotate 180 rose.png
The kernel handles interrupts by delivering to the current process one of the signals familiar to every Unix programmer; handles exceptions to recover from the anomalous condition, such as a Page Fault or a request via an assembly language instruction such as int or sysenter for a kernel service.
http://www.thegeekstuff.com/2013/09/man-command/
$ man man
Search in NAME section
$ man -f printf
printf (3) - formatted output conversion
printf (1) - format and print data
Search in NAME and DESCRIPTION section
$ man -k printf
asprintf (3) - print to allocated string
dprintf (3) - print to a file descriptor
fprintf (3) - formatted output conversion
fwprintf (3) - formatted wide-character output conversion
printf (1) - format and print data
printf (3) - formatted output conversion
snprintf (3) - formatted output conversion
sprintf (3) - formatted output conversion
If you ever happen to want to link against installed libraries in a given directory, LIBDIR, you must either use libtool, and specify the full pathname of the library, or use the `-LLIBDIR' flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
$ udevadm info --query=all --name=/dev/sda
yum install epel-release
yum makecache
yum check-update