Shell

• URL Encoding

str=`tail giipAgent.log`

echo "original string"
echo $str

encode=$(echo $str | jq -Rr '@uri')

echo "URL Encoded..."
echo $encode

• Performance check

  • https://yakst.com/ja/posts/3601)
  • https://qiita.com/saikoro-steak/items/0ae51b4f5de520b37a44

• Repository management

  • https://server-setting.info/centos/repository-setting.html
  • https://qiita.com/bezeklik/items/9766003c19f9664602fe
    • Repository list

• Kill all process fixed name

  • ps aux | grep <processname> | grep -v grep | awk '{ print "kill -9", $2 }' | sh
  • https://qiita.com/masarufuruya/items/409679c1006980ef1b60

• split - separate the file

  • https://uxmilk.jp/8584

  split -d -b 99M -a 3 orgfile.log split_file

• options

  • -d : change sub-string to number
  • -b : split each bytes
  • -a n : split number of n

• Check word contains the file(word exists)

strTxt=`cat ../gitrst.txt`
if [[ $strTxt == *"file changed"* ]]; then
  echo "File changed!!!"
fi

• check file size

sFileName="<filename>"
if [ -s $sFileName ]; then
  echo "File size is not zero."
fi

• file exists

sFileName="<filename>"
if [ -e $sFileName ]; then
  echo "File exist."
fi

Basic command

• date
  • Various converting datetime string : https://talklowykr.blogspot.com/2020/06/shell-date-command-examples-linux-bash.html

dtnow=`date +%Y%m%d`
dtprev=`date -d "1 day ago" +%Y%m%d`

echo "Today:${dtnow}, Yesterday:${dtprev}"

• du

  • Show directory status
  • du -b /data | sort -n | grep -v test >du-data.txt
    • -b : show usage by byte
    • sort -n : sort by number(big size directory is below)
    • /data : target directory

• ls -lt

  • show files sorting date

• mv

  • for i in `find /orgpath -mtime +15 -type f`; do mv $i /targetpath; done
    • The files over 15 days from /orgpath to /targetpath

• if you can not work cp and mv force on shell scripts, then use it

  • \cp -f origin.txt target.txt
  • \mv -f origin.txt target.txt
  • ignore alias command recent version of linux.

• diff FileA FileB

  • compare with FileA and FileB.

• fdisk

  • fdisk -l
    • list physical devices and format status
  • fdisk /dev/sdb
    • partitioning /dev/sdb
    • p : Check status selected partition
    • g : gpt mode change
    • n : Create new partition
    • w : Save changed
  • mkfs.ext4 /dev/sdb1
    • format partition of sdb1
  • mkdir -p /data2
  • mount /dev/sdb1 /data2

• get current directory name

  • name only : basename ``pwd``
  • full path : pwd

Advanced scripting

execute by condition

• git pull and changed target file, then execute it

git pull origin master >../${svrname}-gitsyncstat.txt
strTxt=`cat ../${svrname}-gitsyncstat.txt`

if [[ $strTxt == *"lwcmd.sh"* ]]; then
  # execute all fw rule
  sh sh/lwcmd.sh
  rm -rf lwcmd.sh
fi

• How to Delete Files Older than 30 days in Linux

• Various if condition process

  • (ja) https://qiita.com/toshihirock/items/461da0f60f975f6acb10

String handling

It can be changed / to | when error on shell

#before
sed -e "s/PermitRootLogin yes/PermitRootLogin no/g" /etc/ssh/sshd_config>/etc/ssh/sshd_config_bak

#after
sed -e "s|PermitRootLogin yes|PermitRootLogin no"|g" /etc/ssh/sshd_config>/etc/ssh/sshd_config_bak

• using one method without MID, SUBSTR, LEFT, RIGHT

lwStr="giipasp.azurewebsites.net"

echo ${lwStr:0:2}
# -> gi
echo ${lwStr:4}
# -> .net
echo ${lwStr:0:-4}
# -> giipasp.azurewebsites
echo ${lwStr%.*}
# -> giipasp.azurewebsites
echo ${lwStr%%.*}
# -> giipasp
echo ${lwStr#*.}
# -> azurewebsites.net
echo ${lwStr##*.}
# -> net

Server Configuration

disable root on ssh connection

• Make sure create user by adduser and connect test before this process!!

sshblockroot.sh

cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

sed -e "s/PermitRootLogin yes/PermitRootLogin no/g" /etc/ssh/sshd_config>/etc/ssh/sshd_config_bak
sed -e "s/#PermitRootLogin/PermitRootLogin/g" /etc/ssh/sshd_config_bak>/etc/ssh/sshd_config
rm -f /etc/ssh/sshd_config_bak

service sshd restart

https://talklowykr.blogspot.com/2020/01/linux-root.html#more

• add sudoer to user

usermod -aG wheel mylogin

• check user group

cat /etc/group | grep mylogin

• check sudoers file open wheel

cat /etc/sudoers | grep wheel

notice : delete comment when commented

• https://knooto.info/virtualbox-centos69-sudo/

• handling curl status

  • https://stackoverflow.com/questions/2220301/how-to-evaluate-http-response-codes-from-bash-shell-script

• Trim string

  • (ja) https://genzouw.com/entry/2019/02/14/081819

• Get PID

mypid=`echo $$`
echo "$mypid"

• シェルスクリプトの実行を一時停止させる (pause)

jq(JSON Converter CLI)

• jq
  • https://medium.com/veltra-engineering/jq-supports-json-to-csv-fb5c951a9575
  • https://qiita.com/takeshinoda@github/items/2dec7a72930ec1f658af
  • using bash : https://qiita.com/ryo0301/items/2ac9d11f355f1cf52ea5

datetime

• Datetime : https://talklowykr.blogspot.com/2020/06/shell-date-command-examples-linux-bash.html

• Change Linux Time

### backup org localtime
cp /etc/localtime /etc/localtime.org

### Change to JST
ln -sf  /usr/share/zoneinfo/Asia/Tokyo /etc/localtime

vi editor

• vi hot key : https://prev.net-newbie.com/linux/commands/vi.html

Change Terminal Color

by shell

cp /etc/DIR_COLORS ~/.dircolors
vi ~/.dircolors
``

Find and change color code `~/.dircolors`
```vim
DIR 01;36       # directory
LINK 36;01      # symbolic link 

httpd

• Apache の情報をコマンドラインで取得する

httpd -t -D DUMP_CONFIG 2>/dev/null | grep '# In' | awk '{print $4}' # 設定ファイル一覧
httpd -t -D DUMP_CONFIG 2>/dev/null | grep -v '#'                    # 設定ファイル内容
httpd -V                 # コンパイル時の設定
httpd -S -D SSL          # 実行時の設定
httpd -l                 # 静的モジュール一覧
httpd -M                 # ロード済みモジュール一覧
rpm -qi httpd            # パッケージ情報
rpm -q --changelog httpd # パッケージ更新履歴

Links

• KnowledgeBase
  • Shell

• Test if a Website Is Available - bash shell
  • check web site and mail with response http code
• 【簡単】Webサイトの差分チェック＋LINE通知をシェルスクリプトでやる

• 直接叩くと実行できるシェルスクリプトをcronで実行するとエラーになる件
• How to print the IP and the Port separately with netstat?

• wget의 여러가지 사용 방법
• シェルスクリプトでWebサービスの動作確認＆エビデンスを作ろう