Skip to content

charlietag/ubuntu_preparation

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

296 Commits

databag

databag

 
 

functions

functions

 
 

helpers

helpers

 
 

helpers_views

helpers_views

 
 

plugins

plugins

 
 

tasks

tasks

 
 

templates

templates

 
 

tmp

tmp

 
 

.gitignore

.gitignore

 
 

LICENSE

LICENSE

 
 

README.md

README.md

 
 

start.sh

start.sh

 
 

Repository files navigation

Table of Contents

Ubuntu Linux Server OS Preparation

You want initialize your linux server by your own script. But you DO NOT want to use PUPPET , CHEF , Ansible. You can just leverage this initialization project.

This is a small light bash project. Suit small companies which have only few servers to maintain. GIVE IT A TRY!!

Ubuntu server 22.04 environment settings

  • This is useful when

    • You have less than 5 Ubuntu servers to maintain.
    • You are deploying monolithic architecture app.
    • You are building Ruby on Rails / Laravel dev server.

  • This repo contains packages below

    • NGINX + PUMA + PHP-FPM + MariaDB + Rails + Laravel + Redmine + Redis

Environment

  • Ubuntu 24.04
    • ubuntu_preparation
      • release : main v2.x.x
  • Ubuntu 22.04
    • ubuntu_preparation
      • release : v1.x.x

Notice

  • Before ubuntu_security
  • Systemd target
    • Default target (ubuntu_preparation will force to use this target)
      • multi-user.target
        • This command will be executed systemctl set-default multi-user
    • Comparision
      • multi-user.target: analogous to runlevel 3
      • graphical.target: analogous to runlevel 5
    • WARNING If you are under graphical.target NOT under multi-user.target.
      • It is highly recommended that you do the following:
        • Reinstall whole Ubuntu using "Ubuntu Server" Ubuntu Server (minimized)
      • Reference description here
    • Check method
      • systemctl get-default

Warning

  • Please do this in fresh install OS
  • What does this not cover, DO the following manually

    • Login user

      • Change password of root

      • Add GENERAL USER and setup password of GENERAL USER (link)

        useradd -m -s /bin/bash {user}

      • Make GENERAL USER identifiable

        cat << EOF >> /home/{user}/.bashrc
PS1='\${debian_chroot:+(\$debian_chroot)}\u@\h:\w\\$ '
EOF

    • /etc/ssh/sshd_config

      • PermitRootLogin no
      • PasswordAuthentication yes

    • RAM

      • mkswap if RAM is insufficient to start MariaDB

        mkdir /swap
dd if=/dev/zero of=/swap/swapfile bs=1M count=4096
mkswap /swap/swapfile
chmod 0600 /swap/swapfile
/sbin/swapon /swap/swapfile
        cat << EOF > /etc/rc.local
#!/bin/bash
touch /var/lock/subsys/local
/sbin/swapon /swap/swapfile
EOF

chmod 755 /etc/rc.local

        (based on rc-local.service)

Configuration

  • ssh without SendEnv

    • command ssh

      • ssh_to

      • config /etc/ssh/ssh_config

        # SendEnv LANG LC_*

    • iTerm2 setting iterm2_disable_setting_LC_ALL.png

  • Before installation

    apt clean
apt update
apt install git -y
git clone https://github.com/charlietag/ubuntu_preparation.git

  • Make sure config files exists , you can copy from sample to modify.

    cd databag
ls |xargs -I{} bash -c "cp {} \$(echo {}|sed 's/\.sample//g')"

  • Mostly used configuration :

    • DEV use (server in Local / server in Cloud) && Production use (server in Local / server in Cloud)

      databag/
├── F_00_OS_02_env.cfg
├── F_01_ENV_04_ssh_config.cfg
└── _gitconfig.cfg

    • IP / DNS / NTP (server in Local / server in Cloud) for who needs customization IP, DNS, NTP

      databag/
├── F_00_OS_01_ip.cfg
└── F_00_OS_02_env.cfg

  • Verify config files (with syntax color).

    cd databag

echo ; \
ls *.cfg | xargs -I{} bash -c " \
echo -e '\e[0;33m'; \
echo ---------------------------; \
echo {}; \
echo ---------------------------; \
echo -n -e '\033[00m' ; \
echo -n -e '\e[0;32m'; \
cat {} | grep -vE '^\s*#' |sed '/^\s*$/d'; \
echo -e '\033[00m' ; \
echo "

  • Verify ONLY modified config files (with syntax color).

    cd databag

echo ; \
ls *.cfg | xargs -I{} bash -c " \
echo -e '\e[0;33m'; \
echo ---------------------------; \
echo {}; \
echo ---------------------------; \
echo -n -e '\033[00m' ; \
echo -n -e '\e[0;32m'; \
cat {} | grep -v 'plugin_load_databag.sh' | grep -vE '^\s*#' |sed '/^\s*$/d'; \
echo -e '\033[00m' ; \
echo "

Easy Installation

I'm a lazy person. I want to install ALL and give me default configurations running Nginx , MariaDB, php-fpm, puma 5 (rails), redis. And help me to create default projects about "Rails" and "Laravel"

  • Command

    ./start.sh -a
reboot

  • disable geoipupdate timer (for some cases) - ubuntu_preparation makes this default

    systemctl list-unit-files |grep -i ^geoipupdate | awk '{print $1}' | xargs | xargs -I{} bash -c "systemctl stop {}; systemctl disable {}"

  • Default project path

    • DEFAULT user for rails/laravel developer is not ssh allowed

      • /etc/ssh/sshd_config

        DenyGroups no-ssh-group

    • group "no-ssh-group" add to default dev user

      • phpuser (this name can be modified)
      • rubyuser (this name can be modified)
      • jsuser (this name can be modified)
      • pythonuser (this name can be modified)
      • podmanuser (this name can be modified)

    • rails

      • default user: rubyuser (can be changed)
    /home/${current_user}/rails_sites/myrails/
--->
/home/rubyuser/rails_sites/myrails/
    • Redmine
      • default user: rubyuser (can be changed)
    /home/${current_user}/rails_sites/redmine/
--->
/home/rubyuser/rails_sites/redmine/
    • laravel
      • default user: phpuser (can be changed)
    /home/${current_user}/laravel_sites/myrails/
--->
/home/phpuser/laravel_sites/myrails/

  • Config your client hosts file (/etc/hosts) for browser

    <192.168.x.x> myrails.ubuntu22.localdomain
<192.168.x.x> redmine.ubuntu22.localdomain
<192.168.x.x> mylaravel.ubuntu22.localdomain

  • Browse URL

    http://myrails.ubuntu22.localdomain
http://redmine.ubuntu22.localdomain (default account: admin/admin)
http://mylaravel.ubuntu22.localdomain

Advanced Installation

I want to choose specific part to install.

  • Command

    ./start.sh -h
usage: start.sh
  -a                   ,  run all functions
  -i func1 func2 func3 ,  run specified functions

Customize your own function

Folder

  • functions/

    • Write your own script here, file named start with F_[0-9][0-9]_YourOwnFuntionName.sh

    • Run command

      ./start.sh -i YourOwnFuntionName

  • templates/

    • Put your own templates here, folder named the same as YourOwnFuntionName

  • databag/

    • Put your special config variables here, file named the same as YourOwnFuntionName
    • How to use

      • In databag/YourOwnFunctionName

        • local your_vars_here

      • In templates/YourOwnFunctionName/yourowntemplate_file

        • You can use ${your_vars_here}

      • In YourOwnFuntionName , you can call

        # Method : eval "echo \"$variable\""
# Might have escape issue, if template is complicated
RENDER_CP ${$CONFIG_FOLDER}/yourowntemplate_file /SomeWhere/somewhere
        # Method : cat template | sed 's/\{\{var\}\}/$var/g'
# BETTER method for rendering template
RENDER_CP_SED ${$CONFIG_FOLDER}/yourowntemplate_file /SomeWhere/somewhere

        instead of

        cp ${$CONFIG_FOLDER}/yourowntemplate_file /SomeWhere/somewhere

      • In YourOwnFuntionName , you just want to LOAD VARIABLES ONLY from databag, try add a comment into your function script

        # For Load Variables Only Usage, add the following single comment line with keyword DATABAG_CFG:enable
# DATABAG_CFG:enable

  • helpers/

    • Write your own script here, file named start with helper_YourOwnHelperName.sh
    • Works with helpers_views

  • helpers_views/

    • Put your own templates for ONLY helper USE here, folder named the same as YourOwnHelperName

  • tasks/

    • Write your own script here, file named start with task_YourOwnTaskName.sh , _task_YourOwnTaskName.sh
    • Scripts here will automatically transfer to function, just like scripts under "functions/"
    • But this is for global use for ubuntu_preparation , ubuntu_security. So it's been moved to ubuntu_preparation_lib

  • plugins/

    • Only scripts which can be called everywhere like, ${HELPERS}/plugins_scripts.sh
    • Use this as a script, not function

Predefined variables

(root)# ./start.sh -i F_00_debug
#############################################
         Preparing required lib
#############################################
Updating required lib to lastest version...
Already up to date.

#############################################
            Running start.sh
#############################################

---------------------------------------------------
NTP(systemd-timesyncd) ---> pool.ntp.org
---------------------------------------------------
---------------------------------------------------


==========================================================================================
        F_00_debug
==========================================================================================
-----------lib use only--------
CURRENT_SCRIPT : /root/ubuntu_preparation/start.sh
CURRENT_FOLDER : /root/ubuntu_preparation
FUNCTIONS      : /root/ubuntu_preparation/functions
LIB            : /root/ubuntu_preparation/../ubuntu_preparation_lib/lib
TEMPLATES      : /root/ubuntu_preparation/templates
TASKS          : /root/ubuntu_preparation/../ubuntu_preparation_lib/tasks
HELPERS        : /root/ubuntu_preparation/helpers
HELPERS_VIEWS  : /root/ubuntu_preparation/helpers_views

-----------lib use only - predefined vars--------
FIRST_ARGV     : -i
ALL_ARGVS      : F_00_debug

-----------function use only--------
PLUGINS            : /root/ubuntu_preparation/plugins
TMP                : /root/ubuntu_preparation/tmp
CONFIG_FOLDER      : /root/ubuntu_preparation/templates/F_00_debug
DATABAG            : /root/ubuntu_preparation/databag
DATABAG_FILE       : /root/ubuntu_preparation/databag/F_00_debug.cfg

-----------function extended use only--------
IF_IS_SOURCED_SCRIPT  : True: use 'return 0' to skip script
IF_IS_FUNCTION        : True: use 'return 0' to skip script
IF_IS_SOURCED_OR_FUNCTION  : True: use 'return 0' to skip script

${BASH_SOURCE[0]}    : /root/ubuntu_preparation/functions/F_00_debug.sh
${0}                 : ./start.sh
${FUNCNAME[@]}          : source F_00_debug L_RUN L_RUN_SPECIFIED_FUNC source source main
Skip script sample    : [[ -n "$(eval "${IF_IS_SOURCED_OR_FUNCTION}")" ]] && return 0 || exit 0
Skip script sample short : eval "${SKIP_SCRIPT}"

================= Testing ===============
----------Helper Debug Use-------->>>

-------------------------------------------------------------------
        helper_debug
-------------------------------------------------------------------
HELPER_VIEW_FOLDER : /root/ubuntu_preparation/helpers_views/helper_debug


----------Task Debug Use-------->>>

-----------------------------------------------
        task_debug
-----------------------------------------------

Note

Installed Packages

Folder privilege

After this installation repo, the server will setup with "Nginx + Puma (socket)" , "Nginx + PHP-FPM (socket)" , so your Rails, Laravel, can run on the same server. The following is something you have to keep an eye on it.

  1. folder privilege

  • Rails Project

    rails new <rails_project> -d trilogy -j bun -c bootstrap
cd <rails_project>
chown -R ${current_user}:${current_user} log tmp

  • Laravel Project

    composer create-project --prefer-dist laravel/laravel <laravel_project>
cd <laravel_project>
chown -R ${current_user}:${current_user} storage
chown -R ${current_user}:${current_user} bootstrap/cache
  1. Command

  • Rails

    rails new <rails_project> -d trilogy -j bun -c bootstrap

  • Rails 7 has intergrated with stimulusjs, stop using jquery

  • Rails 5.1 has dropped dependency on jQuery, you might want it back via yarn

    1. Add npm of jquery using Yarn

      cd <rails_project>
yarn add jquery

    2. Setup jquery npm for asset pipeline

      vi <rails_project>/app/assets/javascripts/application.js
      //= require rails-ujs
//= require turbolinks
//= require jquery/dist/jquery
//= require bootstrap/dist/js/bootstrap
//= require_tree .

    3. Yarn works with rails 5.1 asset pipeline as below

    • Usage for default path: <rails_project>/node_modules/{pkg_name}/dist/{pkgname}.{js,css}

      //= require jquery

    • If package is different from this rule, ex: bootstrap. You might specify explicitly (better)

      //= require jquery
      //= require jquery/dist/jquery
//= require bootstrap/dist/js/bootstrap

  • Laravel

    composer create-project --prefer-dist laravel/laravel <laravel_project>

  • Useful script snippet

    • If you are always get disconnected, and you want to kill last failed connection of SSH

      netstat -palunt |grep -i est | awk '{print $7}'| cut -d'/' -f1 |xargs -I{} bash -c "ps aux |grep sshd |grep {}|grep -v grep" | head -n -1 | awk '{print $2}' |xargs -I{} kill {}

Ruby gem config

  • gem install without making document

    • Deprecated

      no-ri, no-rdoc

    • Config

      echo "gem: --no-document" > ~/.gemrc

Database configuration for production

  • Remove test database and setup root password

    After doing this, still need some tweak, try to manage database with https://www.adminer.org/

    $ mysql_secure_installation

    Just keep hitting <ENTER>, to USE ALL DEFAULT SETTING

  • After mysql_secure_installation

    • MariaDB 10.5+ auth method will just like MariaDB 10.3

  • Database tools - Adminer

Extra functions

  • RENDER_CP

    • Render template using eval (Might have escape issue, if template is complicated)

      # Method : eval "echo \"$variable\""

    • Sample

      • databag
      local var="Hello World"
      • template (${$CONFIG_FOLDER}/yourowntemplate_file)
      This is $var
      • function
      RENDER_CP ${$CONFIG_FOLDER}/yourowntemplate_file /SomeWhere/somewhere
      • result (/SomeWhere/somewhere)
      This is Hello World

  • RENDER_CP_SED

    • Render template using sed (BETTER method for rendering template)

      # Method : cat template | sed 's/\{\{var\}\}/$var/g'

    • Sample

      • databag
      local var="Hello World"
      • template (${$CONFIG_FOLDER}/yourowntemplate_file)
      This is {{var}}
      • function
      RENDER_CP_SED ${$CONFIG_FOLDER}/yourowntemplate_file /SomeWhere/somewhere
      • result (/SomeWhere/somewhere)
      This is Hello World

  • SAFE_DELETE

    • Check file names and path before rm any dangerous files, preventing from destoying whole server

      • check for the following dangerous key words

        .
..
*
/
.*
*.*
        "$(echo "$(find / -maxdepth 1 ;  readlink -m /* )" | sort -n | uniq)"

    • Sample

      # --- Should be failed ---
DELETE_FILE="/root/delete_me/.*"
# --- safe delete command usage ---
SAFE_DELETE "${DELETE_FILE}"

(Git) Stash details

  • Ref. https://git-scm.com/docs/git-stash

  • (Git) stash list

    $ git stash list
stash@{0}: WIP on redmine_4.0.7: a853fc0 Fix sort projects table by custom field (#32769).
stash@{1}: WIP on redmine_4.0.6: 22ebc68 tagged version 4.0.6
    • redmine_4.0.6 / redmine_4.0.7, these mean branch name
    • if you want to restore data, you'd better checkout the the related branch

  • Display all stash contents

    git stash list | cut -d':' -f1 | xargs -I{} bash -c "\
  echo; \
  echo ----------------------------------------------- {} -----------------------------------------------;\
  git stash show -p {}; echo\
"

(Git) Push and Pull

  • Push git commits to remote

    git push

  • Push git tags to remote

    git push --tags

  • Fetch git commits to local

    git fetch

  • Fetch git tags to local

    git fetch --tags

  • Fetch git commits to local and then MERGE to Working Directory

    git pull

Upgrading Redmine

Reference

Backup current redmine

  • Database
    • mysqldump -u {db_user} -p --lock-all-tables --skip-tz-utc -B redmine > redmine_$(date +"%Y%m%d")_skip-tz-utc.sql
  • Application & files
    • cp -a redmine redmine_bak

Customized files

  • plugins
    • /home/rubyuser/rails_sites/redmine/plugins/redmine_*
  • themes
    • /home/rubyuser/rails_sites/redmine/public/themes/{a1,circle,PurpleMine2}
  • session token
    • /home/rubyuser/rails_sites/redmine/config/initializers/secret_token.rb
  • uploaded files
    • /home/rubyuser/rails_sites/redmine/files/

(Method 1) Upgrading from a git checkout

  • Stop puma server

    • puma-systemd-mgr -p -i redmine

  • Go to the Redmine root directory and run the following command:

    cd redmine
git stash
    git checkout master
git fetch
git fetch --tags
git pull

    sometimes git pull will not fetch tags, instead, we need to fetch tags by git fetch --tags

    especially when tags name or tags <-> commit , has been changed

    git co 4.0.7 -b redmine_4.0.7
git stash pop
git status |grep 'both modified:' |awk '{print $3}' |xargs -I{} bash -c "echo --- git reset HEAD {} ---; git reset HEAD {}"

  • Fix conflicts

  • Perform the upgrade

    # gemset name using redmine version
echo "gemset_redmine_4.1.0" > .ruby-gemset

# switch to the new gemset
cd
cd -

# Update gem / bundler for this gemset
gem update --system
gem install bundler

# Install the required gems by running the following command
bundle update

# Update the database
bundle exec rake db:migrate RAILS_ENV=production
bundle exec rake redmine:plugins RAILS_ENV=production

# Clean up
bundle exec rake tmp:cache:clear RAILS_ENV=production

  • Start puma server

    • puma-systemd-mgr -s -i redmine

  • Go to "Admin -> Roles & permissions" to check/set permissions for the new features, if any.

  • Finally, clear browser's cached data (To avoid strange CSS error)

    • Chrome -> History -> Clear History -> Choose ONLY "Cached images and files"

(Method 2) Upgrading from a fresh installation

  • Stop puma server

    • puma-systemd-mgr -p -i redmine

  • Backup current redmine

  • Remove the following lines from script functions/F_02_PKG_06_ruby_09_redmine_create.sh (F_02_PKG_06_ruby_09_redmine_create_diff.png)

    if [[ -z "${redmine_db_pass}" ]]; then
  mysql -u root -e "CREATE DATABASE ${redmine_db_name} CHARACTER SET utf8mb4;"
else
  mysql -u root -p${redmine_db_pass} -e "CREATE DATABASE ${redmine_db_name} CHARACTER SET utf8mb4;"
fi
    su -l $current_user -c "cd ${redmine_web_root} && bundle _${this_redmine_bundler_version}_ exec rake generate_secret_token"
    if [[ -n "${redmine_default_lang}" ]]; then
  su -l $current_user -c "cd ${redmine_web_root} && bundle _${this_redmine_bundler_version}_ exec rake redmine:load_default_data RAILS_ENV=production REDMINE_LANG=${redmine_default_lang}"
fi

  • Perform the fresh installation

    • ./start -i F_02_PKG_06_ruby_09_redmine_create

  • Restore files from backup

    • redmine/config/initializers/secret_token.rb
    • redmine/files/

  • Start puma server

    • puma-systemd-mgr -s -i redmine

  • Go to "Admin -> Roles & permissions" to check/set permissions for the new features, if any.

  • Finally, clear browser's cached data (To avoid strange CSS error)

    • Chrome -> History -> Clear History -> Choose ONLY "Cached images and files"

Uninstall Redmine Plugins

Ref. https://www.redmine.org/projects/redmine/wiki/plugins#Uninstalling-a-plugin

  • Backup current redmine

    • Database
      • mysqldump -u {db_user} -p --lock-all-tables --skip-tz-utc -B redmine > redmine_$(date +"%Y%m%d")_skip-tz-utc.sql
    • Application & files
      • cp -a redmine redmine_bak

  • Steps of uninstalling {plugin_name}

    • Uninstall from database

      bundle exec rake redmine:plugins:migrate NAME={plugin_name} VERSION=0 RAILS_ENV=production

    • Remove your plugin from the plugins folder: #{RAILS_ROOT}/plugins.

    • Restart Redmine

Upgrading MariaDB

For some/view cases, we need to upgrade MariaDB without data lost. Here is my note about this.

Reference (mariadb.com)

How to Upgrade

  • Backup current database

    # mysqldump -u root -p --lock-all-tables --skip-tz-utc -A > all_`date +"%Y%m%d"`_skip-tz-utc.sql

  • Stop MariaDB

    # systemctl stop mariadb

  • Uninstall the old version of MariaDB

    apt purge -y mariadb-server libmariadb-dev libmariadb-dev-compat

  • Modify the repository configuration to newer version

  • Install the new version of MariaDB

    apt install -y mariadb-server libmariadb-dev libmariadb-dev-compat

  • Make any desired changes to configuration options in option files, such as my.cnf. This includes removing any options that are no longer supported.

    # cat /etc/mysql/mariadb.conf.d/50-server.cnf | grep -B1 '127.0.0'
[mysqld]
bind-address = 127.0.0.1

  • Start MariaDB

    # systemctl start mariadb

  • Run mysql_upgrade

    • mysql_upgrade does two things:
      • Ensures that the system tables in the#mysql database are fully compatible with the new version.
      • Does a very quick check of all tables and marks them as compatible with the new version of MariaDB .
    • mysql_upgrade -u root -p

      • After this command, there would be a file generated for letting you know this database has already been upgraded. (owner of the file is root)

        $ ls /var/lib/mysql | grep upgrade
-rw-r--r--  1 root  root    15 Sep  9 14:24 mysql_upgrade_info
        $ cat mysql_upgrade_info
10.5.5-MariaDB

  • Restart MariaDB - Done

    • It would be better to restart MariaDB, if it's allowed.

      # systemctl restart mariadb

Ubuntu notes

crontab environment

  • Default SHELL and PATH

    • /etc/crontab

      # make sure crontab use bash and PATH is correct
sed -i /SHELL/d /etc/crontab
sed  '1s/^/SHELL\=\/bin\/bash\n/' -i /etc/crontab

sed -re 's/^#PATH\=/PATH\=/g' -i /etc/crontab

  • Default command argv (most deference CentOS vs Ubuntu)

    • /etc/default/*

    • Sample

      # cat /etc/default/useradd | grep -vE '^#'
SHELL=/bin/sh

Network

  • Network for Ubuntu 22

    • By default - no more NetworkManager, use netplan + Systemd-networkd instead

    • https://netplan.io/

        • Netplan currently works with these supported renderers

        • Senario

          • NetworkManager
            • Use for Desktop, such as wifi
          • Systemd-networkd
            • Use for server, such as netplan use

        • Commands

          • Netplan uses a set of subcommands to drive its behavior:
            • netplan generate: Use /etc/netplan to generate the required configuration for the renderers.
            • netplan apply: Apply all configuration for the renderers, restarting them as necessary.
            • netplan try: Apply configuration and wait for user confirmation; will roll back if network is broken or no confirmation is given.

    • check current DNS setting

      • resolvectl

gnupg2 (gpg)

  • Public key types

    • Binary (--dearmor)

      • CAN BE imported by

        gpg --import redis-archive-keyring.gpg

      • Command

        curl -fsSL https://packages.redis.io/gpg | gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg

    • ASCII (--enarmor)

      • CAN BE imported by

        gpg --import redis.asc

      • Command

        gpg --enarmor < redis-archive-keyring.gpg > redis.asc

  • List public keys under ~/.gnupg

    • gpg -k

  • List private keys under ~/.gnupg

    • gpg -K

  • Display content of public from key files (ascII or binary)

    • gpg --show-keys {redis.gpg|redis.asc}
    • Originally
      • gpg import {plain ascii[redis.asc] | gpg bin file[redis.gpg]}
      • Then gpg -k

  • Sample

    • enarmor ascII (key.asc) transfor to bin file (key.gpg)

      curl -fsSL https://packages.redis.io/gpg | gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg

gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

  • How the command above does

    • import 2 public keys

      curl -sSL https://raw.githubusercontent.com/charlietag/github_share_folder/master/rvm_gpg_public_keys/mpapis.asc | gpg --import -
curl -sSL https://raw.githubusercontent.com/charlietag/github_share_folder/master/rvm_gpg_public_keys/pkuczynski.asc | gpg --import -

    • Add 2 public keys into ownertrust

      echo 409B6B1796C275462A1703113804BB82D39DC0E3:6: | gpg2 --import-ownertrust
echo 7D2BAF1CF37B13E2069D6956105BD0E739499BDB:6: | gpg2 --import-ownertrust

    • Ownertrust

  • Generate self gpg key pairs

  • Use for software (apt sign_by)

    • Ref. verify-pgp-signature-software-downloads-linux
    • Verify packages via public key procedure
      • wget VeraCrypt/VeraCrypt_PGP_public_key.asc
      • gpg --show-keys VeraCrypt_PGP_public_key.asc
      • compare keys listed above with website mentioned (use like md5sum)
      • gpg --import VeraCrypt_PGP_public_key.asc
      • gpg --verify veracrypt-1.24-Update7-Ubuntu-20.04-amd64.deb.sig veracrypt-1.24-Update7-Ubuntu-20.04-amd64.deb
        • good signature

  • Use for apt-key (keyring.gpg)

    • Ref. apt-key-deprecated

      • The gpg --dearmor part is important because the mechanism expects you to have the keys in binary format.

    • Old school (deprecated)

      • apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

    • Nowadays

      • Just put binary format gpg keyring under /etc/apt/trusted.gpg.d

        curl -sSL https://www.mongodb.org/static/pgp/server-6.0.asc | gpg --dearmor -o mongodb-server-keyring.gpg

    • Sample command

      root@u22:~# apt-key list
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
------------------------------------------------------
pub   rsa4096 2012-05-11 [SC]
      8439 38DF 228D 22F7 B374  2BC0 D94A A3F0 EFE2 1092
uid           [ unknown] Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg
------------------------------------------------------
pub   rsa4096 2018-09-17 [SC]
      F6EC B376 2474 EDA9 D21B  7022 8719 20D1 991B C93C
uid           [ unknown] Ubuntu Archive Automatic Signing Key (2018) <ftpmaster@ubuntu.com>

root@u22:~# cd /etc/apt/trusted.gpg.d
root@u22:/etc/apt/trusted.gpg.d# ll
total 16
drwxr-xr-x 2 root root 4096 Nov 28 23:33 ./
drwxr-xr-x 8 root root 4096 Nov 28 23:37 ../
-rw-r--r-- 1 root root 2794 Mar 26  2021 ubuntu-keyring-2012-cdimage.gpg
-rw-r--r-- 1 root root 1733 Mar 26  2021 ubuntu-keyring-2018-archive.gpg
root@u22:/etc/apt/trusted.gpg.d# curl -sSL https://www.mongodb.org/static/pgp/server-6.0.asc | gpg --dearmor -o mongodb-server-keyring.gpg
root@u22:/etc/apt/trusted.gpg.d# apt-key list
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
/etc/apt/trusted.gpg.d/mongodb-server-keyring.gpg
-------------------------------------------------
pub   rsa4096 2022-02-23 [SC] [expires: 2027-02-22]
      39BD 841E 4BE5 FB19 5A65  400E 6A26 B1AE 64C3 C388
uid           [ unknown] MongoDB 6.0 Release Signing Key <packaging@mongodb.com>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
------------------------------------------------------
pub   rsa4096 2012-05-11 [SC]
      8439 38DF 228D 22F7 B374  2BC0 D94A A3F0 EFE2 1092
uid           [ unknown] Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg
------------------------------------------------------
pub   rsa4096 2018-09-17 [SC]
      F6EC B376 2474 EDA9 D21B  7022 8719 20D1 991B C93C
uid           [ unknown] Ubuntu Archive Automatic Signing Key (2018) <ftpmaster@ubuntu.com>

APT - Interactive settings

APT command

  • Remove package

    • Remove packages only

      apt remove -y {package}

    • Remove packages + delete config files (Mostly used - Useful for some cases)

      apt remove -y --purge {package}
      apt purge -y {package}

    • Remove packages + remove related packages without warning message

      apt remove --autoremove -y {package}
      apt autoremove -y {package}

    • Combined usage (Mostly used)

      apt autoremove -y --purge {package}
      apt remove --purge --autoremove -y {package}

    • Remove all unused packages (Useful)

      apt autoremove -y --purge

DPKG usage

  • Find specific package

    dpkg -l | grep {package}

  • Fine specific file belongs to what package

    dpkg -S {filename}

  • List all files belongs to specific package

    dpkg -L {package}

  • Find config files of specific package

    dpkg -L {package} |grep "^\/etc"

  • Show info of specific package

    apt show {package}

    (apt info {package} is an alias of show)

Package name convention

  • {pkg}
  • lib{pkg}
  • lib{pkg}-dev
    • Usually this contains lib{pkg}
  • Sample
    • (CentOS) openssl openssl-libs openssl-devel
    • (Ubuntu) openssl libssl-dev

3rd party repo (BE CAREFUL)

Be CAREFUL using this

packages here are not verified by Ubuntu

  • Platform (Launchpad - PPA) hosted by Ubuntu, but packages are not maintained by Ubuntu

  • You can contribute your own packages on PPA platform

  • Install package software-properties-common (add-apt-repository) first

  • Install PPA

    • Install php 8.2 in Ubuntu 22.04

      • ppa:ondrej/php is like remi in RedHat world

      • Ref. install-php-8-2-ubuntu-22-04

        add-apt-repository ppa:ondrej/php
apt update
apt install php8.2 php8.2-fpm php8.2-mysql

  • Remove PPA

    • Uninstall package entirely

      apt remove --purge --autoremove -y php8.2 php8.2-fpm php8.2-mysql

    • Remove PPA

      add-apt-repository --remove ppa:ondrej/php

snapd

snapd.png

  • Useful sandbox application - out-of-the-box application (same as RedHat - flatpak)

    • Pros - Easy to use
    • Cons - needs more disk spaces

  • Almost a lot of Desktop UI features are based on Snapd

  • Cannot be removed, especial rails package libvips is based on Snapd

  • Remove snapd totally

  • Note

    • If apt remove snapd (NOT RECOMMEND) is triggered, the systemd is changed, the follow command should be executed

      systemctl daemon-reload

ssh client known hosts hash

  • /etc/ssh/ssh_config
    • By default - debian based ssh client , HashKnownHosts=yes, for some cases you might want to set it to no
    • Otherwise, you need to find public key of hosts by command as below

      • Check existence

        ssh-keygen -F dev.server.name

      • Remove existence

        ssh-keygen -R dev.server.name

Editor

User manipulation

  • Add user

    • (PREFERED) Have to specify shell and create home manually

      useradd -m -s /bin/bash {user}

    • (Alternative , NOT prefered) use perl script - adduser

      • Interactive (Ask a lot of questions)

        adduser {user}

      • Don't ask, just create user like useradd -m -s {user} does

        adduser -q --disabled-login --gecos "" <user>

  • Delete user

    • (PREFERED) Delete home directory

      userdel -r {user}

    • (Alternative , NOT prefered) use perl script - deluser

      deluser --remove-home {user}

  • Delete user (DANGER)

    • DANGER DO NOT USE perl script
      • find / belongs to username and delete it
        • deluser --remove-all-files {user}

  • Lock user

    usermod -s "/bin/false"

  • List users (useful)

    lslogin

Python notes

pyenv

Similar with rvm

  • List all versions

    • rvm list known
    • pyenv install -l | grep -E "[[:space:]]+[[:digit:]\.]+$"

  • Install specific version

    • rvm install 3.2
    • pyenv install 3.11

  • Switch version for current session

    • rvm use 3.2
    • pyenv shell 3.11

  • Change DEFAULT version

    • rvm use 3.2 --default
    • pyenv global 3.11 (config locates $(pyenv root)/version)

  • Change version for current folder

    • Just edit version in file folder/.ruby-version
    • In pyenv, edit version in file folder/.python-version (can just use command pyenv local 3.11)

poetry

  • Install poetry
    • (Python > 3.7) curl -sSL https://install.python-poetry.org | python3 -
    • If you install poetry through pyenv python (ie. 3.11), DO NOT uninstall the version(ie. 3.11), otherwise poetry will not be able to use (cannot find python3.11.so)...

Similar with bundler

  • Execute command within venv

    • bundle exec rails
    • poetry run django-admin startproject basic_django .
      • Sample template(Django) link

  • Excute command within venv (easier way), after making sure poetry env is created (poetry env list)

    • poetry shell
    • Then all python command is within venv

  • New poetry Pypi package (not for create project --- use poetry init -n instead for project)

    • similliar with bundle gem ruby-gem-demo
    • poetry new pip-package-demo

  • New poetry project

    • mkdir project-demo
    • cd project-demo
    • pyenv local 3.11
    • poetry init -n (do not use -q, quiet, otherwise it will include current project as package while poetry install)
    • cat pyproject.toml |grep python | grep '3.11'
    • poetry env use python
    • poetry install
    • Reference (poetry-new_project)

  • Make sure project is using correct python version (ex. ^3.11``) for both pyenv (.python-version)andpoetry (pyproject.toml)`

    cd poetry-demo
cat pyproject.toml |grep python | grep '3.11'
pyenv local 3.11

  • Create venv (ex. using Python 3.11) using poetry

    cd poetry-demo
poetry env use python

  • Make sure venv is created

    • poetry env list

  • Install packages (poetry.lock generated)

    • poetry install

  • Add packages into project (poetry.lock updated)

    • bundle add rails
    • poetry add django

pip

Conclusion for below: Just use poetry to manage python packages

  • If pip package(ie. panda) is not installed by user root

    • (general user) pip will store the changes under user home folder, will not effect global version

  • If pip package(ie. panda) IS installed by user root

    • pip will detect if package is already installed
      • if no, install local (pip install panda--user)
      • if yes, try to install under /usr/local/lib/python3.10/dist-packages, and if no writeable priv, it will install with --user by default

  • show detail info

    $ pip show panda
Name: panda
Version: 0.3.1
Summary: A Python implementation of the Panda REST interface
Home-page: http://www.pandastream.com
Author: pandastream.com
Author-email: support@pandastream.com
License: MIT
Location: /usr/local/lib/python3.10/dist-packages
Requires: requests, setuptools
Required-by:

Upgrade pyenv and poetry

  • pyenv
    • cd ~/.pyenv ; git pull
  • poetry
    • poetry self update

CHANGELOG

About

Ubuntu Server Preparation - Packages Pre Installation (Ubuntu server)

Topics

ruby rails shell bash nginx ruby-gem laravel ubuntu mariadb php-fpm puma linux-server

Resources

Readme

License

MIT license
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

65 tags

Packages

No packages published

Languages