New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PREP_SYSTEM_FOR_DIETPI.sh | Automate #1285

Closed
Fourdee opened this Issue Dec 10, 2017 · 136 comments

Comments

@Fourdee
Owner

Fourdee commented Dec 10, 2017

Status: Beta

What is this?

  • This script will install and convert any 'bloated' Debian/Raspbian installation, into a lightweight DietPi system.

What this script does NOT do:

  • Support converting existing installed software (eg: nextcloud, Plex Media Server) over to the DietPi system.
    All existing software (APT) and user data will be deleted.

Step 1: Ensure a Debian/Raspbian OS is running on the system

  • For best results, we recommend a fresh/clean minimal Debian/Raspbian installation
  • Native PC users: please install Debian stable before hand: https://www.debian.org/distrib/netinst
  • Desktop images should work, however, the minimal the image, the quicker the installation, as less packages will need to be removed.
  • NB: We do not support Ubuntu, or have any plans to do so.

Step 2: Pre-req packages

  • Critical packages
    These should already exist on most systems, however pure minimal images may require the following installations)
    apt-get update; apt-get install -y systemd-sysv ca-certificates sudo wget

Step 3: Run DietPi installer

  • Must be run from SSH or local term
    eg: Outside of desktop environment
  • In case of SSH, to prevent connection loss, an Ethernet connection is required.

Ensure you have elevated privileges (eg: login as root, or use sudo su).

  • Due to sudo not exporting vars across script var loading.

Copy and paste all into term.

wget https://raw.githubusercontent.com/Fourdee/DietPi/master/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/Fourdee/DietPi/beta/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/Fourdee/DietPi/dev/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
chmod +x PREP_SYSTEM_FOR_DIETPI.sh
./PREP_SYSTEM_FOR_DIETPI.sh

Follow the onscreen prompts.

@Fourdee Fourdee added this to the v160 milestone Dec 10, 2017

@Fourdee Fourdee self-assigned this Dec 10, 2017

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Dec 10, 2017

Just in theory: If we really get this automated secure for at least some devices, this means we do not necessarily need to create images for those. We could just provide the preparation script to upgrade any Debian system into DietPi. By use of apt-mark, as suggested here: #1266, we would be sure that possibly disturbing software is removed and provide our own config files/adjustments for our dependencies. Kernel/bootleader apt packages are generally excluded from autoremove on every device I checked.
As running systems are, yeah running, we could extract boot critical information (especially fstab/boot.ini/network details) from the running system in case and create our configs around those.

Hehe just a little optimistic view into the feature 😆.

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 10, 2017

@MichaIng

We could just provide the preparation script to upgrade any Debian system into DietPi.
As running systems are, yeah running, we could extract boot critical information (especially fstab/boot.ini/network details) from the running system in case and create our configs around those.
Hehe just a little optimistic view into the feature

I like where this is going 👍

Fourdee added a commit that referenced this issue Dec 11, 2017

v160
+ Initial start of: #1285

NB: failed replace on $OPTION casing WHIP_OPTION

Fourdee added a commit that referenced this issue Dec 11, 2017

v160
+ Dry run of AGP. continue:
#1285
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 12, 2017

@MichaIng

I've not looked into manual holding packages yet. I'am simply converting the old script into automated. Then we can work on it.

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Dec 12, 2017

@Fourdee
Great work so far, like the idea of setting options by whiptail, first steps into end user direction 😄.

Just consider: #1266

If you generally like the idea, I will recreate the PR for the automated script. Other small adjustments based on #1219 (comment) and following as well.

Just tell, after you implemented you ideas, then I will create PR.

Fourdee added a commit that referenced this issue Dec 13, 2017

v160
+ DietPi-Software | Plex Media Server: Resolved uninstall to include
/var/lib/plexmediaserver in removal (which is not completed via apt
purge).

+ Sparky SBC: Matrix Audio X-SPDIF 2, native DSD is now added to kernel,
many thanks @sudeep: sparkysbc/Linux#3

+ #1285

Fourdee added a commit that referenced this issue Dec 13, 2017

v160
+ Package removal test: #1285
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 13, 2017

Ok, so the new package removal system:

Scraped, in favor of @MichaIng APT removal method.

  • We provide a list of essential minimal packages that must be installed for DietPi
  • The script then finds all the dependencies for each package we need and stores it into an array
  • Package removal list, is generated by comparing the currently installed, to the array we generated. If any packages exist that we dont need, DietPi will remove them.

This allows us to exclude the old system of manually removing "known" packages we dont want. Instead, having everything generated automatically, based on the current installed system.

/DietPi/dietpi/PREP_SYSTEM_FOR_DIETPI.sh                         

 [Ok] Generating list of minimal packages required for DietPi installation


 [Ok] Obtaining list of currently installed packages


 [Info] (0): Passed


 [Ok] Generating a list of deps, required for the DietPi packages
This may take some time, please wait...

Checking deps: acl
 - Adding deps: libacl1
 - Adding deps: libattr1
 - Adding deps: libc6
Checking deps: adduser
 - Adding deps: passwd
 - Adding deps: debconf
Checking deps: apt
 - Adding deps: adduser
 - Adding deps: gpgv
 - Adding deps: debian-archive-keyring
 - Adding deps: init-system-helpers
 - Adding deps: libapt-pkg5.0
 - Adding deps: libgcc1
 - Adding deps: libstdc++6
Checking deps: apt-transport-https
 - Adding deps: libcurl3-gnutls
Checking deps: apt-utils
 - Adding deps: apt
 - Adding deps: libapt-inst2.0
 - Adding deps: libdb5.3
Checking deps: base-files
Checking deps: base-passwd
 - Adding deps: libdebconfclient0
Checking deps: bash
 - Adding deps: dash
 - Adding deps: libtinfo5
 - Adding deps: base-files
 - Adding deps: debianutils
Checking deps: bash-completion
 - Adding deps: dpkg
 - Adding deps: bash
Checking deps: bc
 - Adding deps: libreadline7
Checking deps: bsdmainutils
 - Adding deps: libbsd0
 - Adding deps: libncurses5
 - Adding deps: bsdutils
Checking deps: bsdutils
 - Adding deps: libsystemd0
Checking deps: bzip2
 - Adding deps: libbz2-1.0
Checking deps: ca-certificates
 - Adding deps: openssl
Checking deps: console-common
 - Adding deps: console-data
 - Adding deps: kbd
 - Adding deps: lsb-base
Checking deps: console-data
Checking deps: console-setup
 - Adding deps: console-setup-linux
 - Adding deps: console-setup-freebsd
 - Adding deps: xkb-data
 - Adding deps: keyboard-configuration
Checking deps: console-setup-linux
 - Adding deps: initscripts
Checking deps: coreutils
 - Adding deps: libselinux1
Checking deps: cpio
Checking deps: crda
 - Adding deps: libnl-3-200
 - Adding deps: libnl-genl-3-200
 - Adding deps: libssl1.1
 - Adding deps: wireless-regdb
 - Adding deps: iw
Checking deps: cron
 - Adding deps: libpam0g
 - Adding deps: libpam-runtime
Checking deps: curl
 - Adding deps: libcurl3
 - Adding deps: zlib1g
Checking deps: dash
Checking deps: dbus
 - Adding deps: libapparmor1
 - Adding deps: libaudit1
 - Adding deps: libcap-ng0
 - Adding deps: libdbus-1-3
 - Adding deps: libexpat1
Checking deps: debconf
 - Adding deps: perl-base
Checking deps: debian-archive-keyring
Checking deps: debianutils
 - Adding deps: sensible-utils
Checking deps: diffutils
Checking deps: dmidecode
Checking deps: dmsetup
 - Adding deps: libdevmapper1.02.1
Checking deps: dosfstools
 - Adding deps: libudev1
Checking deps: dphys-swapfile
 - Adding deps: dc
Checking deps: dpkg
 - Adding deps: liblzma5
 - Adding deps: tar
Checking deps: e2fslibs
Checking deps: e2fsprogs
 - Adding deps: e2fslibs
 - Adding deps: libblkid1
 - Adding deps: libcomerr2
 - Adding deps: libss2
 - Adding deps: libuuid1
 - Adding deps: util-linux
Checking deps: ethtool
Checking deps: fake-hwclock
Checking deps: fbset
 - Adding deps: udev
 - Adding deps: makedev
Checking deps: findutils
Checking deps: firmware-atheros
Checking deps: firmware-brcm80211
Checking deps: firmware-ralink
Checking deps: firmware-realtek
Checking deps: fuse
 - Adding deps: libfuse2
 - Adding deps: mount
 - Adding deps: sed
Checking deps: gnupg
 - Adding deps: gnupg-agent
 - Adding deps: libassuan0
 - Adding deps: libgcrypt20
 - Adding deps: libgpg-error0
 - Adding deps: libksba8
 - Adding deps: libsqlite3-0
Checking deps: gpgv
Checking deps: grep
 - Adding deps: libpcre3
 - Adding deps: install-info
Checking deps: groff-base
Checking deps: gzip
Checking deps: hdparm
Checking deps: hfsplus
 - Adding deps: libhfsp0
Checking deps: hostname
Checking deps: htop
 - Adding deps: libncursesw5
Checking deps: ifupdown
 - Adding deps: iproute2
Checking deps: init
 - Adding deps: systemd-sysv
 - Adding deps: sysvinit-core
Checking deps: init-system-helpers
Checking deps: initramfs-tools
 - Adding deps: initramfs-tools-core
 - Adding deps: linux-base
Checking deps: initscripts
 - Adding deps: sysvinit-utils
 - Adding deps: sysv-rc
 - Adding deps: file-rc
 - Adding deps: openrc
 - Adding deps: coreutils
Checking deps: insserv
Checking deps: iproute2
 - Adding deps: libelf1
 - Adding deps: libmnl0
Checking deps: iputils-ping
 - Adding deps: libcap2
 - Adding deps: libidn11
 - Adding deps: libnettle6
Checking deps: isc-dhcp-client
 - Adding deps: libdns-export162
 - Adding deps: libisc-export160
Checking deps: isc-dhcp-common
Checking deps: iw
Checking deps: kbd
Checking deps: keyboard-configuration
 - Adding deps: liblocale-gettext-perl
Checking deps: klibc-utils
 - Adding deps: libklibc
Checking deps: kmod
 - Adding deps: libkmod2
Checking deps: less
Checking deps: locales
 - Adding deps: libc-bin
 - Adding deps: libc-l10n
Checking deps: login
 - Adding deps: libpam-modules
Checking deps: lsb-base
Checking deps: mawk
Checking deps: mount
 - Adding deps: libmount1
 - Adding deps: libsmartcols1
Checking deps: multiarch-support
Checking deps: nano
Checking deps: ncurses-base
Checking deps: ncurses-bin
Checking deps: net-tools
Checking deps: ntfs-3g
 - Adding deps: fuse
 - Adding deps: libgnutls30
 - Adding deps: libntfs-3g871
Checking deps: ntp
 - Adding deps: netbase
 - Adding deps: libedit2
 - Adding deps: libopts25
Checking deps: p7zip-full
 - Adding deps: p7zip
Checking deps: parted
 - Adding deps: libparted2
Checking deps: passwd
 - Adding deps: libsemanage1
Checking deps: perl-base
Checking deps: procps
 - Adding deps: libprocps6
Checking deps: psmisc
Checking deps: readline-common
Checking deps: resolvconf
 - Adding deps: ifupdown
Checking deps: sed
Checking deps: sensible-utils
Checking deps: startpar
Checking deps: sudo
Checking deps: systemd
 - Adding deps: libcryptsetup4
 - Adding deps: libip4tc0
 - Adding deps: liblz4-1
 - Adding deps: libseccomp2
 - Adding deps: procps
Checking deps: systemd-sysv
 - Adding deps: systemd
Checking deps: sysvinit-utils
Checking deps: tar
Checking deps: tzdata
Checking deps: udev
Checking deps: unzip
Checking deps: usbutils
 - Adding deps: libusb-1.0-0
Checking deps: util-linux
 - Adding deps: libfdisk1
Checking deps: wget
 - Adding deps: libpsl5
Checking deps: whiptail
 - Adding deps: libnewt0.52
 - Adding deps: libpopt0
 - Adding deps: libslang2
Checking deps: wireless-regdb
Checking deps: wireless-tools
 - Adding deps: libiw30
Checking deps: wpasupplicant
 - Adding deps: libpcsclite1
 - Adding deps: libssl1.0.2
Checking deps: wput
 - Adding deps: libgnutls-openssl27
Checking deps: zip
Checking deps: intel-microcode
 - Adding deps: iucode-tool
Checking deps: amd64-microcode
Checking deps: firmware-linux-nonfree
 - Adding deps: firmware-misc-nonfree
 - Adding deps: firmware-amd-graphics

 [Ok] Generating a list of packages, not required by DietPi, to be removed from system.
This may take some time, please wait...


 [Info] The following packages will be removed
adwaita-icon-theme alsa-utils bind9-host binutils build-essential checkinstall cifs-utils cmake cmake-data cpp cpp-6 cryptsetup cryptsetup-bin dh-python dhcpcd5 dialog discover discover-data distro-info-data dnsmasq dnsmasq-base dnsutils dropbear dropbear-bin dropbear-initramfs dropbear-run efibootmgr fakeroot file fontconfig fontconfig-config fonts-dejavu-core g++ g++-6 galculator gcc gettext-base git git-man glib-networking:amd64 glib-networking-common glib-networking-services gparted gpicview grub2-common gsettings-desktop-schemas gtk-update-icon-cache hicolor-icon-theme installation-report iptables irqbalance laptop-detect leafpad light-locker lightdm lightdm-gtk-greeter lighttpd linux-libc-dev:amd64 lsb-release lsof lxappearance lxappearance-obconf lxde lxde-common lxde-core lxde-icon-theme lxhotkey-core lxhotkey-gtk lxinput lxlock lxmenu-data lxpanel lxpanel-data lxpolkit lxrandr lxsession lxsession-data lxsession-edit lxsession-logout lxterminal make manpages manpages-dev mime-support mysql-common netcat netcat-traditional openbox openbox-lxde-session os-prober patch pciutils pcmanfm perl perl-modules-5.24 php-apcu php-apcu-bc php-cgi php-common php-curl php-fpm php-gd php-mbstring php-mcrypt php-xml php-zip php7.0-cgi php7.0-cli php7.0-common php7.0-curl php7.0-fpm php7.0-gd php7.0-json php7.0-mbstring php7.0-mcrypt php7.0-opcache php7.0-readline php7.0-xml php7.0-zip pinentry-curses policykit-1 python python-minimal python-talloc python2.7 python2.7-minimal python3 python3-minimal python3.5 python3.5-minimal samba-common samba-libs:amd64 sgml-base shared-mime-info smbclient spawn-fcgi sysbench tcl tcl8.6 tigervnc-common tigervnc-standalone-server tk tk8.6 ucf upower vnc4server wamerican x11-common x11-utils x11-xkb-utils x11-xserver-utils x11vnc x11vnc-data xarchiver xauth xbitmaps xcompmgr xdg-user-dirs xfonts-base xfonts-encodings xfonts-utils xinit xml-core xserver-common xserver-xorg xserver-xorg-core xserver-xorg-input-all xserver-xorg-input-libinput xserver-xorg-video-all xserver-xorg-video-amdgpu xserver-xorg-video-ati xserver-xorg-video-fbdev xserver-xorg-video-nouveau xserver-xorg-video-radeon xserver-xorg-video-vesa xserver-xorg-video-vmware xterm xz-utils zerofree


 [Info] The following packages will be installed
acl adduser apt apt-transport-https apt-utils base-files base-passwd bash bash-completion bc bsdmainutils bsdutils bzip2 ca-certificates console-common console-data console-setup console-setup-linux coreutils cpio crda cron curl dash dbus debconf debian-archive-keyring debianutils diffutils dmidecode dmsetup dosfstools dphys-swapfile dpkg e2fslibs e2fsprogs ethtool fake-hwclock fbset findutils firmware-atheros firmware-brcm80211 firmware-ralink firmware-realtek fuse gnupg gpgv grep groff-base gzip hdparm hfsplus hostname htop ifupdown init init-system-helpers initramfs-tools initscripts insserv iproute2 iputils-ping isc-dhcp-client isc-dhcp-common iw kbd keyboard-configuration klibc-utils kmod less locales login lsb-base mawk mount multiarch-support nano ncurses-base ncurses-bin net-tools ntfs-3g ntp p7zip-full parted passwd perl-base procps psmisc readline-common resolvconf sed sensible-utils startpar sudo systemd systemd-sysv sysvinit-utils tar tzdata udev unzip usbutils util-linux wget whiptail wireless-regdb wireless-tools wpasupplicant wput zip intel-microcode amd64-microcode firmware-linux-nonfree

Fourdee added a commit that referenced this issue Dec 14, 2017

v160
+ Switch to @MichaIng APT removal method, much more efficent (scrap
array based removal): #1285

+ Buster DISTRO
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 14, 2017

Buster notes:

  • I've disabled option for now.

RPI:

W: The repository 'https://archive.raspberrypi.org/debian buster Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://archive.raspberrypi.org/debian/dists/buster/main/binary-armhf/Packages  404  Not Found

C2:

W: The repository 'http://ftp.debian.org/debian buster-backports Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch http://ftp.debian.org/debian/dists/buster-backports/main/binary-arm64/Packages  404  Not Found [IP: 130.89.148.12 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 14, 2017

Note to self:

ToDo:

  • Automate TZ/Locale??? Had issues with this in the past. Need to research it again.
  • Add ARMbian linux kernel to install list, or, add a general "do not remove" linux-* to script?
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 14, 2017

Ready for early internal testing to fix bugs and improve.

Not stable enough for public users, yet.

Tested:

  • 🈯️ Odroid C2 Stretch
@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Dec 14, 2017

@Fourdee

Buster notes:

I've disabled option for now.

  • Yes, the raspberrypi.org repo was always behind: https://archive.raspberrypi.org/debian/dists/ I remember stretch distro to appear there just around it became release for raspbian, long after it became release for debian and long after I switched already to stretch with my RPi. Jessie distro worked fine there, as it mainly provides just firmware/kernel/bootloader. I will upgrade my RPi to buster after doing some tests on my buster VM to then tell more reliably if buster + raspberrypi.org stretch repos work together. We need to further implement buster for this into DietPi, but let's first get the automated script ready for public testing.
  • About buster debian repo: Of course no buster-backport exists, as it is already the testing repo with up-to-date packages 😆: http://ftp.debian.org/debian/dists/ Okay sid also exists, but so far this looks just like a total experimental developer playground, with also many old (pre-)jessie packages (like php5...) inside. So for this, we can just comment out buster-backports and choose the main repo always inside our software scripts.
  • About debian repo generally: Found this: https://deb.debian.org/
    It is already default on new debian installations and seems to be kind of mirror director. Works well, even is reachable via https, as the link shows. Only on Jessie I got the following errors on apt update, if https was used. With http everything works fine:
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_main_binary-amd64_Packages.diff_Index is not what the server reported 9376 345
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_non-free_binary-amd64_Packages.diff_Index is not what the server reported 736 349
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_main_i18n_Translation-en.diff_Index is not what the server reported 3688 343
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_non-free_i18n_Translation-en.diff_Index is not what the server reported 736 347
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_main_binary-amd64_Packages.diff_Index is not what the server reported 27796 347
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_contrib_binary-amd64_Packages.diff_Index is not what the server reported 24964 15799
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_non-free_binary-amd64_Packages.diff_Index is not what the server reported 22858 351
W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_main_i18n_Translation-en.diff_Index is not what the server reported 27796 345
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 15, 2017

@MichaIng

Legend 👍

In regards to mirror:

root@DietPi:~# cat /etc/apt/sources.list
deb http://deb.debian.org/debian stretch main contrib non-free
deb http://deb.debian.org/security stretch main contrib non-free
deb http://ftp.debian.org/debian/ stretch-backports main contrib non-free

1st run

Ign:28 http://deb.debian.org/security stretch/non-free Translation-en
Err:14 http://deb.debian.org/security stretch/main amd64 Packages
  404  Not Found [IP: 151.101.0.204 80]
Ign:15 http://deb.debian.org/security stretch/main all Packages

2nd run

Ign:20 http://deb.debian.org/security stretch/non-free Translation-en_GB
Err:6 http://deb.debian.org/security stretch/main i386 Packages
  404  Not Found [IP: 151.101.0.204 80]
Ign:7 http://deb.debian.org/security stretch/main amd64 Packages

This is why I dislike using mirror services. They are unreliable.

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 15, 2017

Moved:

@Fourdee
About the dependency list:

  • I just wrote a little script that shows installed dependants of all installed packages. With this we can further reduce our dependency list to the final packages that are really directly used by dietpi / usual system operations. Here your dependency list where I added 🈯️ > dependant, if one within our anyway installed packages exist:
    'acl' 🈺 > systemd on Jessie, but not on Stretch+Buster, so we really need this, getfacl or setfacl
    'adduser' 🈯️ > systemd+apt+cron+udev+several other packages, depending on distro
    'apt' 🈯️ > apt-utils+apt-transport-https
    'apt-transport-https'
    'apt-utils'
    'base-files' 🈯️ > bash
    'base-passwd' 🈯️ essential package will be never autoremoved
    'bash' 🈯️ > bash-completion + essential
    'bash-completion'
    'bc'
    'bsdmainutils' 🈺 Do we need this? Was not in the list before!
    'bsdutils' 🈯️ essential
    'bzip2'
    'ca-certificates'
    'console-common' 🈺 console-setup supersedes or depends on other console related packages
    'console-data' 🈺 console-setup supersedes or depends on other console related packages
    'console-setup'
    'console-setup-linux' 🈺 console-setup supersedes or depends on other console related packages
    'coreutils' 🈯️ essential
    'cpio' 🈯️ (> initramfs-tools-core) > initramfs-tools
    'crda'
    'cron'
    'curl'
    'dash' 🈯️ > bash > bash-completion
    'dbus'
    'debconf' 🈯️ > debconf-utils
    'debian-archive-keyring' 🈯️ > apt and wrong on Raspbian, where raspbian-archive-keyring takes place!
    'debianutils' 🈯️ > cron+bash+dash+some others, depending on distro
    'diffutils' 🈯️ essential
    'dmidecode' 🈺 Do we need this? Was not in the list before!
    'dmsetup' 🈯️ > > > parted
    'dosfstools' 🈺 Could be left out on VMs, as we do not have fat boot partition there.
    'dphys-swapfile'
    'dpkg' 🈯️ > cron+bash-completion+others+anyway essential
    'e2fslibs' 🈯️ > e2fsprogs
    'e2fsprogs' 🈯️ essential
    'ethtool'
    'fake-hwclock'
    'fbset'
    'findutils' 🈯️ essential
    'firmware-atheros' 🈺 was x86 only before?
    'firmware-brcm80211' 🈺 was x86 only before?
    'firmware-ralink' 🈺 was x86 only before?
    'firmware-realtek' 🈺 was x86 only before?
    'firmware-misc-nonfree' 🈺 was x86 only before?
    'fuse' 🈺 Didn't see this before, do we need it? https://packages.debian.org/de/stretch/fuse
    'gnupg' 🈺 We used this before, but gpgv seems to be a lightweight version of it.
    'gpgv' 🈺 Superseded by gnupg, but more lightweight, maybe we need to switch?
    'grep' 🈯️ essential
    'groff-base' 🈺 Do we need this? Was not in the list before!
    'gzip' 🈯️ essential
    'hdparm'
    'hfsplus'
    'hostname' 🈯️ essential
    'htop'
    'ifupdown' 🈯️ > resolvconf
    'init' 🈯️ essential
    'init-system-helpers' 🈯️ > init and essential
    'initramfs-tools'
    'initscripts' 🈺 Was dependency of systemd on Jessie, but since Stretch not anymore. I guess systemd now supersedes that package.
    'insserv' 🈺 same than initscripts, don't have any of them on my systems (besides Jessie VM) and absolutely no issues.
    'iproute2' 🈯️ > ifupdown > resolvconf
    'iputils-ping' 🈺 Helpful indeed sometimes, but not necessary? Also superseded by curl, wget etc?
    'isc-dhcp-client'
    'isc-dhcp-common' 🈯️ This is just man pages!
    'iw'
    'kbd' 🈺 console-setup supersedes or depends on other console related packages
    'keyboard-configuration' 🈺 console-setup supersedes or depends on other console related packages
    'klibc-utils' 🈯️ initramfs-tools-core > initramfs-tools
    'kmod' 🈯️ (> initramfs-tools-core) > initramfs-tools
    'less' 🈺 Helpful indeed sometimes, but not necessary?
    'locales'
    'login' 🈯️ essential
    'lsb-base' 🈯️ > cron+udev+swapfile+resolvconf+others
    'mawk' 🈯️ > dash > bash
    'mount' 🈯️ > systemd
    'multiarch-support' 🈯️ > hundrets of libs, also e2fslibs > e2fsprogs, which is essential
    'nano' not necessary, but yeah I use this several times a day, everybody needs a simple editor!
    'ncurses-base' 🈯️ essential
    'ncurses-bin' 🈯️ essential
    'net-tools'
    'ntfs-3g'
    'ntp'
    'p7zip-full'
    'parted'
    'passwd' 🈯️ > adduser
    'perl-base' 🈯️ > init-system-helpers+debconf > debconf-utils
    'procps' 🈯️ > udev+systemd
    'psmisc'
    'readline-common' 🈯️ > parted+gnu(pg)
    'resolvconf'
    'rfkill' #Used by some onboard WiFi chipsets
    'rsync' 🈯️ Not necessary here, as it will be installed on demand by dietpi-backup/sync
    'sed' 🈯️ essential
    'sensible-utils' 🈺 Dependency of debianutils until buster. Just wondered that it is not autoremoved on my buster VM (marked as auto), but no dependency anymore and not essential. Just purged it on my Buster VM without any warnings or additional removed packages. Let's see what happens 😄.
    'startpar' 🈺 Was dependency of sysv-rc > systemd on Jessie, but since Stretch, systemd does not depend on sysv-rc anymore. Seems to solve the tasks differently now (same as insserv)?
    'sudo'
    'systemd' 🈯️ > systemd-sysv > init
    'systemd-sysv' 🈯️ > init
    'sysvinit-utils' 🈯️ essential
    'tar' 🈯️ essential
    'tzdata'
    'udev' 🈯️ > initramfs-tools-core > initramfs-tools
    'unzip'
    'usbutils'
    'util-linux' 🈯️ > udev+systemd+others+essential
    'wget'
    'whiptail'
    'wireless-regdb' 🈯️ > crda
    'wireless-tools'
    'wpasupplicant'
    'wput'
    'zip'

By removing the green ones from the list, there should be no additional packages removed. By removing also some of the orange ones, you get the list I used in my PR: https://github.com/Fourdee/DietPi/pull/1266/files#diff-e9e0c6c64b4937739e13ffac58f4a888R74

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Dec 15, 2017

@Fourdee

Err:14 http://deb.debian.org/security stretch/main amd64 Packages
404 Not Found [IP: 151.101.0.204 80]

Hehe this is just the wrong URL, use: deb https://deb.debian.org/debian-security/ <distro>/updates main contrib non-free


Going on through the script:


But yeah everything not much more than cosmetic. I can do PR if you want. Otherwise REALLY great work!! 👍 👍 👍 Also found the new fstab creation. Looks really professional! I will test the script tomorrow on my VMs.


If we are already touching this:

  • Perhaps we could also allow choosing whether to install wifi related packages, like for bt. We could actually move them (Wifi and Bt) into an own software stack to allow removing and installing them.
  • Would love to see nfs/samba/ftp log files and mount folders created on demand.

But both definitely no high priority 😉

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Dec 15, 2017

@MichaIng

Epic list #1285 (comment), many thanks for that 👍


In regards to relying on deps to pull in packages we manually listed (eg: sed)
In general, there is no harm is us manually listing additional packages that DietPi uses in its scripts, instead of relying on other packages to pull it in.

Benefits of listing (example sed):

  • We can see what packages DietPi fully requires/uses to function
  • By not relying on deps, if Debian were to change sed essential deps overnight, no risk of breaking script.

Firmware:

Covers us for all WiFi chipsets etc across all devices. We really should have it in for all images.

firmware-misc-nonfree can be x86_64 only, mostly intel/nvidia stuff https://packages.debian.org/stretch/firmware-misc-nonfree


Fuse

Required for ntfs-3g https://packages.debian.org/de/stretch/ntfs-3g:

root@DietPi:~# apt-get purge fuse
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  fuse* ntfs-3g*
0 upgraded, 0 newly installed, 2 to remove and 1 not upgraded.
After this operation, 1,241 kB disk space will be freed.
Do you want to continue? [Y/n]

Fourdee added a commit that referenced this issue Dec 15, 2017

v160
Tweaks based on @MichaIng feedback:
#1285 (comment)
@lateboomer

This comment has been minimized.

lateboomer commented May 10, 2018

When I ran the scripts for Beaglebone Black, half way through it lost networking services and etho was not configured. May I know how do I configure network services to continue the run? may be it purges the network packages.

@Fourdee

This comment has been minimized.

Owner

Fourdee commented May 10, 2018

@lateboomer

Thanks for the report 👍

Which Pre-image did you use, official BBB image?

@lateboomer

This comment has been minimized.

lateboomer commented May 10, 2018

I am using this pre-image as suggested above:

http://debian.beagleboard.org/images/bone-debian-9.3-iot-armhf-2018-01-28-4gb.img.xz

The BBB image I downloaded the latest one from the download page.

Note:
It is not BBB image, what I meant is it is PREP_SYSTEM_FOR_DIETPI.sh scripts I downloaded from the download page. When I run the scripts half way eth0 and dns name derivation all lost.

@valerivp

This comment has been minimized.

valerivp commented May 28, 2018

Orange Pi i96 on Debian after instalation DietPi (with out errors), and reboot wrote:

Please login to continue

But not show command promt
2018-05-28_1642

@alexz-kh

This comment has been minimized.

alexz-kh commented Jun 26, 2018

Folks , is it possible to pass predefined

export PREIMAGE_INFO="http://example.com"
export G_HW_MODEL='35'
export WIFI_REQUIRED=1

variables ? to be able build w\o manual input ?

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Jun 26, 2018

@kuzinnv
Not yet, but the idea is great to allow predefining all user input 👍.
I will add PR for this later.

@johbar

This comment has been minimized.

johbar commented Jul 5, 2018

As far as I see pure 32 bit architectures (i686) are not supported. Why? Is there a workaround available? (I hope that's the right place for this kind of question.)

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Jul 5, 2018

@johbar
Please read: #1854 (comment)

We would need to find install options for x32 for all our offered software titles, implement and test them. It's just a too huge rework for our limited man power, although I understand that especially for very old x32 machines in particular a very lightweight OS is interesting.

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Jul 26, 2018

Note:
Dev/Testing branch currently has issues (globals?):

Seems fine here:

[  OK  ] DietPi-PREP | Step 2 (inputs): Image info / Hardware / WiFi / Distro:
[ INFO ] DietPi-PREP | -----------------------------------------------------------------------------------
[ INFO ] DietPi-PREP | Setting G_HW_MODEL index of: 42
[ INFO ] DietPi-PREP | CPU ARCH = 3 : aarch64
[ INFO ] DietPi-PREP | WiFi selection
[ INFO ] DietPi-PREP | Marking WiFi as needed
[ INFO ] DietPi-PREP | Disabled Distro downgrade: index 3

Not here:

root@rockpro64:/home/rock64# G_HW_MODEL=42 /DietPi/dietpi/func/dietpi-set_software apt-mirror 'default'
[  OK  ] Root access verified.
[  OK  ] DietPi-Drive_Manager | RootFS R/W access verified.


 DietPi-Set_software
─────────────────────────────────────────────────────
 Mode: apt-mirror (default)

[  OK  ] DietPi-Set_software | Desired setting in /DietPi/dietpi.txt was already set: CONFIG_APT_RASPBIAN_MIRROR=http://raspbian.raspberrypi.org/raspbian
[  OK  ] apt-mirror http://raspbian.raspberrypi.org/raspbian | Completed

DietPi-Set_software always reporting G_HW_MODEL=0 during debug.

Due to these changes:

	#Device details
	# - Declare
	G_HW_MODEL=0
	G_HW_MODEL_DESCRIPTION='NULL'
	G_HW_ARCH=0
	G_HW_ARCH_DESCRIPTION='NULL'
	G_HW_CPUID=0
	G_HW_CPU_CORES=$(nproc --all)

They need to be static init.

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Jul 26, 2018

@Fourdee
I guess this is related: #1960

Wait... Raspbian on Rock64?

Fourdee pushed a commit that referenced this issue Jul 26, 2018

Daniel (Fourdee)
v6.13
+ Static init required: #1285 (comment)
@Fourdee

This comment has been minimized.

Owner

Fourdee commented Jul 26, 2018

@MichaIng

Wait... Raspbian on Rock64?

😉 🤣 👍

Yep, issues with globals script changes:
Fixed with 36111dc

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Jul 27, 2018

@MichaIng

You were right, systemd-sysv deps on systemd, i've updated the pre-req in 1st post 👍

@k-plan

This comment has been minimized.

Collaborator

k-plan commented Oct 3, 2018

Hi,

have playing around a little bit with PREP_SYSTEM_FOR_DIETPI.sh.
Read in Changelog, dietpi.txt and Getting started guide
we now need /boot/dietpi-wifi.txt since v6.16.

After doing PREP_SYSTEM_FOR_DIETPI.sh , where will no /boot/dietpi-wifi.txt generated/copied.
Yes, I have see this dietpi-wifi.txt in your prebuild DietPi_v6.16_RPi-ARMv6-Stretch.img:

#---------------------------------------------------------------
# - Entry 0
#	WiFi SSID (Case Sensitive)
aWIFI_SSID[0]=""
#	Key options: If no key (open), leave this blank
aWIFI_KEY[0]=""
#	Available options: NONE (no key/open) | WPA-PSK | WEP | WPA-EAP (then use settings below)
aWIFI_KEYMGR[0]="WPA-PSK"
# - WPA-EAP Options
aWIFI_PROTO[0]=""
aWIFI_PAIRWISE[0]=""
aWIFI_AUTH_ALG[0]=""
aWIFI_EAP[0]=""
aWIFI_IDENTITY[0]=""
aWIFI_PASSWORD[0]=""
aWIFI_PHASE1[0]=""
aWIFI_PHASE2[0]=""
#	Location of the certificate file (eg: /boot/mycert.cer)
aWIFI_CERT[0]=""
#---------------------------------------------------------------
# - Entry 1
#	WiFi SSID (Case Sensitive)
aWIFI_SSID[1]=""
#	Key options: If no key (open), leave this blank
aWIFI_KEY[1]=""
#	Available options: NONE (no key/open) | WPA-PSK | WEP | WPA-EAP (then use settings below)
aWIFI_KEYMGR[1]="WPA-PSK"
# - WPA-EAP Options
aWIFI_PROTO[1]=""
aWIFI_PAIRWISE[1]=""
aWIFI_AUTH_ALG[1]=""
aWIFI_EAP[1]=""
aWIFI_IDENTITY[1]=""
aWIFI_PASSWORD[1]=""
aWIFI_PHASE1[1]=""
aWIFI_PHASE2[1]=""
#	Location of the certificate file (eg: /boot/mycert.cer)
aWIFI_CERT[1]=""
#---------------------------------------------------------------
# - Entry 2
#	WiFi SSID (Case Sensitive)
aWIFI_SSID[2]=""
#	Key options: If no key (open), leave this blank
aWIFI_KEY[2]=""
#	Available options: NONE (no key/open) | WPA-PSK | WEP | WPA-EAP (then use settings below)
aWIFI_KEYMGR[2]="WPA-PSK"
# - WPA-EAP Options
aWIFI_PROTO[2]=""
aWIFI_PAIRWISE[2]=""
aWIFI_AUTH_ALG[2]=""
aWIFI_EAP[2]=""
aWIFI_IDENTITY[2]=""
aWIFI_PASSWORD[2]=""
aWIFI_PHASE1[2]=""
aWIFI_PHASE2[2]=""
#	Location of the certificate file (eg: /boot/mycert.cer)
aWIFI_CERT[2]=""
#---------------------------------------------------------------
# - Entry 3
#	WiFi SSID (Case Sensitive)
aWIFI_SSID[3]=""
#	Key options: If no key (open), leave this blank
aWIFI_KEY[3]=""
#	Available options: NONE (no key/open) | WPA-PSK | WEP | WPA-EAP (then use settings below)
aWIFI_KEYMGR[3]="WPA-PSK"
# - WPA-EAP Options
aWIFI_PROTO[3]=""
aWIFI_PAIRWISE[3]=""
aWIFI_AUTH_ALG[3]=""
aWIFI_EAP[3]=""
aWIFI_IDENTITY[3]=""
aWIFI_PASSWORD[3]=""
aWIFI_PHASE1[3]=""
aWIFI_PHASE2[3]=""
#	Location of the certificate file (eg: /boot/mycert.cer)
aWIFI_CERT[3]=""
#---------------------------------------------------------------
# - Entry 4
#	WiFi SSID (Case Sensitive)
aWIFI_SSID[4]=""
#	Key options: If no key (open), leave this blank
aWIFI_KEY[4]=""
#	Available options: NONE (no key/open) | WPA-PSK | WEP | WPA-EAP (then use settings below)
aWIFI_KEYMGR[4]="WPA-PSK"
# - WPA-EAP Options
aWIFI_PROTO[4]=""
aWIFI_PAIRWISE[4]=""
aWIFI_AUTH_ALG[4]=""
aWIFI_EAP[4]=""
aWIFI_IDENTITY[4]=""
aWIFI_PASSWORD[4]=""
aWIFI_PHASE1[4]=""
aWIFI_PHASE2[4]=""
#	Location of the certificate file (eg: /boot/mycert.cer)
aWIFI_CERT[4]=""

Or do I miss something?

Thanks and see you.

k-plan

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Oct 3, 2018

@k-plan
Hi, nice too see you here again 😃.

Actually
https://github.com/Fourdee/DietPi/blob/master/PREP_SYSTEM_FOR_DIETPI.sh#L1407-L1409
should lead to dietpi-wifi.txt being created. Did this not happen during your test?

An issue I could think is that private WiFi SSID/key could be stored inside. Either users should be informed about that, their current WiFi credentials from image creation system being stored on the image, or we should anonymize it, or simply put a raw default in place?

@k-plan

This comment has been minimized.

Collaborator

k-plan commented Oct 3, 2018

Hi @MichaIng ,

should lead to dietpi-wifi.txt being created. Did this not happen during your test?

Sadly no.

The device has no Wifi buildin, connected or Wifi connection configured.
So Wifi was disabled in dietpi-config
But i have choose I require WiFi functionality, keep/install related packages.
Is this a problem?

An issue I could think is that private WiFi SSID/key could be stored inside

Puuh... think that's not the best idea ... 😞

... or simply put a raw default in place?

Yes, I will vote for this please. 😃

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Oct 3, 2018

@k-plan

Hello you, long time no see, hope all is well? 😃

I believe there was a bug in the v6.16 PREP (and master branch), where it would not generate the dietpi-wifi.db file automatically, if it did not exist (whoopsie!)

This is now resolved in the v6.17 update (via testing branch currently). If you run PREP with testing branch, should be fine :)

@k-plan

This comment has been minimized.

Collaborator

k-plan commented Oct 3, 2018

@Fourdee

Hi Dan,
I'm fine, thanks.

Sounds good, will try this, if I find some spare time.

Thanks.

Congratulations to @Fourdee and @MichaIng for coding progress on your DietPi. Pretty nice work! ❤️

@Fourdee

This comment has been minimized.

Owner

Fourdee commented Oct 13, 2018

Changes to PREP run script, for v6.17 release:

wget https://raw.githubusercontent.com/Fourdee/DietPi/master/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/Fourdee/DietPi/beta/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/Fourdee/DietPi/dev/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
chmod +x PREP_SYSTEM_FOR_DIETPI.sh
sudo ./PREP_SYSTEM_FOR_DIETPI.sh
@Akito13

This comment has been minimized.

Akito13 commented Oct 14, 2018

Greetings,

I want to run this script on my Orange Pi i96, but this device has neither an HDMI nor a LAN interface, so I really rely completely on being able to SSH into this thing over Wi-Fi. How do I make sure that I can connect to the system, after all the existing software (APT) and user data will be deleted?
I really like the aforementioned idea of predefining user input.

@MichaIng

This comment has been minimized.

Collaborator

MichaIng commented Oct 14, 2018

@Akito13
Currently DietPi-PREP does not run reliably on WIFI-only devices without HDMI. Note: https://github.com/Fourdee/DietPi/blob/master/PREP_SYSTEM_FOR_DIETPI.sh#L5-L9

E.g. if NetworkManager is installed, it will be uninstalled, which drops the connection, thus SSH will be lost and script cancelled.

  • Just found an open issue about this: #2054

So either a local terminal is needed (thus HDMI) or an Ethernet connection that assures ongoing SSH connection without relying on a certain connection manager/method.

Theoretically it should be possible to start the script via SSH within a screen session. This would resume, even if SSH is lost. All the interactive part is done, before any connection related changes are done, so theoretically it works. No ordinary shutdown can be done then, but waiting better longer then shorter until every activity stopped, then plug off from power, should do it.

But we had some WiFi/interface init issues with v6.16, mostly on Jessie systems. But since we cleaned up all the interface/connection stuff at (first) boot, I would wait for v6.17 before doing much effort for the above attempts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment