Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DietPi-Globals | Replace duplicate function/var code in our scripts #1311

Closed
Fourdee opened this issue Dec 17, 2017 · 27 comments
Closed

DietPi-Globals | Replace duplicate function/var code in our scripts #1311

Fourdee opened this issue Dec 17, 2017 · 27 comments
Assignees
Milestone

Comments

@Fourdee
Copy link
Collaborator

Fourdee commented Dec 17, 2017

By simply loading this script on demand to export global vars.

Also allows for logged in bash user session to use our commands (eg: AGI) etc.

@Fourdee Fourdee added this to the v160 milestone Dec 17, 2017
@Fourdee Fourdee self-assigned this Dec 17, 2017
@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 17, 2017

@MichaIng

root@DietPi:~# AGI thispackagedoesnotexist_Test
 [Info] APT is processing, please wait...

E: Unable to locate package thispackagedoesnotexist_Test

Fourdee referenced this issue Dec 17, 2017
+ Globals test with dietpi-software:
https://github.com/Fourdee/DietPi/issues/1311
Fourdee referenced this issue Dec 17, 2017
+ Switch to DietPi-Global imports in all scripts (excluding those within
the script itself): https://github.com/Fourdee/DietPi/issues/1311
Fourdee referenced this issue Dec 17, 2017
+ Rename to support error checking for AGI etc:
https://github.com/Fourdee/DietPi/issues/1311
Fourdee referenced this issue Dec 17, 2017
+ Minor, revised function names to match those of Global:
https://github.com/Fourdee/DietPi/issues/1311
@MichaIng
Copy link
Owner

MichaIng commented Dec 17, 2017

Nice! 👍
Just some minor questions/hints:

  • Why not include the error check/output to base AGI etc. directly? Would be helpful in every script and terminal, I think.
  • Since -qq includes -y, we can remove the latter: https://wiki.ubuntuusers.de/apt/apt-get/ Why ubuntu wiki is so much better/more detailed than debian ones? Totally missing complete command overviews with all sub commands and options there! 😛.

@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 18, 2017

@MichaIng

Why not include the error check/output to base AGI etc. directly? Would be helpful in every script and terminal, I think.

Makes sense, i'll take a look. 👍

Since -qq includes -y, we can remove the latter

Its a habit thing more than anything, although, having -y in there, ensures if we (or apt) change any options in the future, the -y should always ensure start of install. I'll leave it in for now.

@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 19, 2017

ToDo:

  • 🈯️ Global dietpi-notify
  • 🈯️ Replace existing /DietPi/dietpi/dietpi-notify commands in dietpi scripts with dietpi-notify etc
  • 🈯️ Replace existing apt-get commands in dietpi scripts with AGI etc
  • 🈯️ Send my commit
  • 🈯️ Manually fix conflicts on outstanding PR's
    -- 🈯️+ using global dietpi-notify
    -- 🈯️+ using global AGX commands

  • dietpi-drive_manager not listing USB drives, list mode is 1, caused by:
    -- Print_Input_String() in dietpi-notify in dietpi-global.
    --- 🈯️ Fixed, seems i was not local to dietpi-global and being passed back to script loading dietpi-notify
    --- confirm defined i is not changed by dietpi-notify
root@DietPi:~# i=1230
root@DietPi:~# . /DietPi/dietpi/func/dietpi-globals
root@DietPi:~# dietpi-notify 0 Test program
 [Ok] Test program
root@DietPi:~# echo $i
1230

#and visa versa
root@DietPi:~# unset i
root@DietPi:~# . /DietPi/dietpi/func/dietpi-globals
root@DietPi:~# dietpi-notify 0 Test program
 [Ok] Test program
root@DietPi:~# echo $i

Fourdee referenced this issue Dec 19, 2017
local ```i``` in dietpi-notify, to avoid overwriting host script:
https://github.com/Fourdee/DietPi/issues/1311#issuecomment-352638142
@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 19, 2017

@MichaIng

Why not include the error check/output to base AGI etc. directly? Would be helpful in every script and terminal, I think.

We can provide a notify for result of the apt-get command, but if we want to handle the end result, we'll need to do this outside the global script.
eg: in dietpi-software, we need to handle the exit code, then we can match it to software title etc if its a failure. If we simply exit in dietpi-global AGX, we lose the ability to do that.

image

Fourdee referenced this issue Dec 19, 2017
@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 19, 2017

This is now completed as intended, however, we'll continue to add/improve it as needed in the future.

@Fourdee Fourdee closed this as completed Dec 19, 2017
@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 20, 2017

dietpi-drive_manager move userdata:

root@DietPi:~# readlink -f /mnt/dietpi_userdata
/mnt/00293bdc-f422-41a3-99f9-7bfd50eeb266/dietpi_userdata
root@DietPi:~# readlink -f /mnt/dietpi_userdata
/mnt/dietpi_userdata

root@DietPi:~# echo $(/DietPi/dietpi/func/dietpi-set_userdata return_source)
[Info] DietPi-Globals loaded  [Info] Checking for (elevated) root access, please wait..  [Ok] Root access verified. /mnt/dietpi_userdata

Ok,

  • 🈴 Disable any print for a default load of globals?, nope, we need error information if rootfs rw, root user access fails etc.
    image

@Fourdee Fourdee reopened this Dec 20, 2017
Fourdee referenced this issue Dec 20, 2017
+ Remaning hard coded "$FP_DIETPI_USERDATA", updated to use globals:
https://github.com/Fourdee/DietPi/issues/1311
@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 20, 2017

Completed (again).

@Fourdee Fourdee closed this as completed Dec 20, 2017
@Fourdee Fourdee reopened this Dec 20, 2017
Fourdee referenced this issue Dec 20, 2017
+ Use func return value for handling AGX errors, instead of a global
+ Check_URL connection test now global command
https://github.com/Fourdee/DietPi/issues/1311
@Fourdee
Copy link
Collaborator Author

Fourdee commented Dec 20, 2017

  • use of G_ at start of all global commands/vars to clearly define global status, and, avoid variable interaction.
root@DietPi:~# G_
G_AGA              G_AGI              G_AGUP             G_CHECK_URL
G_AGDUG            G_AGP              G_CHECK_ROOTFS_RW  G_DIETPI-NOTIFY
G_AGF              G_AGUG             G_CHECK_ROOT_USER

Fourdee referenced this issue Dec 20, 2017
+ use of G_ at start of all global commands/vars to clearly define
global status, and, avoid variable interaction:
https://github.com/Fourdee/DietPi/issues/1311

+ define git_branch to use before run:
https://github.com/Fourdee/DietPi/issues/1285#issue-280771944

+ Testing dietpi-globals in PREP_SYSTEM:
https://github.com/Fourdee/DietPi/issues/1285#issuecomment-352756354
Fourdee referenced this issue Dec 31, 2017
+ Prevent duplicate checks under same session:
https://github.com/Fourdee/DietPi/issues/1311
@Fourdee
Copy link
Collaborator Author

Fourdee commented Jan 1, 2018

Testing/notes: https://github.com/Fourdee/DietPi/pull/1364

  • Processing only gets replaced by ok/failure, with G_RUN_CMD and few others. Ah i see, NOTIFY -2 rollout.
  • G_CHECK_URL does not replace
  • 🈯️ Fourdee@d5b7c01 G_CHECK_URL hangs on a failed connection. No result. Due to error output being printed in background. Due escaping overwrite string, not needed.
  • 🈺 Replace does not replace, if output hits screen width :
    -- Maybe another idea is to use echo -en "\033[1A 1 \033[0m" which will print on the previous line?
root@DietPi:~# G_RUN_CMD systemctl enable dietpi-fs_partition_resize.service
 [Processing] Running command: systemctl enable dietpi-fs_partition_resize.servi [    Ok    ] systemctl enable dietpi-fs_partition_resize.service

root@DietPi:~# G_RUN_CMD echo 1
 [    Ok    ] echo 1

Fourdee referenced this issue Jan 1, 2018
@Fourdee
Copy link
Collaborator Author

Fourdee commented Jan 2, 2018

Notes: https://github.com/Fourdee/DietPi/pull/1365

  • @MichaIng Looks much better 👍
  • 🈯️ Fixed with log file generation. G_CHECK_URL http://google.comxxxx hangs, stuck in BG. kill tail process, whip appears.
    image

@Fourdee
Copy link
Collaborator Author

Fourdee commented Jan 8, 2018

Notes:

  • Incorrect usermsg (not cleared from previous)
  • Missing dietpi-software info
 [  Error   ] DietPi-Software: G_AGI: mosquitto
 [   Info   ] exit_code = 100
 [   Info   ] VERSION:v6.-1 | HW_MODEL:3 | HW_ARCH:2 | DISTRO:4
 [   Info   ] Log file contents:
E: Failed to fetch http://repo.mosquitto.org/debian/pool/main/m/mosquitto/mosquitto_1.4.12-0mosquitto1_armhf.deb  404  Not Found
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
 [   Info   ] User message:
Connection failed while trying to access: http://dietpi.com/downloads/binaries/all/libwebsockets3_1.2.2-1_armhf.deb. The URL may be offline, or invalid.
 [   Info   ] If problems persist, please report this to DietPi for investigation, including a screenshot of this error! (https://github.com/Fourdee/DietPi/issues).


G_ERROR_HANDLER_ONERROR_USERMSG must never be defined changed in dietpi-globals additional functions. It must only be exported by the sourcing script, to ensure it never gets overwritten.

The downside:

  • G_ERROR_HANDLER_ONERROR_USERMSG will be displayed for any future errors, regardless if its outdated. Unless we manually add unset G_ERROR_HANDLER_ONERROR_USERMSG in the sourcing script after each event (messy)

Solution for now, remove G_ERROR_HANDLER_ONERROR_USERMSG from scripts. See if we can implement this better in the future.

Fourdee referenced this issue Jan 8, 2018
@Fourdee
Copy link
Collaborator Author

Fourdee commented Jan 14, 2018

I'll mark this as completed. Will reopen if required.

@Fourdee Fourdee closed this as completed Jan 14, 2018
@Fourdee Fourdee mentioned this issue Jan 28, 2018
Fourdee referenced this issue Jan 28, 2018
**v6.0** (previously v160)
(28/01/18)

**Important Information:**
**All DietPi images have been re-created. Existing installations (v159 or lower), can no longer be updated, or supported. To continue support, users must install the latest v6.0 image.**
 - https://github.com/Fourdee/DietPi/issues/1385
 - All images are now Debian Stretch (excluding Odroid's)
 - ARMbian based images are now mainline kernel 4.13+.
 - Native PC (EFI): is now an ISO, with clonezilla bundled. Simplifies installation via Rufus write: https://github.com/Fourdee/DietPi/issues/1171#issuecomment-336522021
 - If you are happy with your existing installation of v159 (or lower), you are not required to install the v6.0 image, however, we cannot continue to provide support for v159 (or lower) installations.

**Minor notes:**
The XMAS tree has now been taken down, stored away on github history for next year. Hope you all had a good one :D

**Changes / Improvements / Optimizations:**

General | DietPi RPi kernel, now reverted to stock RPi kernel: https://github.com/Fourdee/DietPi/issues/1378

General | We have completed much needed backbone work for DietPi, which will allow for improved expansion in source code. This includes the use of dietpi-globals.

DietPi-Globals | New script which optimizes most used DietPi commands and vars, throughout our scripts. Also exported to bash session, please type 'G_' then press 'TAB' to see a full list of options: https://github.com/Fourdee/DietPi/issues/1311

General | FHS compliance completed. /etc/dietpi has moved to /var/lib/dietpi. RAMlog store has moved to /var/tmp/dietpi: https://github.com/Fourdee/DietPi/issues/1297#issuecomment-352241193

General | We have refreshed our terminal messages look & feel, oriented on RPi boot messages, and with process animation: https://github.com/Fourdee/DietPi/pull/1377

General | wget: Now set to prefer IPv4 by default (generally faster, can be changed by 'CONFIG_PREFER_IPVERSION' in dietpi.txt): https://github.com/Fourdee/DietPi/issues/1285#issuecomment-353230187

General | APT: Now set to force IPv4 by default (generally faster, can be changed by 'CONFIG_PREFER_IPVERSION' in dietpi.txt): https://github.com/Fourdee/DietPi/issues/1285#issuecomment-353230187

General | SparkySBC: CPU gov default changed to Performance, reports of increased stability.

General | Swapfile generation is now completed during 1st run of dietpi-software (previously boot stage): https://github.com/Fourdee/DietPi/issues/1270#issue-278797206

General | DietPi-Funtime: Removed from DietPi. Although it looked pretty, it did absolutely nothing (except slow down a program)

DietPi-Automation | All dietpi.txt entries have been renamed and cleaned up.

DietPi-Automation | dietpi.txt: CONFIG_NTP_MODE will now be applied during 1st run of device: https://github.com/Fourdee/DietPi/issues/1379

DietPi-Boot | Improved the method of initial FS_partition and FS_expansion during 1st run, via systemD services. 'fs_force_resize=' in dietpi.txt is no longer supported: https://github.com/Fourdee/DietPi/issues/1285#issuecomment-352159930

DietPi-Banner | IP: Will now also list the active network adapter used (eg: eth0/wlan0)

DietPi-Config | Dion Audio LOCO V1/V2: Soundcards added for RPi.

DietPi-Config | Locale: en_GB.UTF-8 is now automatically installed, alongside user selected choice. Required for DietPi scripts to function.

DietPi-Drive_Manager | Added support for exFAT, many thanks @MichaIng : https://github.com/Fourdee/DietPi/pull/1312

DietPi-Globals | Global variables and functions are now exported during login. Please see the sourcecode for more information: https://github.com/Fourdee/DietPi/issues/1311

DietPi-Set_Hardware | Sparky SBC: enable aotg.aotg1_speed compatibility setting for USB 1.1, when USB-DAC configured: https://github.com/Fourdee/DietPi/issues/1301

DietPi-Set_Software | "pool" directive is now used for NTPD: https://github.com/Fourdee/DietPi/pull/1404

DietPi-Software | NAA Daemon: Updated to latest (3.5.2-36). Existing installs will be patched automatically: https://github.com/Fourdee/DietPi/issues/1305

DietPi-Software | PHP-FPM: Increased from "$CPU_CORES_TOTAL" to "pm.max_children = $(( $CPU_CORES_TOTAL * 3 ))". This should avoid failed forking of PHP-FPM processes/requests : https://github.com/Fourdee/DietPi/issues/1298

DietPi-Software | ownCloud/Nextcloud: Added option to choose data directory via dietpi.txt pre installation: https://github.com/Fourdee/DietPi/issues/1314#issuecomment-352782055

DietPi-Software | ownCloud/Nextcloud: Switch to pretty URLs (without "index.php") on Apache

DietPi-Software | ownCloud/Nextcloud: Automated backup restoring on install and creation und uninstall to ownCloud/Nextcloud data directory

DietPi-Software | ownCloud: Switch to non-package/archive installation. This allows usage of preferred web based updater.

DietPi-Software | Nextcloud: Resolved OPcache admin panel warnings now also on Lighttpd

DietPi-Software | UrBackup: Installation updated to latest version 2.1.20. For new installations only: https://github.com/Fourdee/DietPi/issues/1335

DietPi-Software | NodeRed: Corrected user which nodered runs under, now runs as its own user, created during install: https://github.com/Fourdee/DietPi/issues/1294#issuecomment-354314318

DietPi-Software | SqueezeBox/LMS (Stretch): Installation resolved: https://github.com/Fourdee/DietPi/issues/1124

DietPi-Software | MySQL: Completely remove MySQL from DietPi in favour of MariaDB: https://github.com/Fourdee/DietPi/issues/1397

DietPi-Software | Ampache: MySQL DB and configs have been updated (adds correct userdata folder for music by default): https://github.com/Fourdee/DietPi/issues/1420

run_ntpd | Added support for systemd-timesyncd completion/detection: https://github.com/Fourdee/DietPi/issues/1379

**Bug Fixes:**

General | Fixed two systemd error messages during shutdown and boot: https://github.com/Fourdee/DietPi/issues/1330

DietPi-Automation | Resolved an issue where AUTO_SETUP_TIMEZONE was not being applied correctly, thanks @k-plan: https://github.com/Fourdee/DietPi/issues/1285#issuecomment-356310496

DietPi-Automation | dietpi.txt: CONFIG_NTP_MIRROR will now be applied to systemd-timesyncd configuration: https://github.com/Fourdee/DietPi/issues/1379

DietPi-Config | Resolved an issue with WiFi Country code, failing to set on some devices: https://github.com/Fourdee/DietPi/issues/838

DietPi-Config | Resolved an issue where disabling IPv6 didn't have an effect on AMD64 devices: https://github.com/Fourdee/DietPi/issues/1343#issuecomment-359652751

DietPi-Services | dietpi-wifi-monitor: Is no longer controlled, to prevent WiFi drop during software installs/updates etc: https://github.com/Fourdee/DietPi/issues/1288#issuecomment-350653480

DietPi-Software | General: MySQL using software titles now have their own database user, instead of accessing as "root": https://github.com/Fourdee/DietPi/issues/1397#issuecomment-359655198

DietPi-Software | qBittorrent: Resolved an issue with inability to log into web interface: https://github.com/Fourdee/DietPi/issues/1366

DietPi-Software | Resolved an issue where our custom LD_LIBRARY_PATH would cause APT failures. LD_LIBRARY_PATH has now been reverted, apologies if this effected your system: https://github.com/Fourdee/DietPi/issues/1329

DietPi-Software | Resolved an issue where APT installations would fail if services were masked. All known 

DietPi software services, will be enabled/unmasked, before installation: https://github.com/Fourdee/DietPi/issues/1320

DietPi-Software | WiFi Hotspot (Stretch): Resolved an issue where hostapd would fail to run due to missing libssl1.0.0 lib, not available in repos: https://github.com/Fourdee/DietPi/issues/1299

DietPi-Software | Shairport-sync (Stretch): Resolved an issue where this would fail to install, due to pre-req URLS becomming invalid: https://github.com/Fourdee/DietPi/issues/1303

DietPi-Software | Plex Media Server: Resolved uninstall to include /var/lib/plexmediaserver in removal (which is not completed via apt purge).

DietPi-Software | MariaDB: Resolved an issue where MariaDB would fail to uninstall correctly: https://github.com/Fourdee/DietPi/pull/1280

DietPi-Software | Aira2 (Stretch): Resolved installation, now used APT installation: https://github.com/Fourdee/DietPi/issues/1310

DietPi-Software | Mosquitto: Resolved various issues with failed install, due to Mosq repo not being maintained (deb's missing from repo header list, requires non-stretch available packages). deb's are now hosted on dietpi.com: https://github.com/Fourdee/DietPi/issues/1306

DietPi-Software | ownCloud/Nextcloud: Fixed an installation issue on Jessie with MariaDB: https://github.com/Fourdee/DietPi/pull/1319

DietPi-Software | Google AIY: Updated install to gitbranch=voicekit. Many thanks to @mpember for the heads up: https://github.com/Fourdee/DietPi/issues/1065#issuecomment-354304388

DietPi-Software | OpenJDK: Replaces OracleJDK: https://github.com/Fourdee/DietPi/issues/1401

DietPi-Update | dietpi.txt is now checked for missing entries, and, will now be patched during the update: https://github.com/Fourdee/DietPi/issues/1292#issuecomment-350818969

Sparky SBC | Kernel updated, which resolves issues with HQPlayer playback: https://www.computeraudiophile.com/forums/topic/32132-allo-sparky-usbridge/?do=findComment&comment=753100

**Allo Web Interface v5:**

Sparky SBC: Matrix Audio X-SPDIF 2, native DSD is now added to kernel, many thanks @sudeep: sparkysbc/Linux#3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants