Script: bsdinstall.sh

[mihakralj]

I created a shell script that finds out all binary packages and their dependencies in FreeBSD repo - without having repo installed on OPNsense. I use it instead of using ports to build missing tools from the source.

for example, I love to use bat (a cat clone with syntax highlighting and Git integration), but do not want to build bat from ports as it requires installing rust and what not - a massive over-burden just to get bat working. So, I run ./bsdinstall.sh bat instead and get list of required pkg add commands (in reverse order):

sudo pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/bat-0.23.0_3.pkg
sudo pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/libgit2-1.5.2_1.pkg
sudo pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/http-parser-2.9.4.pkg
sudo pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/libssh2-1.11.0,3.pkg
sudo pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/pcre2-10.42.pkg
sudo pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/oniguruma-6.9.8_1.pkg

This is not_at_all preferred method of installing packages without repo and no lineage, but it is still better than building packages from ports.

Here is the script, if it makes sense, let's add it to the community repo.

#!/usr/local/bin/bash

if [ -z "$1" ]; then
  echo "Usage: $0 <package-name>"
  exit 1
fi

###############################

function get_package_url() {
  local package_name="$1"

  # Find the line with the requested package
  local line=$(awk -v package="$package_name" '$0 ~ "{\"name\":\"" package "\"" {print $0}' "${tmp_dir}/packagesite.yaml")

  # Check if the line is empty
  if [ -z "$line" ]; then
    echo "Error: Package $package_name not found."
    exit 1
  fi

  # Get URL
  local repopath="https://pkg.freebsd.org/FreeBSD:${freebsd_version}:${freebsd_abi}/latest/$(echo "$line" | grep -o '"repopath":"[^"]*"' | cut -d\" -f4)"
  echo "sudo pkg add $repopath"

  # Extract dependencies
  local dep_content=$(echo "$line" | awk -F'"deps":{' '{print $2}' | awk -F'},"categories"' '{print $1}')
  local deps=$(echo "$dep_content" | grep -o '"[^"]*":{"origin"' | sed 's/":{"origin"//g' | tr -d '"')

  # Recursive call for each dependency
  for dep in $deps; do
    get_package_url "$dep"
  done
}

###############################

# Package name
package_name="$1"

# Determine FreeBSD version and ABI
freebsd_version=$(freebsd-version -u | cut -d- -f1 | cut -d. -f1) # Taking the major version only
freebsd_abi=$(uname -m)

# URL to the packagesite file
url="https://pkg.freebsd.org/FreeBSD:${freebsd_version}:${freebsd_abi}/latest/packagesite.txz"

# Create a temporary directory if it doesn't exist
tmp_dir="/tmp/pkg_site_tmp_dir"
mkdir -p "$tmp_dir"

# Fetch and unpack the packagesite file if packagesite.yaml doesn't exist
if [ ! -f "${tmp_dir}/packagesite.yaml" ]; then
  fetch -o "${tmp_dir}/packagesite.txz" "$url"
  tar xf "${tmp_dir}/packagesite.txz" -C "$tmp_dir"
  rm "${tmp_dir}/packagesite.txz" # Delete the tar file after extracting
fi

get_package_url "$package_name"

[mimugmail]

Looks good, but using vanilla pkg is highly unstable as pkg version sometimes differs and will break your system

[mihakralj]

Aren't these packages exactly the same as building them from ports? Risk is exactly the same, the difference is that building from ports demands the whole build chain of tools to be installed as well (go, rust or whatever)

If someone wants a tool from FreeBSD to run on OPNsense, what is worse: direct install of binary with pkg add or local compile from ports?

[mimugmail]

The problem is when opnsense builds against 1.19 and freebsd is build against 1.20 it will break. Already seen with 1.17 and 1.18 last year with 22.1

[mihakralj]

...and the same problem would happen with bsd ports? Or not?
The advantage of pkg add in this case is that it will never follow the version upgrades as it is not attached to BSD repo. One pull of binary and that's it. If it passes the first install on opnsense and works, pkg add no longer depends on version upgrades of freebsd repo.

This is - of course - also a curse, as it will never auto-upgrade (just like ports) and it is on admin to keep its track.

[mihakralj]

The question is: is this script worth adding among the tinkering tools to the community repo?
Here are couple of reasons (modern unix tools) why I think it matters to help OPNsense users to install direct freebsd tools that OPNsense is missing (the script above can generate direct pkg add link to all of them):

• doggo - command-line DNS analysis client (does DOT and DOH as well)
• drill - another dig alternative
• gping - a CLI for ping that draws graphs, measures jitter and oh-my
• bottom - a modern insightful replacement for top (after install, the command is btm)
• duf - a modern alternative to df
• ngrep - command line network packet analyzer
• hyperfine - statistical benchmarking of any task across multiple runs
• lazygit - using git without having PhD in git

Installing any/all of these tools without direct link to freebsd binaries is a nightmare. Not having these tools on opnsense is even greater nightmare. :-)