installation without docker don't run speedtest

[Fabick75]

Because of my setup (I need to use an LXC container on Proxmox and don't want to overload the machine with a VM that run docker) I've done an LXC container (basically a striped vm with debian), downloaded the latest release and installed like a normal laravel project. But don't start speedtest-cli to do the measurements.

Steps to reproduce the behavior:

1. Install a VM with Debian (in my case an LXC container on Proxmox)
2. download latest .tar.gz from github repository and deflate in a destination dir
3. install dependencies (php8.2, composer 2, speedtest-cli, curl, mariadb)
4. run one time speedtest-cli to be sure that works
5. create an empty DB
6. copy .env.example in .env, in the root directory of the archive, then set the database credentials
7. from the root archive directory run: composer install --no-interaction --prefer-dist --optimize-autoloader --no-dev --no-cache
8. then, from the same place, run : php artisan app:install
9. if everything goes fine, run the service with : php artisan serve --host 0.0.0.0
10. open the browser and go to the webpage : http://yourip:8000, then login with default credentials
11. set the interval to make tests (to make some tests I've set every 2 minutes)

Expected behavior:
I expect that the results populate the graphs. Even If i add test with the "queue speedtest" button, nothing appen. Nothing seems to indicate problems in the artisan console

Environment :

• OS: Debian 12.2.0
• Architecture: amd64 (virtualized on Proxmox)
• Browser : Mozilla Firefox 115.7
• Version : 0.15.2

Nothing to add. Only one observation: I cannot find a button to fire up a test directly, I can only add to queue, and there isn't a way to view the queue from the web interface. Or is just me that not understand the UI .

[Fabick75]

With this .env file :

APP_NAME="Speedtest Tracker"
APP_ENV=local
SPEEDTEST_CLI_PATH="/usr/bin/speedtest"

# Application key
# Used for encryption where needed, if a key is not generated run: php artisan key:generate
APP_KEY=base64:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

APP_DEBUG=false
APP_LOG=daily
APP_URL=http://127.0.0.1

LOG_CHANNEL=stack
LOG_DEPRECATIONS_CHANNEL=null
LOG_LEVEL=debug

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=speedtracker
DB_USERNAME=speedtracker
DB_PASSWORD=xxxxxxxxxx

BROADCAST_DRIVER=log
CACHE_DRIVER=database
FILESYSTEM_DISK=local
QUEUE_CONNECTION=database
SESSION_DRIVER=database
SESSION_LIFETIME=120

MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="Speedtest Tracker"

TELEGRAM_BOT_TOKEN=null

and after setting correctly the time zone in php.ini and on the OS (with timedatectl), the project seems to run, but it's needed to run :

php artisan serve
php artisan queue:work
php artisan schedule:work

in three separate consoles.

If someone suggest another way to launch the project outside of a container, I will be grateful.

[alexjustesen]

Converting this to a discussion, not currently supporting bare metal installs but if you get it working it'll be nice to have documented.

[Fabick75]

Good to see your interest in the process! If this can help, I could write a quick guide with the main steps inside to launch the code. Because I'm not a programmer, but a sysadmin, then I need a little clarification on the three parts to run :

php artisan serve
php artisan queue:work
php artisan schedule:work

for what I know about laravel, when you start "artisan serve" this provide to launch the other modules as needed. There is some particular configuration from your side, or is something that I need to find about the configuration of my testing system?

[sunnycloudy1337]

you need to use supervisor

i have mine setup with 3 config files, 1 for the web interface, 1 for the schedule and 1 for the queue

[alexjustesen]

👆 this is correct, you can use the supervisord.conf file copied into the image to point you in the right direction.

Source: https://github.com/alexjustesen/speedtest-tracker/blob/main/docker/8.2/supervisord.conf

[Fabick75]

Thankyou, I will try to adopt this in the next few days and after I will wrote a simple guide to help others to use the program without dockerization

[alexjustesen]

Also give https://laravel.com/docs/10.x/deployment a good read to understand deployment requirements of the framework.

[Fabick75]

I've tried to wrote a supervisor config file that works, but for some reason the following configuration :

[supervisord]
nodaemon=true
user=root
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid

[program:speedtest_tracker_serve]
command=/usr/bin/php /opt/speedtest-tracker/artisan serve --host=0.0.0.0 --port=8000
directory=/opt/speedtest-tracker
autostart=true
autorestart=true
stderr_logfile=/var/log/speedtest_tracker_serve.err.log
stdout_logfile=/var/log/speedtest_tracker_serve.out.log

[program:speedtest_tracker_queue_work]
command=/usr/bin/php /opt/speedtest-tracker/artisan queue:work
directory=/opt/speedtest-tracker
autostart=true
autorestart=true
stderr_logfile=/var/log/speedtest_tracker_queue_work.err.log
stdout_logfile=/var/log/speedtest_tracker_queue_work.out.log

[program:speedtest_tracker_schedule_work]
command=/usr/bin/php /opt/speedtest-tracker/artisan schedule:work
directory=/opt/speedtest-tracker
autostart=true
autorestart=true
stderr_logfile=/var/log/speedtest_tracker_schedule_work.err.log
stdout_logfile=/var/log/speedtest_tracker_schedule_work.out.log

don't permit to the queue:work to execute correctly. This is what spit out in the logs :

2024-04-12 15:30:01 App\Jobs\Speedtests\ExecuteOoklaSpeedtest ...... RUNNING
2024-04-12 15:30:02 App\Jobs\Speedtests\ExecuteOoklaSpeedtest . 21.42ms FAIL

if I'm launching the process directly from a terminal, all works regularly.

I've tried to debug the reason, but without success. It's not a problem with enviroment variables or similar. What happens with supervisord is not clear to me.

Any hint related to this problem?

I've completed the installation guide, but remain only the startup part that, at the moment is done with a bash script. Because you've suggested to user supervisord, I would to put both of them in the guide before relasing here.

[sunnycloudy1337]

@Fabick75 I had the same problem with both the schedule:work and queue:work that would fail under supervisor, so after a few hours of playing around with them,

I ended up putting the schedule:run to be called every minute in crontab -e

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

* * * * * /usr/bin/php /app/www/artisan schedule:run > /dev/null

and the queue:work as a systemd service

[Unit]
Description=Queue
After=mariadb.service

[Service]
ExecStart=sudo /app/www/artisan queue:work
Restart=on-abort
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Notice the use of sudo, otherwise it would just FAIL like in your case.

Also notice that I set it to start after mariadb server, because otherwise it would throw a connection error in laravel.log (my mysql server is on the same LXC for now, until I move it) but you can also use sqlite I think. I ended up with mysql because I thought that would fix the problems, it did not :P

Lastly my web server is also started by systemd

[Unit]
Description=Web
After=network.target

[Service]
ExecStart=/usr/bin/php /app/www/artisan serve --host=0.0.0.0 --port=80
Restart=on-abort
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

But this one worked from supervisor without any problems, I just decided to uninstall supervisor if I couldn't make it work for queue and schedule and just rely on systemd

PS: My LXC is not debian bookworm, it is based on the ubuntu 22.04 template, just like what @alexjustesen is using for his docker base image

[Fabick75]

Hi. Then seems definitely a problem with supervisord, maybe depends on the way we wrote the config file, I don't know much about this tool. But I think that is not possible that depends from the OS version, or the distro.
In any case, i've written a script that allows you to run the application correctly and it works via TMUX. Then simply install tmux on the LXC and run this. During the day I will put all the config steps in a file an put online there to help who don't want to use the docker version.

[adresner]

This would be amazing and helpful... I am looking to do the same, I use proxmox LXC, it's far more simple than Docker (at least for me). @Fabick75 if you can write up a guide, my noob fingers will follow and report back. Thanks!

[Fabick75]

HI! I hope the guide is useful to you. If you have any feedback on this, I would appreciate it!

[Fabick75]

This would to be a comprehensive basic guide to install the program in an LXC container, on ProxMox platform. I hope that this can help the community. Fell free to use and share!

Installing SpeedTest-Tracker in an LXC container

ProxMox 8.x Steps

Setting Up the LXC Container

1. Create a new container with Debian 12 - 512 MB RAM / 512 MB swap and 2 CPUs - no root privilege and enable SSH (optional but useful if you don't want to operate solely from the ProxMox console).
2. Set the root password.
3. Set a static IP from the LXC interface (for local network access).

Installing Dependencies and Required Software

Access the container terminal and install the following dependencies:

• MariaDB:

apt install mariadb
systemctl enable mariadb && systemctl start mariadb

Create an empty database with corresponding username and password, e.g.:
  > Database name: speedtracker
  > User: speedtracker
  > Password: xxxxx

• Supervisord (article):

  apt install supervisor
  systemctl enable supervisor

• PHP 8.2 with the following dependencies:

libapache2-mod-php8.2
php
php-common
php-pear
php8.2
php8.2-apcu
php8.2-bcmath
php8.2-bz2
php8.2-cgi
php8.2-cli
php8.2-common
php8.2-curl
php8.2-dev
php8.2-gd
php8.2-igbinary
php8.2-imagick
php8.2-imap
php8.2-intl
php8.2-ldap
php8.2-mbstring
php8.2-mcrypt
php8.2-memcache
php8.2-memcached
php8.2-msgpack
php8.2-mysql
php8.2-opcache
php8.2-readline
php8.2-redis
php8.2-soap
php8.2-xml
php8.2-zip
pkg-php-tools

• Install Composer2 and make it globally available for the system (placing it in /usr/local/bin and making it executable, for example).

• Configure the PHP.ini timezone correctly (within /etc/).

• Configure the system-level timezone correctly with the command (for example) :

  timedatectl set-timezone Europe/Rome

• Install the speedtest CLI from the URL speedtest-cli and make it globally available (e.g., in /usr/bin).

Installing SpeedTestTracker from GitHub

• Download the SpeedTestTracker source code archive (.tar.gz) from the GitHub page and decompress it into a selected directory (for this guide, /opt/speedtest-tracker will be used as the destination folder).

• Run the composer install inside that directory :

  composer2 install --no-interaction --prefer-dist --optimize-autoloader --no-dev --no-cache

• If the composer compilation is successful, use the .env.example file as a base for the final .env file, modifying the parameters for the DB connection and any other useful parameters accordingly.

  Example of .env file (referencing the .env.testing file):

APP_NAME="Speedtest Tracker"
APP_ENV=local
SPEEDTEST_CLI_PATH="/usr/bin/speedtest"

# Application key
# Used for encryption where needed, if a key is not generated run: php artisan key:generate
APP_KEY=base64:<app-key-generated>

APP_DEBUG=false
APP_LOG=daily
APP_URL=http://127.0.0.1

LOG_CHANNEL=stack
LOG_DEPRECATIONS_CHANNEL=null
LOG_LEVEL=debug

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=speedtracker
DB_USERNAME=speedtracker
DB_PASSWORD=XXXXXXXXX

BROADCAST_DRIVER=log
CACHE_DRIVER=database
FILESYSTEM_DISK=local
QUEUE_CONNECTION=database
SESSION_DRIVER=database
SESSION_LIFETIME=120

MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="Speedtest Tracker"

TELEGRAM_BOT_TOKEN=null

• From within the installation directory, run the command :
  php artisan app:install

• If everything goes well, to verify the dashboard display, launch the application with the command (in the next session we will look how to run it definitely) :

  php artisan serve --host 0.0.0.0

• Access the web interface from: http://IP-LXC-CONTAINER:8000

• The default login parameters, preset in the DB, are:

user : admin@example.com

pass : password

Updating SpeedTestTracker Version from GitHub

• Stop any running services for the dashboard (stop the TMux script or supervisor service).
• Rename the current directory (e.g., mv /opt/speedtest-tracker /opt/speedtest-tracker.bak).
• Download the archive with the new version of the source code (.tar.gz) of SpeedTestTracker from the GitHub page and decompress it into a selected directory (for this guide, /opt/speedtest-tracker will be used as the destination folder).
• Navigate to the /opt/speedtest-tracker directory and run the command:
  composer2 install --no-interaction --prefer-dist --optimize-autoloader --no-dev --no-cache
• copy in the actual program folder the .dev file from the renamed folder, where it was saved
• Before starting the application, adapt the DB to the new version with the command:
  php artisan migrate

Note : To make it work correctly, I had to launch the following commands in three shells:

php artisan serve --host 0.0.0.0
php artisan queue:work
php artisan schedule:work

Otherwise, it doesn't execute the ookla running tasks and scheduled tasks.

Grafana Integration

Note : this section need to be extended and tested

The tool also integrates with InfluxDBv2.0, so we can use Grafana for time series visualization.

Setting up a new influx empty db and point the config file of the project

Two pre-compiled dashboards are available here:

https://grafana.com/grafana/dashboards/17460-speedtest-tracker-v2/
https://gist.github.com/PovilasID/113c5d2ef6fc9fbddcd19864466399a6

Startup Modes

Using a Bash Script in a Tmux Session

To correctly start everything at server boot, you can use a supervisord configuration or a bash script that manages tmux for launching processes, Now we will view a bash script that use tmux shell to run correctly the processes:

start-speedtest.sh

#!/bin/bash

mysess="speedtest"

# Start tmux
tmux new-session -d -s $mysess

# Add windows and execute commands
tmux send-keys -t $mysess:0 'cd /opt/speedtest-tracker

; php artisan serve --host 0.0.0.0' C-m

# Split the first window into two vertical panes
tmux split-window -v -t $mysess:0
sleep 1

tmux send-keys -t $mysess:0.1 'cd /opt/speedtest-tracker; php artisan queue:work' C-m

# Split the second window into two panes
tmux split-window -h -t $mysess:0.1
sleep 1

tmux send-keys -t $mysess:0.2 'cd /opt/speedtest-tracker; php artisan schedule:work' C-m

# Enter the first window
#tmux select-pane -t $mysess:0

# Activate the monitor to view the output
tmux attach-session -t $mysess

Insert the startup into the root's cron:

@reboot /bin/bash /opt/start-speedtest.sh

Using a Configuration File for Supervisor

Another startup mode is to use a supervisor configuration, via a configuration file similar to the following example.

The file is a modified version of the file: /opt/speedtest-tracker/docker/8.2/supervisord.conf

Create the file: /etc/supervisor/conf.d/speedtracker-worker.conf

and put the following configuration inside :

[supervisord]
nodaemon=true
user=root
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid

[program:speedtest_tracker_serve]
command=/usr/bin/php /opt/speedtest-tracker/artisan serve --host=0.0.0.0 --port=8000
directory=/opt/speedtest-tracker
autostart=true
autorestart=true
stderr_logfile=/var/log/speedtest_tracker_serve.err.log
stdout_logfile=/var/log/speedtest_tracker_serve.out.log

[program:speedtest_tracker_queue_work]
command=/usr/bin/php /opt/speedtest-tracker/artisan queue:work
directory=/opt/speedtest-tracker
autostart=true
autorestart=true
stderr_logfile=/var/log/speedtest_tracker_queue_work.err.log
stdout_logfile=/var/log/speedtest_tracker_queue_work.out.log

[program:speedtest_tracker_schedule_work]
command=/usr/bin/php /opt/speedtest-tracker/artisan schedule:work
directory=/opt/speedtest-tracker
autostart=true
autorestart=true
stderr_logfile=/var/log/speedtest_tracker_schedule_work.err.log

then restart supervisord service (or user supervisorctl commands to reload configurations).

[manelrodero]

I came here looking for how to install speedtest-tracker on an LXC but I'm not convinced by carrying out so many manual steps (especially in the case of reinstallation).

I have it running in Docker within LXC (https://www.manelrodero.com/blog/docker-en-proxmox-lxc-con-turnkey-core) and it works correctly but I am not convinced by the test results. Speeds fluctuate a lot even when using the same server.

On the other hand, the same Docker running on a Raspberry Pi 4 obtains very similar values ​​for each test and very close to the speed of my connection. The graphs are very linear.

Right now I am looking for information about the network in Proxmox: network card parameters, its optimization for LXCs, etc.

To rule out Proxmox, the cable and the router port, I just installed speedtest-cli on Proxmox and scheduled the same tests with a cron. If I get linear results, the problem will be in the network of an LXC (or a Docker in LXC).

Have you observed the same behavior?

[svenvg93]

Hi @manelrodero
Looking at the results; We have seen this in the past that speedtest on the hour mark are lower than at a random moment during the hour. Seems that more and more people are testing their speedtest with tools like this at the hour mark. Which makes the Speedtest Server (or the ISP network) to run out of capacity. As you can see your test 5min after the hour are more stable and on the hour mark.

[manelrodero]

You're right ;-)

When I read your message I changed the time of the test and they are much more stable and similar to my contracted speed.

Now I'm trying to figure out why the upload speed from the Proxmox host or from the LXC is ~100Mbps lower than that obtained from the Raspberry Pi 4.

But hey, it's a minor "problem." Now at least I have a line graph so I can easily detect when there is a problem with the ISP.

Thank you very much for the time hint ;--)

[titof2375]

Hello, I have a problem installing mariadb.

[BramSuurdje]

the package is called mariadb-server