MVP#1 Application Server installation issues/observations

[dsilhavy]

This ticket is meant to collect issues and observations that occurred during the installation of the Application Server

Installation on MacOS Monterey Version 12.5.1

Missing module

Following the installation guide I ran into the following error when running 5gms-application-server -h:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/5gms-application-server", line 5, in <module>
    from rt_5gms_as.app import main
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/rt_5gms_as/app.py", line 32, in <module>
    from .context import Context
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/rt_5gms_as/context.py", line 34, in <module>
    from .openapi_5g.model.content_hosting_configuration import ContentHostingConfiguration
ModuleNotFoundError: No module named 'rt_5gms_as.openapi_5g'

Installation on Linux Ubuntu 20.04

Missing folders

Strictly following the installation guide I ran into a problem of missing folders. The following commands are required to create the folders:

mkdir /var/cache/rt-5gms
mkdir /var/log/rt-5gms
mkdir /run/rt-5gms

Nginx port bindings

Binding to port 80 is only possible with root privileges. Running 5gms-application-server external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json lead to a nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied) error.

This can be solved by following the instructions in the development documentation and creating an adjusted configuration using other ports:

[DEFAULT]
log_dir = /tmp/rt-5gms-as/logs
run_dir = /tmp/rt-5gms-as

### 5GMS Application Server specific configurations
[5gms_as]
cache_dir = /tmp/rt-5gms-as/cache
http_port = 8080
https_port = 8443

### 5GMS Application Server nginx specific configuration
[5gms_as.nginx]
root_temp = /tmp/rt-5gms-as

The new config can be provided via 5gms-application-server -c custom-application-server.conf external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json

Afterwards the sample MPD is available at http://localhost:8080/m4d/provisioning-session-d54a1fcc-d411-4e32-807b-2c60dbaeaf5f/BigBuckBunny_4s_onDemand_2014_05_09.mpd

[davidjwbbc]

The missing module is odd, it looks like the OpenAPI bindings were not created.

The build backend collects a dependencies list before it creates the distribution tar file. I suspect the build_scripts/backend.py may need to intercept the call to find the dependencies in the build backend and build the src/rt_5gms_as/openapi_5g directory before passing the call onto the setuptools backend. It already does this for the sdist command (to create the distribution tar), but maybe it needs to perform the action a little earlier in the build process.

I'll do some investigating and see if what I suspect is true.

[davidjwbbc]

The application server should create any missing folders it sees from its configuration. But obviously can't if it doesn't have permission to do so. The defaults are set assuming the script will be run as root for a proper system service. Using an alternative configuration pointing to directories that the user can create/modify allows this to be run as an unprivileged user.

[jordijoangimenez]

Issues related to the Documentation (when Running)

generate_openapi should be generate_5gms_as_openapi

"generate_openapi" should be renamed to "generate_5gms_as_openapi" in section "Running the example without building" of /docs/README.md
FIXED

config file moved from /docs to /external

../docs/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json
should be
../external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json
FIXED

[jordijoangimenez]

Issues when Running

'regex' not found

In the last step to run the example, I obtain a "ModuleNotFoundError: No module named 'regex'". It may be related to a problem finding the path but I'm not sure at the moment how to fix it
The issue is on the "import regex" inside proxies/nginx.py

fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server/src$ which pip
/usr/bin/pip
fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server/src$ which python3
/usr/bin/python3
fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server/src$ python3 -m pip install regex
Requirement already satisfied: regex in /home/fivegmag/.local/lib/python3.8/site-packages (2022.9.13)

[davidjwbbc]

In the last step to run the example, I obtain a "ModuleNotFoundError: No module named 'regex'". It may be related to a problem finding the path but I'm not sure at the moment how to fix it The issue is on the "import regex" inside proxies/nginx.py

fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server/src$ which pip
/usr/bin/pip
fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server/src$ which python3
/usr/bin/python3
fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server/src$ python3 -m pip install regex
Requirement already satisfied: regex in /home/fivegmag/.local/lib/python3.8/site-packages (2022.9.13)

I'm not sure why you are getting the import error, but judging from the output quoted above I have a couple of possible reasons:

1. You are running the application server as a user other than fivegmag. The regex package is only installed to that local user (as seen from the site-packages path above), so if you try to run as another user it won't find the extra installed package. If this is the case then running sudo python3 -m pip install regex, or installing your distribution package for the module, e.g. sudo apt -y install python3-regex, should fix the import problem as that will install the regex module system wide.
2. For some reason the version of python 3 being used for the application server is not 3.8, so the extra installed package is not being found.

[jordijoangimenez]

David, thank you for checking. Actually I could finally fix it with sudo apt -y install python3-regex.

Regarding the 3 vs 3.8 I checked that it doesn't seem to make a difference and it is all executed correctly from the 3.8 folder.

[rjb1000]

With the exception of the MacOS report, I think all problems have been solved, so now it's just a case of updating documentation, right, @davidjwbbc?

[rjb1000]

Anything more to be done on this issue @davidjwbbc?

Can we link it to documentation update PR(s) before closing?

[jordijoangimenez]

Hello, my issues have been resolved in the latest updates (all running on Ubuntu). For the issues raised by @dsilhavy, we may need to check with him, though I think the documentation regarding the ports was updated

[jordijoangimenez]

I'm trying to redo the rt-5gms-application-server from scratch to check the development branch. I'm getting "bad substitution" after Regenerating the 5G API bindings

~/rt-5gms-application-server/build_scripts/generate_5gms_as_openapi
/home/fivegmag/rt-5gms-application-server/build_scripts/generate_5gms_as_openapi: 33: Bad substitution

[jordijoangimenez]

I used "#! /bin/bash" instead of "#! /bin/sh" and it run

[jordijoangimenez]

I'm stuck with (apparently) the Nginx configurations.
On the sites-enabled or sites-available I can only see default. Default comes with port 80.
This seems to be always active, as when trying to launch the rt_5gms_as.app there is a conflict that says address already in use. I'm wondering when is any other configuration used to change ports.

If I manually change the port of the nginx default to 8080 and do systemctl restart nginx, then I can run rt_5gms_as.app
If I then launch the m3_client_cli.py I get:
INFO:rt-5gms-as:Adding content hosting configuration ps1...
INFO:rt-5gms-as:Reloading proxy daemon...
but no success! message. I'm also unable to retrieve the mpd nor the video from neither port 80 nor 8080

[jordijoangimenez]

After a while I got a success.

fivegmag@fivegmag-NUC8i7HVK:~/rt-5gms-application-server$ tests/m3_client_cli.py -H localhost:7777 add ps1 external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json
Success!

[davidjwbbc]

I'm stuck with (apparently) the Nginx configurations. On the sites-enabled or sites-available I can only see default. Default comes with port 80. This seems to be always active, as when trying to launch the rt_5gms_as.app there is a conflict that says address already in use. I'm wondering when is any other configuration used to change ports.

By default the 5GMS Application server will try to use port 80 for HTTP and port 443 for HTTPS. If you have the nginx service running, it too will use port 80 or 443 by default. Therefore one of these will need to move and personally, since the 5GMS Application Server is under test I'd change that one.

The 5GMS Application Server default ports can be can be easily changed by setting the appropriate values in the configuration file, for example:

[5gms_as]
http_port = 8080
https_port = 8443

These can either be set in the default configuration file (/etc/rt-5gms/application-server.conf or ~/.rt-5gms/application-server.conf) or you can put them in a separate file and use the -c command line option to use a different configuration file. Personally I have the following in local-dev.conf:

### Defaults for the 5G-MAG Reference Tools: 5GMS applications
[DEFAULT]
log_dir = /tmp/rt-5gms-as/logs
run_dir = /tmp/rt-5gms-as

### 5GMS Application Server specific configurations
[5gms_as]
log_level = debug
cache_dir = /tmp/rt-5gms-as/cache
certificates_cache = /tmp/rt-5gms-as/certificates
http_port = 8080
https_port = 8443
#access_log = %(log_dir)s/application-server-access.log
#error_log = %(log_dir)s/application-server-error.log
#pid_path = %(run_dir)s/application-server.pid

### 5GMS Application Server nginx specific configuration
[5gms_as.nginx]
#root_temp = /var/cache/rt-5gms/as
root_temp = /tmp/rt-5gms-as
#client_body_temp = %(root_temp)s/client-body-tmp
#proxy_temp = %(root_temp)s/proxy-tmp
#fastcgi_temp = %(root_temp)s/fastcgi-tmp
#uwsgi_temp = %(root_temp)s/uwsgi-tmp
#scgi_temp = %(root_temp)s/scgi-tmp
#pid_path = %(root_temp)s/5gms-as-nginx.pid

...and use venv/bin/5gms-application-server -c local-dev.conf to run the application server while testing. This redirects all logs and caches to /tmp and runs the application using ports 8080 and 8443.

If I manually change the port of the nginx default to 8080 and do systemctl restart nginx, then I can run rt_5gms_as.app If I then launch the m3_client_cli.py I get:

INFO:rt-5gms-as:Adding content hosting configuration ps1...
INFO:rt-5gms-as:Reloading proxy daemon...

but no success! message. I'm also unable to retrieve the mpd nor the video from neither port 80 nor 8080

That is strange, as though it got part way through deploying, but never replied to the M3 client. It's difficult to know what's going on there, I've not seen that before.

[jordijoangimenez]

Will check.
this is how my ./rt_5gms/application-server.conf looks like:

[DEFAULT]
log_dir = /tmp/rt-5gms-as/logs
run_dir = /tmp/rt-5gms-as

### 5GMS Application Server specific configurations
[5gms_as]
log_level = info
cache_dir = /tmp/rt-5gms-as/cache
certificates_cache = /tmp/rt-5gms-as/certificates
http_port = 8080
https_port = 8443
m3_listen = localhost
m3_port = 7777
m4d_path_prefix = /m4d/provisioning-session-{provisioningSessionId}/

### 5GMS Application Server nginx specific configuration
[5gms_as.nginx]
root_temp = /tmp/rt-5gms-as

I'm going to create the local-dev version

[jordijoangimenez]

I've checked the entire AS instructions. All seems to work subject to:

• using "#! /bin/bash" instead of "#! /bin/sh" at ~/rt-5gms-application-server/build_scripts/generate_5gms_as_openapi
• installing a few dependencies I've listed on my development branch: https://github.com/jordijoangimenez/rt-5gms-application-server/blob/development/docs/README.md
• and using -nodes insted of -noenc when using OpenSSL v1.1.1s
  I'd like to check all again.

[davidjwbbc]

I've got a fix pending for the first one of those.

The second is a documentation update, however most of those should automatically get installed when you do python3 -m pip install . as they're listed dependencies in the pyproject.toml file. The only prerequisites are up to date pip, setuptools and build modules, the rest should be automatic.

The -nodes option is an older (now deprecated) version of the -noenc option, I'll take a look at the script to make self signed certificates and see if I can make it pick the right one depending on your local installed version of openssl. This should be a relatively easy fix.

[jordijoangimenez]

David,
I think it is very good to have the dependencies in the install. In that case we should remove the installation of those from the documentation.
The -nodes comes because in Ubuntu 20.04 the OpenSSL version is 1.1.1f and I was able to update to 1.1.1s. Not to 3.0 though. What you propose, making this able to pick the right option, would be ideal.

[jordijoangimenez]

Hello, I can confirm that in Ubuntu 22.04.1 the OpenSSL is version 3.
I'm in the process of testing the AS with AF.

When building the AS I get the following:

jordijoan@jordijoan:~/rt-5gms-application-server/dist$ sudo python3 -m pip install rt-5gms-application-server-1.1.0.tar.gz
Processing ./rt-5gms-application-server-1.1.0.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: UNKNOWN
  Building wheel for UNKNOWN (pyproject.toml) ... done
  Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=5520 sha256=6b90aa35205d38b47af9a98f9da2e8f5ebfd954c1b0e1939c13f81014aba7f04
  Stored in directory: /root/.cache/pip/wheels/19/6c/6a/cebf80ee955ce3814f93d3ce407fbf4bf97fbaa24d80792249
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
  Attempting uninstall: UNKNOWN
    Found existing installation: UNKNOWN 0.0.0
    Uninstalling UNKNOWN-0.0.0:
      Successfully uninstalled UNKNOWN-0.0.0
Successfully installed UNKNOWN-0.0.0

Is that UNKNOWN something to worry about?

I'm unable to execute 5gms-application-server -h. I can only play with the AS using python3 -m rt_5gms_as.app from /src/

[jordijoangimenez]

This has been solved by:

• trying python3 -m pip install --upgrade pip build setuptools
• then delete src/rt_5gms_application_server.egg-info before doing the python3 -m pip install . (as it may contain bad package metadata from the previous build with the old setuptools)
• The installation may warn about the need to update the PATH:

WARNING: The script 5gms-application-server is installed in '/home/fivegmag/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

[jordijoangimenez]

I've opened a new issue (#57) related to the MVP#2 testing (which I'm doing with AS "development" and AF "1.0.1-rc"). The error I'm getting is happening always. I've just launched 5gms-application-server -c ../tests/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json and I obtain the same issue