uwsgi + nginx db backup does not work (RM #7580)

[dpage]

Issue migrated from Redmine: https://redmine.postgresql.org/issues/7580
Originally created by Jonny Rimkus at 2022-07-31 06:11:53 UTC.

Hi,
I'm running pgadmin 6.12 via uwsgi+nginx and configured it according to the Docs:
https://www.pgadmin.org/docs/pgadmin4/latest/server_deployment.html#nginx-configuration-with-uwsgi

my uwsgi version is 2.0.20 (installed via pip)

I've added the correct binary path, and the Validation shows:
pg_dump: (PostgreSQL) 13.7
pg_dumpall: (PostgreSQL) 13.7
pg_restore: (PostgreSQL) 13.7
psql: (PostgreSQL) 13.7

When I try to use these commands via e.g right click on a Schema->Backup, entering a Filename for the Backup nothing happens.
When I open the Filebrowser I see The file, but it is empty.

I enabled debug logging for pgadmin, and I see the following in the logs:

pgadmin4.log (FILE_LOG_LEVEL = 10 /DEBUG/)

2022-07-31 07:55:21,366: INFO   pgadmin:        Executing the process executor with the arguments: ['/path/to/uwsgi', '/path/to/pgadmin/lib/python3.9/site-packages/p
gadmin4/pgadmin/misc/bgprocess/process_executor.py', '/usr/bin/pg_dump', '--file', '/path/to/pgadmin/storage/folder/backup_file', '--host', 'localhost', '--port', '46787', '--use
rname', 'username', '--no-password', '--verbose', '--format=c', '--blobs', '--schema', 'public', 'schema_name']

uwsgi log

*** Starting uWSGI 2.0.20 (64bit) on [Sun Jul 31 07:53:43 2022] ***
compiled with version: 9.3.1 20200408 (Red Hat 9.3.1-2) on 02 June 2022 09:12:33
os: Linux-3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18 16:02:34 UTC 2022
nodename: server.domain.name
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 20
current working directory: /path/to/uwsgi/conf/dir/apps-enabled
detected binary path: /path/to/uwsgi/bin/uwsgi
chdir() to /path/to/pgadmin/lib/python3.9/site-packages/pgadmin4
your processes number limit is 103763
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /path/to/uwsgi/socket/pgadmin.uwsgi.sock fd 6
Python version: 3.9.13 (main, May 23 2022, 15:17:00)  [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)]
PEP 405 virtualenv detected: /path/to/pgadmin
Set PythonHome to /path/to/pgadmin
Python main interpreter initialized at 0x13bdbf0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 647728 bytes (632 KB) for 25 cores
*** Operational MODE: threaded ***
mounting pgAdmin4:app on /
WSGI app 0 (mountpoint='/') ready in 7 seconds on interpreter 0x13bdbf0 pid: 24486 (default app)
spawned uWSGI master process (pid: 24486)
spawned uWSGI worker 1 (pid: 24685, cores: 25)
announcing my loyalty to the Emperor...
*** has_emperor mode detected (fd: 9) ***
/path/to/bin/uwsgi: unrecognized option '--host'

I think the relevant line is /path/to/bin/uwsgi: unrecognized option '--host' from the uwsgi.log and seems identical to the error described here: https://www.reddit.com/r/PostgreSQL/comments/osb3t7/comment/hb5w2z0/?utm_source=share&utm_medium=web2x&context=3

What can I do to get the Backup functionality working with nginx+uwsgi

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-1
Originally created by Nikhil Mohite at 2022-08-05 11:13:05 UTC.

Hi,

From the logs, it looks like the python executable taken as '/path/to/uwsgi' which is incorrect. Can you please check PYTHON_HOME env var?

Redmine ticket header update:

Name	Old Value	New Value
Assigned To changed		Jonny Rimkus

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-2
Originally created by Jonny Rimkus at 2022-08-05 12:45:04 UTC.

Hi, I have no env var PYTHON_HOME.

pgadmin4 is running from a virtualenv by the way. And uwsgi was installed for my user via pip install

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-3
Originally created by Akshay Joshi at 2022-08-12 09:43:40 UTC.

Redmine ticket header update:

Name	Old Value	New Value
Sprint changed		EDB Sprint 124
Assigned To changed	Jonny Rimkus

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-4
Originally created by Yogesh Mahajan at 2022-08-30 07:36:36 UTC.

Redmine ticket header update:

Name	Old Value	New Value
Assigned To changed		Yogesh Mahajan
Status changed	New	In Progress

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-5
Originally created by Akshay Joshi at 2022-08-31 10:33:00 UTC.

Redmine ticket header update:

Name	Old Value	New Value
Sprint changed	EDB Sprint 124	EDB Sprint 125

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-6
Originally created by Yogesh Mahajan at 2022-09-02 07:43:34 UTC.

Applied in changeset commit:9a317f3ba6f68d1ede2a485e4218bf96fe377513.

Redmine ticket header update:

Name	Old Value	New Value
Done Ratio changed	0	90
Status changed	In Progress	In Testing

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-7
Originally created by Akshay Joshi at 2022-09-02 09:56:24 UTC.

Redmine ticket header update:

Name	Old Value	New Value
Fixed Version changed		6.14

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-8
Originally created by Jonny Rimkus at 2022-09-04 10:43:55 UTC.

Hi, very cool that you work an that issue 🙂. I was Looking at the changes made here: https://redmine.postgresql.org/projects/pgadmin4/repository/2/revisions/9a317f3ba6f68d1ede2a485e4218bf96fe377513/diff/web/pgadmin/misc/bgprocess/processes.py

And was thinking if it could be wrong, because the uwsgi app is installed globally and inside uwsgi installation folder there is no python app. Pgadmin4 is running in a virtualenv and uses a separate folder, you can see it in the uwsgi start log. Wouldn't it be easier to just return 'python' or the best solution somehow detect the python executable from within the virtualenv directory?

Hope it helps🙂
Kind regards
Jonny

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-9
Originally created by Jonny Rimkus at 2022-09-04 22:21:36 UTC.

I mean this log entries🙂:
PEP 405 virtualenv detected: /path/to/pgadmin
Set PythonHome to /path/to/pgadmin

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-10
Originally created by Fahar Abbas at 2022-09-12 06:38:11 UTC.

Redmine ticket header update:

Name	Old Value	New Value
Assigned To changed	Yogesh Mahajan	Fahar Abbas

[dpage]

Comment migrated from Redmine: https://redmine.postgresql.org/issues/7580#note-11
Originally created by Fahar Abbas at 2022-09-13 06:27:36 UTC.

This is resolved in the latest snapshot build:
https://www.postgresql.org/ftp/pgadmin/pgadmin4/snapshots/2022-09-12/yum/

Redmine ticket header update:

Name	Old Value	New Value
Done Ratio changed	90	100
Status changed	In Testing	Resolved

[dpage]

Issue closed on Redmine.

[ravermeister]

Hi,
unfortunately this issue is still not resolved.
As I had mentioned earlier the python path isn't correctly determined here:

pgadmin4/web/pgadmin/misc/bgprocess/processes.py

Line 269 in 5a5da5f

interpreter = interpreter.split('uwsgi')[0] + 'python'

The Scenario of this issue is as follows:

1. python is installed system wide

[raver7@finlay ~]$ which python
alias python='python3.9'
	/usr/bin/python3.9

2. The UWSGI installation was done via pip install --user uwsgi

[raver7@finlay ~]$ ls -l /home/raver7/.local/bin/
insgesamt 1260
-rwxrwxr-x. 1 raver7 raver7     223 22. Aug 10:51 pip
-rwxrwxr-x. 1 raver7 raver7     223 22. Aug 10:51 pip3
-rwxrwxr-x. 1 raver7 raver7     223 22. Aug 10:51 pip3.9
-rwxrwxr-x. 1 raver7 raver7 1276576 17. Jul 08:57 uwsgi

3. pgadmin3 was installed using a virtualenv in /home/raver7/share/pgadmin:

[raver7@finlay ~]$ source /home/raver7/share/pgadmin/bin/activate
(pgadmin) [raver7@finlay ~]$ echo $VIRTUAL_ENV
/home/raver7/share/pgadmin

4. So the correct Python path would be inside the virtualenv folder:

(pgadmin) [raver7@finlay ~]$ ls -l $VIRTUAL_ENV/bin
insgesamt 88
-rw-r--r--. 1 raver7 raver7 1904  2. Jun 12:02 activate
-rw-r--r--. 1 raver7 raver7  853  2. Jun 12:02 activate.csh
-rw-r--r--. 1 raver7 raver7 1993  2. Jun 12:02 activate.fish
-rw-r--r--. 1 raver7 raver7 8834  2. Jun 12:02 Activate.ps1
-rwxrwxr-x. 1 raver7 raver7  237 16. Sep 09:57 alembic
-rwxrwxr-x. 1 raver7 raver7  245 16. Sep 09:57 chardetect
-rwxrwxr-x. 1 raver7 raver7  238 16. Sep 09:57 email_validator
-rwxrwxr-x. 1 raver7 raver7  232 16. Sep 09:57 flask
-rwxrwxr-x. 1 raver7 raver7 1719 16. Sep 09:57 jp.py
-rwxrwxr-x. 1 raver7 raver7  237 16. Sep 09:57 mako-render
-rwxrwxr-x. 1 raver7 raver7  268 17. Jul 09:25 normalizer
-rwxrwxr-x. 1 raver7 raver7  240 16. Sep 09:58 pgadmin4
-rwxrwxr-x. 1 raver7 raver7  245 27. Aug 11:18 pip
-rwxrwxr-x. 1 raver7 raver7  245 27. Aug 11:18 pip3
-rwxrwxr-x. 1 raver7 raver7  245 27. Aug 11:18 pip3.9
-rwxrwxr-x. 1 raver7 raver7  246 16. Sep 09:57 pybabel
drwxrwxr-x. 2 raver7 raver7   31 16. Sep 09:57 __pycache__
lrwxrwxrwx. 1 raver7 raver7    9  2. Jun 12:02 python -> python3.9
lrwxrwxrwx. 1 raver7 raver7    9  2. Jun 12:02 python3 -> python3.9
lrwxrwxrwx. 1 raver7 raver7   18  2. Jun 12:02 python3.9 -> /usr/bin/python3.9
-rwxrwxr-x. 1 raver7 raver7  245 16. Sep 09:57 qr
-rwxrwxr-x. 1 raver7 raver7  240 16. Sep 09:57 sqlformat
-rwxrwxr-x. 1 raver7 raver7  242 16. Sep 09:57 sshtunnel
-rwxrwxr-x. 1 raver7 raver7  232  2. Jun 12:03 wheel

I've tested with the mentioned Version https://ftp.postgresql.org/pub/pgadmin/pgadmin4/snapshots/2022-09-12/pgadmin4-6.13-py3-none-any.whl and I get the Error which correctly says it couldn't find the python executable at the determined directory:

In my Opinion the best solution would be to check if pgadmin is running from within a virtualenv by checking if the System Environment Variable $VIRTUAL_ENV exists, and then for the existance of $VIRTUAL_ENV/bin/python