Bugreport - 2022.02 reposync ignores HTTP Proxy Settings

[jstelzle]

Problem description

The HTTP proxy settings in the Uyuni interface are ignored. The configured proxy server is not used. Instead, the download is attempted directly.

Version of Uyuni Server and Proxy (if used)

# zypper info Uyuni-Server-release
Loading repository data...
Reading installed packages...


Information for package Uyuni-Server-release:
---------------------------------------------
Repository     : Uyuni Server Stable
Name           : Uyuni-Server-release
Version        : 2022.02-171.1.uyuni1
Arch           : x86_64
Vendor         : obs://build.opensuse.org/systemsmanagement:Uyuni
Support Level  : Level 3
Installed Size : 1.4 KiB
Installed      : Yes
Status         : out-of-date (version 2022.01-165.2.uyuni1 installed)
Source package : Uyuni-Server-release-2022.02-171.1.uyuni1.src
Summary        : Uyuni Server
Description    :
    Uyuni lets you efficiently manage physical, virtual,
    and cloud-based Linux systems. It provides automated and cost-effective
    configuration and software management, asset management, and system
    provisioning.

Uyuni-Proxy is not used.

Details about the issue

First seen after Update from 2021.12 to 2022.01.

Example Log reposync called by Uyuni:

# cat /var/log/rhn/reposync/rpmfusion-unifi.log
...
2022/03/02 06:03:03 +02:00 Command: ['/usr/bin/spacewalk-repo-sync', '-p', 'centos7', '--type=yum']
2022/03/02 06:03:03 +02:00 Sync of channel started.
2022/03/02 06:05:17 +02:00
2022/03/02 06:05:17 +02:00   Importing comps file b3f0abafab4f566b1be092047b7960af44a88ba28cdb0137e59beb7c9031d08e-comps-el7.xml.
2022/03/02 06:05:17 +02:00   Renaming non-standard filename b3f0abafab4f566b1be092047b7960af44a88ba28cdb0137e59beb7c9031d08e-comps-el7.xml to comps.xml.
2022/03/02 06:05:17 +02:00 *** NOTE: Importing comps file for the channel 'rpmfusion-unifi'. Previous comps will be discarded.
2022/03/02 06:05:17 +02:00 Repo URL: http://download1.rpmfusion.org/nonfree/el/updates/7/x86_64/
2022/03/02 06:05:17 +02:00     Packages in repo:                94
2022/03/02 06:05:17 +02:00     Packages already synced:         93
2022/03/02 06:05:17 +02:00     Packages to sync:                 1
2022/03/02 06:05:17 +02:00     New packages to download:         1
2022/03/02 06:05:17 +02:00   Downloading packages:
2022/03/02 06:07:28 +02:00 ERROR: Download failed: http://download1.rpmfusion.org/nonfree/el/updates/7/x86_64/m/mock-rpmfusion-nonfree-36.0-1.el7.noarch.rpm - [Errno 14] curl#7 - "Failed to connect to download1.rpmfusion.org port 80: Connection timed out".
2022/03/02 06:07:28 +02:00     1/1 : mock-rpmfusion-nonfree-36.0-1.el7.noarch.rpm (failed)
2022/03/02 06:07:29 +02:00 Importing packages started.
2022/03/02 06:07:29 +02:00
2022/03/02 06:07:29 +02:00   Importing packages to DB:
2022/03/02 06:07:29 +02:00   Package batch #1 of 1 completed...
2022/03/02 06:07:29 +02:00 Importing packages finished.
2022/03/02 06:07:29 +02:00
2022/03/02 06:07:29 +02:00   Patches in repo: 0.
...

Calling "spacewalk-repo-sync" on the commandline with http_proxy and https_proxy set works as expected.

# echo $http_proxy
http://10.1.1.1:3124
# echo $https_proxy
http://10.1.1.1:3124

2022/03/02 16:34:25 +02:00 Command: ['/usr/bin/spacewalk-repo-sync', '-p', 'centos7', '--type=yum']
2022/03/02 16:34:25 +02:00 Sync of channel started.
2022/03/02 16:34:29 +02:00
2022/03/02 16:34:29 +02:00   Importing comps file b3f0abafab4f566b1be092047b7960af44a88ba28cdb0137e59beb7c9031d08e-comps-el7.xml.
2022/03/02 16:34:29 +02:00   Renaming non-standard filename b3f0abafab4f566b1be092047b7960af44a88ba28cdb0137e59beb7c9031d08e-comps-el7.xml to comps.xml.
2022/03/02 16:34:29 +02:00 *** NOTE: Importing comps file for the channel 'rpmfusion-unifi'. Previous comps will be discarded.
2022/03/02 16:34:29 +02:00 Repo URL: http://download1.rpmfusion.org/nonfree/el/updates/7/x86_64/
2022/03/02 16:34:29 +02:00     Packages in repo:                94
2022/03/02 16:34:29 +02:00     Packages already synced:         93
2022/03/02 16:34:29 +02:00     Packages to sync:                 1
2022/03/02 16:34:29 +02:00     New packages to download:         1
2022/03/02 16:34:29 +02:00   Downloading packages:
2022/03/02 16:34:30 +02:00     1/1 : mock-rpmfusion-nonfree-36.0-1.el7.noarch.rpm
2022/03/02 16:34:30 +02:00 Importing packages started.
2022/03/02 16:34:30 +02:00
2022/03/02 16:34:30 +02:00   Importing packages to DB:
2022/03/02 16:34:35 +02:00   Package batch #1 of 1 completed...
2022/03/02 16:34:35 +02:00 Importing packages finished.
2022/03/02 16:34:35 +02:00
2022/03/02 16:34:35 +02:00   Linking packages to the channel.
2022/03/02 16:34:35 +02:00     1 packages linked
2022/03/02 16:34:35 +02:00
2022/03/02 16:34:35 +02:00   Patches in repo: 0.
2022/03/02 16:34:35 +02:00 Sync completed.

Proxy settings in /etc/rhn/rhn.conf:

# grep proxy /etc/rhn/rhn.conf
# Use proxy FQDN, or FQDN:port
server.satellite.http_proxy = 10.1.1.1:3124
server.satellite.http_proxy_username =
server.satellite.http_proxy_password =
# no_proxy is a comma seperated list
server.satellite.no_proxy =

[joeyray0815]

Hi!
We have the same problem here. Proxy is set in UYUNI GUI and in /etc/sysconfig/proxy. Proxy works fine for other applications.

[vossi76]

Hi,
we can also confirm this problem....
The OS is set to use proxy setting and also UYUNI as described

[wombelix]

Hey guys,

took me a while but I think the Issue is related to the Upgrade from python3-urlgrabber < 4 to python3-urlgrabber >= 4. introduced in #4604

All params are from type byte, but scheme has to be str when passed to find_proxy, otherwise proxy will be set to None, relevant line:

uyuni/backend/satellite_tools/download.py

Line 117 in f4b4804

opts.find_proxy(url, scheme)

Example

Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from urlgrabber.grabber import URLGrabberOptions
>>> opts = URLGrabberOptions(proxy=None, proxies={'http': 'http://172.16.5.39:80', 'https': 'http://172.16.5.39:80', 'ftp': 'http://172.16.5.39:80'})
>>> opts.proxy
>>> 
>>> opts.proxies
{'http': 'http://172.16.5.39:80', 'https': 'http://172.16.5.39:80', 'ftp': 'http://172.16.5.39:80'}
>>> opts.find_proxy(b'http://test.example.com', b'http')
>>> opts.proxy
>>> 
>>> opts.find_proxy(b'http://test.example.com', b'http'.decode("utf-8"))
>>> opts.proxy
'http://172.16.5.39:80'
>>> opts.find_proxy(b'http://test.example.com', b'http')
>>> opts.proxy
>>> 

Changing

opts.find_proxy(url, scheme)

to

opts.find_proxy(url, scheme.decode("utf-8"))

in /usr/lib/python3.6/site-packages/spacewalk/satellite_tools/download.py, as quick & dirty workaround, did the trick in my Environment, opts.proxy now contains the proxy url instead None and a repo sync triggered by taskomatic is able to download packages again.

It looks like when a proxy is configured and enabled also in /etc/sysconfig/proxy and you run spacewalk-repo-sync from the command line, then curl, which is at the end triggered by urlgrabber, will silently fallback to these global proxy settings instead and just work.

I can only assume that the http_proxy environment variables from /etc/sysconfig/proxy are just not available when spacewalk-repo-sync get started by taskomatic via Runtime.exec(), therefore no fallback is possible and curl runs into the reported timeout.

uyuni/java/code/src/com/redhat/rhn/manager/satellite/SystemCommandThreadedExecutor.java

Line 119 in cdda7ea

Process p = r.exec(args);

Cheers
Dominik

[joeyray0815]

@wombelix Thanks for the tip! This did the trick for us! Have a great week!