Using cobbler for On-Premise Ubuntu Server Repo

[Kirandp6]

Version

Cobbler 2.8.5
source: ?, ?
build time: Tue Oct 15 01:59:43 2019

Question

I would like Using cobbler for On-Premise Ubuntu Server Repo , ie on-premise server will sync internet repo on a schedule & then it will act as a On-premise repo.

I have Centos 7 server on which cobbler is installed. Here is the error i am getting

#######################################

Commands
cobbler repo add --mirror=http://mirror.enzu.com/ubuntu/dists/bionic/ --name=bionic-all

cobbler reposync --only="bionic-all"

#######################################

Error is :
http://mirror.enzu.com/ubuntu/dists/bionic/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

#######################################
Details

task started: 2020-10-17_202120_reposync
task started (id=Reposync, time=Sat Oct 17 20:21:20 2020)
hello, reposync
run, reposync, run!
creating: /var/www/cobbler/repo_mirror/bionic-all/config.repo
creating: /var/www/cobbler/repo_mirror/bionic-all/.origin/bionic-all.repo
running: /usr/bin/reposync -l -n -d --config=/var/www/cobbler/repo_mirror/bionic-all/.origin/bionic-all.repo --repoid=bionic-all --download_path=/var/www/cobbler/repo_mirror
received on stdout:
received on stderr: Error setting up repositories: failure: repodata/repomd.xml from bionic-all: [Errno 256] No more mirrors to try.
http://mirror.enzu.com/ubuntu/dists/bionic/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 120, in run
self.sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 164, in sync
return self.yum_sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 432, in yum_sync
utils.die(self.logger,"cobbler reposync failed")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

reposync failed, tries left: 0
reposync failed, retry limit reached, skipping
running: chown -R root:apache /var/www/cobbler/repo_mirror/bionic-all
received on stdout:
received on stderr:
running: chmod -R 755 /var/www/cobbler/repo_mirror/bionic-all
received on stdout:
received on stderr:
Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 82, in run
rc = self._run(self)
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 243, in runner
3), name=name, nofail=nofail, logger=self.logger)
File "/usr/lib/python2.7/site-packages/cobbler/api.py", line 804, in reposync
return reposync.run(name)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 149, in run
utils.die(self.logger,"overall reposync failed, at least one repo failed to synchronize")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

!!! TASK FAILED !!!

#######################################

Any suggestion Please

Thanks

[Kirandp6]

Also tried below

cobbler repo add --mirror=http://mirror.enzu.com/ubuntu/dists/bionic/ --name=bionic-all-v1 --arch=x86_64 --breed=apt
cobbler reposync --only="bionic-all-v1"

task started: 2020-10-17_202908_reposync
task started (id=Reposync, time=Sat Oct 17 20:29:08 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=mirror.enzu.com --root=/ubuntu/dists/bionic/ --dist='' --section='' /var/www/cobbler/repo_mirror/bionic-all-v1 --nosource -a amd64
received on stdout:
received on stderr: Option dist requires an argument
Option section requires an argument
Usage: /usr/bin/debmirror [options]

For details, see man page.

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

..

[Kirandp6]

I did look at https://help.ubuntu.com/community/Debmirror ... but did not get clue how i can use this with cobbler ..

[hbokh]

Since debmirror complains about a missing argument, try this:

cobbler repo add --mirror=http://mirror.enzu.com/ubuntu/ --name=bionic-all-v1 --arch=x86_64 --breed=apt --apt-dists=bionic --apt-components=main

---

I am unable to reproduce this on Cobbler 3.1.2. It seems to be broken, so I can't do anything more than what I mentioned above. Cobbler error:

# cobbler reposync --only="bionic-all-v1"
task started: 2020-10-18_101741_reposync
task started (id=Reposync, time=Sun Oct 18 10:17:41 2020)
running python triggers from /var/lib/cobbler/triggers/task/reposync/pre/*
running shell triggers from /var/lib/cobbler/triggers/task/reposync/pre/*
shell triggers finished successfully
hello, reposync
run, reposync, run!
Exception occured: <class 'AttributeError'>
Exception value: 'NoneType' object has no attribute 'keys'
Exception Info:
  File "/usr/lib/python3.6/site-packages/cobbler/remote.py", line 84, in run
    rc = self._run(self)

  File "/usr/lib/python3.6/site-packages/cobbler/remote.py", line 248, in runner
    name=name, nofail=nofail, logger=self.logger)

  File "/usr/lib/python3.6/site-packages/cobbler/api.py", line 760, in reposync
    action_reposync.run(name)

  File "/usr/lib/python3.6/site-packages/cobbler/actions/reposync.py", line 141, in run
    for k in list(env.keys()):

!!! TASK FAILED !!!

Possible cause: reposync is Python2 (from package yum-utils-1.1.31-54.el7_8.noarch on CentOS 7.8.2003).

[Kirandp6]

@hbokh thanks for reply & help ! that seems working, but any idea on how i will get the keys imported ..

i did below

cobbler repo add --mirror=http://mirror.enzu.com/ubuntu/ --name=bionic-all-v3 --arch=x86_64 --breed=apt --apt-dists=bionic --apt-components=main

cobbler reposync --only="bionic-all-v3"

Now error is

task started: 2020-10-18_013253_reposync
task started (id=Reposync, time=Sun Oct 18 01:32:53 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=mirror.enzu.com --root=/ubuntu/ --dist=bionic --section=main /var/www/cobbler/repo_mirror/bionic-all-v3 --nosource -a amd64
received on stdout: [GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1524785920 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: keyblock resource `/var/lib/cobbler/.gnupg/trustedkeys.gpg': No such file or directory
gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32
gpgv: Can't check signature: No public key
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 01 1524785920 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: keyblock resource `/var/lib/cobbler/.gnupg/trustedkeys.gpg': No such file or directory
gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32
gpgv: Can't check signature: No public key
Errors:
 .temp/.tmp/dists/bionic/Release.gpg signature does not verify
 .temp/.tmp/dists/bionic/InRelease signature does not verify

received on stderr: rsync: failed to connect to mirror.enzu.com (23.89.0.110): No route to host (113)
rsync: failed to connect to mirror.enzu.com (2605:f700:40::1759:6e): Network is unreachable (101)
rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2]
Warning: failed to use rsync to download extra files.
Failed to download some Release, Release.gpg or InRelease files!
releasing 1 pending lock... at /usr/share/perl5/vendor_perl/LockFile/Simple.pm line 206.

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
  File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
    raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
  File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 120, in run
    self.sync(repo)
   File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 166, in sync
    return self.apt_sync(repo)
   File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 541, in apt_sync
    utils.die(self.logger,"cobbler reposync failed")
   File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
    raise CX(msg)

reposync failed, tries left: 0
reposync failed, retry limit reached, skipping
running: chown -R root:apache /var/www/cobbler/repo_mirror/bionic-all-v3
received on stdout:
received on stderr:
running: chmod -R 755 /var/www/cobbler/repo_mirror/bionic-all-v3
received on stdout:
received on stderr:
Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
  File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
    raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
  File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 82, in run
    rc = self._run(self)
   File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 243, in runner
    3), name=name, nofail=nofail, logger=self.logger)
   File "/usr/lib/python2.7/site-packages/cobbler/api.py", line 804, in reposync
    return reposync.run(name)
   File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 149, in run
    utils.die(self.logger,"overall reposync failed, at least one repo failed to synchronize")
   File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
    raise CX(msg)

!!! TASK FAILED !!!

[hbokh]

Not sure, but I found this: #1900 (comment)

And this as a reference: https://wiki.ordi49.fr/doku.php/en:tech:dev:debmirror

[Kirandp6]

thanks again .. i was also looking for below response in google.. seems that option is not there now or i do not how to use

ie passing flag --no-check-gpg.
#######################################

https://serverfault.com/questions/669809/sync-ubuntu-repository-with-cobbler-server-fails

You haven't added the keys to the gpg keychain. I am running a near identical configuration to yours (CentOS 6.6/trusty) and resolved this particular issue by passing the flag --no-check-gpg.

A more ideal solution would be to download and manually add the keys to your keyring.
#######################################

[dansou901]

Hi, try something like gpg --homedir /var/lib/cobbler --recv-keys 3B4FE6ACC0B21F32 prior to importing the repo.

[Kirandp6]

@dansou901 thanks i will try .. BTW i tried below but still error .. my cobbler box is CentOS 7 & i am trying this Ubuntu repo on it ...

===
wget http://mirror.pnl.gov/ubuntu//pool/main/u/ubuntu-keyring/ubuntu-keyring_2012.05.19_all.deb
dpkg -x ubuntu-keyring_2012.05.19_all.deb ~
gpg --keyring ~/usr/share/keyrings/ubuntu-archive-keyring.gpg --export|gpg --no-default-keyring --keyring /var/data/keyrings/ubuntu/trustedkeys.gpg --import

So ran above on a Ubunto box & copied it over ...

==

task started: 2020-10-18_020720_reposync
task started (id=Reposync, time=Sun Oct 18 02:07:20 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=mirror.enzu.com --root=/ubuntu/ --dist=bionic --section=main /var/www/cobbler/repo_mirror/bionic-all-v3 --nosource -a amd64
received on stdout: [GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1524785920 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: keyblock resource /var/lib/cobbler/.gnupg/trustedkeys.gpg': Not supported gpgv: Note: This version of GPG does not support the Keybox format gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32 gpgv: Can't check signature: No public key [GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 01 1524785920 9 [GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32 gpgv: keyblock resource /var/lib/cobbler/.gnupg/trustedkeys.gpg': Not supported
gpgv: Note: This version of GPG does not support the Keybox format
gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32
gpgv: Can't check signature: No public key
Errors:
.temp/.tmp/dists/bionic/Release.gpg signature does not verify
.temp/.tmp/dists/bionic/InRelease signature does not verify

[Kirandp6]

Still no luck

gpg --homedir /var/lib/cobbler --recv-keys 3B4FE6ACC0B21F32
gpg: WARNING: unsafe permissions on homedir /var/lib/cobbler' gpg: keyring /var/lib/cobbler/secring.gpg' created
gpg: keyring `/var/lib/cobbler/pubring.gpg' created
gpg: no keyserver known (use option --keyserver)
gpg: keyserver receive failed: Syntax error in URI

===

cobbler reposync --only="bionic-all-v3"
task started: 2020-10-18_021014_reposync
task started (id=Reposync, time=Sun Oct 18 02:10:14 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=mirror.enzu.com --root=/ubuntu/ --dist=bionic --section=main /var/www/cobbler/repo_mirror/bionic-all-v3 --nosource -a amd64
received on stdout: [GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1524785920 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: keyblock resource /var/lib/cobbler/.gnupg/trustedkeys.gpg': Not supported gpgv: Note: This version of GPG does not support the Keybox format gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32 gpgv: Can't check signature: No public key [GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 01 1524785920 9 [GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32 gpgv: keyblock resource /var/lib/cobbler/.gnupg/trustedkeys.gpg': Not supported
gpgv: Note: This version of GPG does not support the Keybox format
gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32
gpgv: Can't check signature: No public key

[Kirandp6]

Ubuntu notes i see like below
#######################################

If the maintainer does not tell you how to add the key then you need to find the "key hash" of the repository in order to look up the key on a public key server. Once you know the key hash, the key can be retrieved using the command:

gpg --keyserver [name of keyserver] --recv-keys [keyhash]
For example, if the key hash is CE49EC21, you retrieve the key using the command:

gpg --keyserver subkeys.pgp.net --recv-keys CE49EC21
Then, add the key to Ubuntu's apt trusted keys database using the command:

gpg --export --armor CE49EC21 | sudo apt-key add -
#######################################

[dansou901]

ah sorry, homedir has to be /var/lib/cobbler/.gnupg and it seems you have to use --keyserver and specify one, just search hkp keyservers

[Kirandp6]

gpg --keyserver hkp://keyserver.ubuntu.com:80 --homedir /var/lib/cobbler/.gnupg --recv-keys 3B4FE6ACC0B21F32
gpg: WARNING: unsafe permissions on homedir `/var/lib/cobbler/.gnupg'
gpg: external program calls are disabled due to unsafe options file permissions
gpg: keyserver communications error: General error
gpg: keyserver receive failed: General error

[Kirandp6]

seems my network firewall issue .. i did on other box & works , thanks @dansou901

gpg --keyserver hkp://keyserver.ubuntu.com:80 --homedir . --recv-keys 3B4FE6ACC0B21F32
gpg: WARNING: unsafe permissions on homedir '/root/test/test123'
gpg: keybox '/root/test/test123/pubring.kbx' created
gpg: key 3B4FE6ACC0B21F32: 21 signatures not checked due to missing keys
gpg: /root/test/test123/trustdb.gpg: trustdb created
gpg: key 3B4FE6ACC0B21F32: public key "Ubuntu Archive Automatic Signing Key (2012) ftpmaster@ubuntu.com" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1

i will see what i can do from here ..

[dansou901]

for the unsafe permissions do a chmod 700 /var/lib/cobbler/.gnupg

[dansou901]

and in the command before you forgot to set homedir to /var/lib/cobbler/.gnupg which is home to the keyring cobbler evaluates

[Kirandp6]

i copied the files created over to this CentOS box ... i see new error now

#######################################
files

/var/lib/cobbler/.gnupg : # ls
crls.d private-keys-v1.d pubring.gpg pubring.kbx pubring.kbx~ S.dirmngr secring.gpg S.gpg-agent S.gpg-agent.browser S.gpg-agent.extra S.gpg-agent.ssh trustdb.gpg trustedkeys.gpg

Also did

cp trustdb.gpg trustedkeys.gpg

as that file cobbler wants seems...

#######################################

cobbler reposync --only="bionic-all-v3"
task started: 2020-10-18_023525_reposync
task started (id=Reposync, time=Sun Oct 18 02:35:25 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=mirror.enzu.com --root=/ubuntu/ --dist=bionic --section=main /var/www/cobbler/repo_mirror/bionic-all-v3 --nosource -a amd64
received on stdout: [GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1524785920 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 01 1524785920 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Thu 26 Apr 2018 04:38:40 PM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
Errors:
.temp/.tmp/dists/bionic/Release.gpg signature does not verify
.temp/.tmp/dists/bionic/InRelease signature does not verify

received on stderr: rsync: failed to connect to mirror.enzu.com (23.89.0.110): No route to host (113)
rsync: failed to connect to mirror.enzu.com (2605:f700:40::1759:6e): Network is unreachable (101)
rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2]
Warning: failed to use rsync to download extra files.
Failed to download some Release, Release.gpg or InRelease files!
releasing 1 pending lock... at /usr/share/perl5/vendor_perl/LockFile/Simple.pm line 206.

[Kirandp6]

it could be purely my network firewall.. so i will have to work with firewall folks..

[Kirandp6]

i moved this vm to new subnet .. now error is diff

httpd does not appear to be running and proxying cobbler, or SELinux is in the way. Original traceback:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cobbler/cli.py", line 251, in check_setup
s.ping()
File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in call
return self.__send(self.__name, args)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
verbose=self.__verbose
File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
self.send_content(h, request_body)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
connection.endheaders(request_body)
File "/usr/lib64/python2.7/httplib.py", line 1052, in endheaders
self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 890, in _send_output
self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 852, in send
self.connect()
File "/usr/lib64/python2.7/httplib.py", line 833, in connect
self.timeout, self.source_address)
File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
raise err
error: [Errno 110] Connection timed out

started httpd .. but still the command seems hung

[Kirandp6]

selinux is disabled on the box already .. not sure if it needs python 3

[Kirandp6]

I will be back in 10 odd hrs .. thanks

[hbokh]

it could be purely my network firewall.. so i will have to work with firewall folks..

received on stderr: rsync: failed to connect to mirror.enzu.com (23.89.0.110): No route to host (113)
rsync: failed to connect to mirror.enzu.com (2605:f700:40::1759:6e): Network is unreachable (101)
rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2]

FYI that mirror is not running rsync at all, only http on port 80.

Try a mirror which does, for example the one in the Netherlands: http://nl.archive.ubuntu.com/ - just to see if it works.

Or use option --breed=apt.
From command cobbler repo add --help:

--breed=BREED Breed (valid options: rsync,rhn,yum,apt,wget)

[Kirandp6]

Thanks for reply for now i tried below

nc -zv nl.archive.ubuntu.com 80
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 213.136.12.213:80.
Ncat: 0 bytes sent, 0 bytes received in 0.45 seconds.

#######################################

similar error

cobbler repo add --mirror=http://nl.archive.ubuntu.com/ --name=bionic-all-v4 --arch=x86_64 --breed=apt --apt-components=main

^Chttpd does not appear to be running and proxying cobbler, or SELinux is in the way. Original traceback:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cobbler/cli.py", line 251, in check_setup
s.ping()
File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in call
return self.__send(self.__name, args)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
verbose=self.__verbose
File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
self.send_content(h, request_body)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
connection.endheaders(request_body)
File "/usr/lib64/python2.7/httplib.py", line 1052, in endheaders
self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 890, in _send_output
self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 852, in send
self.connect()
File "/usr/lib64/python2.7/httplib.py", line 833, in connect
self.timeout, self.source_address)
File "/usr/lib64/python2.7/socket.py", line 562, in create_connection
sock.connect(sa)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
KeyboardInterrupt

#######################################

seems no luck with

cobbler repo add --mirror=http://nl.archive.ubuntu.com/ --name=bionic-all-v4 --arch=x86_64 --breed=apt --apt-components=main --breed=rsync

:(

[dansou901]

How is your cobbler set up? Do you have httpd running in front of cobbler and properly configured? Please show us your httpd config files for cobbler.

[Kirandp6]

let me get back on that on how i did the setup of cobbler with http details

#######################################
in mean time i moved server back to old subnet with DNS working ( i moved to new subnet earlier as i was thinking it is network firewall issue ) , now the cmd is not hanging

both below point to

received on stderr: Option dist requires an argument
Usage: /usr/bin/debmirror [options]

#######################################

root@server : Sun Oct 18 : 14:29:51 : ~ : # cobbler repo add --mirror=http://nl.archive.ubuntu.com/ubuntu/dists/bionic/ --name=bionic-all-v7 --arch=x86_64 --breed=apt --apt-components=main
root@server : Sun Oct 18 : 14:30:09 : ~ : # cobbler reposync --only="bionic-all-v7"
task started: 2020-10-18_143013_reposync
task started (id=Reposync, time=Sun Oct 18 14:30:13 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=nl.archive.ubuntu.com --root=/ubuntu/dists/bionic/ --dist='' --section=main /var/www/cobbler/repo_mirror/bionic-all-v7 --nosource -a amd64
received on stdout:
received on stderr: Option dist requires an argument
Usage: /usr/bin/debmirror [options]

For details, see man page.

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 120, in run
self.sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 166, in sync
return self.apt_sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 541, in apt_sync
utils.die(self.logger,"cobbler reposync failed")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

reposync failed, tries left: 0
reposync failed, retry limit reached, skipping
running: chown -R root:apache /var/www/cobbler/repo_mirror/bionic-all-v7
received on stdout:
received on stderr:
running: chmod -R 755 /var/www/cobbler/repo_mirror/bionic-all-v7
received on stdout:
received on stderr:
Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 82, in run
rc = self._run(self)
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 243, in runner
3), name=name, nofail=nofail, logger=self.logger)
File "/usr/lib/python2.7/site-packages/cobbler/api.py", line 804, in reposync
return reposync.run(name)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 149, in run
utils.die(self.logger,"overall reposync failed, at least one repo failed to synchronize")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

!!! TASK FAILED !!!

#######################################

root@server : Sun Oct 18 : 14:30:15 : ~ : # cobbler repo add --mirror=http://mirror.enzu.com/ubuntu/dists/bionic/ --name=bionic-all-v8 --arch=x86_64 --breed=apt --apt-components=main
root@server : Sun Oct 18 : 14:31:01 : ~ : # cobbler reposync --only="bionic-all-v8"
task started: 2020-10-18_143106_reposync
task started (id=Reposync, time=Sun Oct 18 14:31:06 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=mirror.enzu.com --root=/ubuntu/dists/bionic/ --dist='' --section=main /var/www/cobbler/repo_mirror/bionic-all-v8 --nosource -a amd64
received on stdout:
received on stderr: Option dist requires an argument
Usage: /usr/bin/debmirror [options]

For details, see man page.

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 120, in run
self.sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 166, in sync
return self.apt_sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 541, in apt_sync
utils.die(self.logger,"cobbler reposync failed")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

reposync failed, tries left: 0
reposync failed, retry limit reached, skipping
running: chown -R root:apache /var/www/cobbler/repo_mirror/bionic-all-v8
received on stdout:
received on stderr:
running: chmod -R 755 /var/www/cobbler/repo_mirror/bionic-all-v8
received on stdout:
received on stderr:
Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'overall reposync failed, at least one repo failed to synchronize'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 82, in run
rc = self._run(self)
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 243, in runner
3), name=name, nofail=nofail, logger=self.logger)
File "/usr/lib/python2.7/site-packages/cobbler/api.py", line 804, in reposync
return reposync.run(name)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 149, in run
utils.die(self.logger,"overall reposync failed, at least one repo failed to synchronize")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

!!! TASK FAILED !!!
root@server : Sun Oct 18 : 14:31:08 : ~ : #

#######################################

[Kirandp6]

How is your cobbler set up? Do you have httpd running in front of cobbler and properly configured? Please show us your httpd config files for cobbler.

All i did is install cobbler with epel
then ran cobbler check & fix all what is needed , i think one of the thing that time it asked is httpd install
no change change made on httpd configs

i also just checked

https://ip/cobbler_web is not working

[Kirandp6]

i am planning to use it more of yum & apt repos

#######################################

cobbler repo list
bionic
bionic-all
bionic-all-v1
bionic-all-v2
bionic-all-v3
bionic-all-v6
bionic-all-v7
bionic-all-v8
xenial-main

#######################################
removed all

#######################################

then did

cobbler reposync
task started: 2020-10-18_145219_reposync
task started (id=Reposync, time=Sun Oct 18 14:52:19 2020)
hello, reposync
run, reposync, run!
*** TASK COMPLETE ***

#######################################

i remember post fresh install i ran same cmd

& it downloaded 30 odd gbs rpm files

[Kirandp6]

so i think, cobbler repo add , for ubuntu needs more options.. we missing something per error message

[Kirandp6]

tried

sudo cobbler repo add --name=local-trusty --breed=apt --arch=x86_64 --mirror=http://hu.archive.ubuntu.com/ubuntu/ --apt-components=main,restricted,universe,multiverse --apt-dists=trusty,trusty-updates,trusty-security

cobbler reposync

#######################################

task started: 2020-10-18_145729_reposync
task started (id=Reposync, time=Sun Oct 18 14:57:29 2020)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --method=http --host=hu.archive.ubuntu.com --root=/ubuntu/ --dist=trusty,trusty-updates,trusty-security --section=main,restricted,universe,multiverse /var/www/cobbler/repo_mirror/local-trusty --nosource -a amd64
received on stdout: [GNUPG:] ERRSIG 40976EAF437D05B5 17 10 00 1399558833 9
[GNUPG:] NO_PUBKEY 40976EAF437D05B5
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1399558833 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Thu 08 May 2014 07:20:33 AM PDT using DSA key ID 437D05B5
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
gpgv: Signature made Thu 08 May 2014 07:20:33 AM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
[GNUPG:] ERRSIG 40976EAF437D05B5 17 10 00 1590739693 9
[GNUPG:] NO_PUBKEY 40976EAF437D05B5
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1590739693 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Fri 29 May 2020 01:08:13 AM PDT using DSA key ID 437D05B5
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
gpgv: Signature made Fri 29 May 2020 01:08:13 AM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
[GNUPG:] ERRSIG 40976EAF437D05B5 17 10 01 1590739694 9
[GNUPG:] NO_PUBKEY 40976EAF437D05B5
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 01 1590739694 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Fri 29 May 2020 01:08:14 AM PDT using DSA key ID 437D05B5
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
gpgv: Signature made Fri 29 May 2020 01:08:14 AM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
[GNUPG:] ERRSIG 40976EAF437D05B5 17 10 00 1590739611 9
[GNUPG:] NO_PUBKEY 40976EAF437D05B5
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 00 1590739611 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Fri 29 May 2020 01:06:51 AM PDT using DSA key ID 437D05B5
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
gpgv: Signature made Fri 29 May 2020 01:06:51 AM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
[GNUPG:] ERRSIG 40976EAF437D05B5 17 10 01 1590739612 9
[GNUPG:] NO_PUBKEY 40976EAF437D05B5
[GNUPG:] ERRSIG 3B4FE6ACC0B21F32 1 10 01 1590739612 9
[GNUPG:] NO_PUBKEY 3B4FE6ACC0B21F32
gpgv: Signature made Fri 29 May 2020 01:06:52 AM PDT using DSA key ID 437D05B5
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
gpgv: Signature made Fri 29 May 2020 01:06:52 AM PDT using RSA key ID C0B21F32
gpgv: [don't know]: invalid packet (ctb=01)
gpgv: keydb_search failed: Invalid packet
gpgv: Can't check signature: No public key
Errors:
.temp/.tmp/dists/trusty/Release.gpg signature does not verify
.temp/.tmp/dists/trusty-updates/Release.gpg signature does not verify
.temp/.tmp/dists/trusty-updates/InRelease signature does not verify
.temp/.tmp/dists/trusty-security/Release.gpg signature does not verify
.temp/.tmp/dists/trusty-security/InRelease signature does not verify

received on stderr: rsync: failed to connect to hu.archive.ubuntu.com (45.67.159.244): Connection refused (111)
rsync: failed to connect to hu.archive.ubuntu.com (2a09:7ac0::4): Network is unreachable (101)
rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2]
Warning: failed to use rsync to download extra files.
Failed to download some Release, Release.gpg or InRelease files!
releasing 1 pending lock... at /usr/share/perl5/vendor_perl/LockFile/Simple.pm line 206.

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 120, in run
self.sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 166, in sync
return self.apt_sync(repo)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 541, in apt_sync
utils.die(self.logger,"cobbler reposync failed")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

reposync failed, tries left: 0
Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'reposync failed, retry limit reached, aborting'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 106, in die
raise CX(msg)

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'reposync failed, retry limit reached, aborting'
Exception Info:
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 82, in run
rc = self._run(self)
File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 246, in runner
name=None, nofail=nofail, logger=self.logger)
File "/usr/lib/python2.7/site-packages/cobbler/api.py", line 804, in reposync
return reposync.run(name)
File "/usr/lib/python2.7/site-packages/cobbler/action_reposync.py", line 142, in run
utils.die(self.logger,"reposync failed, retry limit reached, aborting")
File "/usr/lib/python2.7/site-packages/cobbler/utils.py", line 114, in die
raise CX(msg)

!!! TASK FAILED !!!

[Kirandp6]

I just below per @hbokh

cobbler repo add --mirror=http://mirror.enzu.com/ubuntu/ --name=bionic-all-v1 --arch=x86_64 --breed=apt --apt-dists=bionic --apt-components=main

then

cobbler reposync --only="bionic-all-v1"
task started: 2020-10-18_151547_reposync
task started (id=Reposync, time=Sun Oct 18 15:15:47 2020)
hello, reposync
run, reposync, run!
*** TASK COMPLETE ***

now running

reposync

showing no output so far ..

[Kirandp6]

reposync is done, not sure if it downloaded anything

what will be client side testing i can do next ... what to update in /etc/apt/sources.list

thanks

[Kirandp6]

so only change in recent time is i removed all old repos

[Kirandp6]

apparently it did not work & as i did so many things to get this working & lost track, i am going to start from scratch

closing this topic

Thanks for those who replied

[SchoolGuy]

@Kirandp6 Please let me know if this works when you try again. I want to make Cobbler work as it did back in the days. This project was dead for a long time but I am trying my best to fix it up again because I think it is quite unique. The only thing I am lacking to fix it up completely is time to code and hardware to test at home. Thus people like you are better then any money since you are using it in the real world!

[Kirandp6]

Sure re-opening

[Kirandp6]

@SchoolGuy thanks for the kind response .. i created new issue where i gave my feedback /issues - #2437 .. if we can discuss in that will help. i am open for a zoom call if that will make it any faster .. i am in PST but is flexible on time , thanks , closing this one