pkg(8) infinite loop with multiple repos containing different versions of pkg #1303

Closed
infracaninophile opened this Issue Aug 12, 2015 · 15 comments

Comments

Projects
None yet
10 participants
@infracaninophile
Member

infracaninophile commented Aug 12, 2015

Hmmm..... at the point I ran this, the desktop repo still had pkg-1.5.5 (hadn't finished building) whereas the server repo had pkg-1.5.6

ox-dell39:~:% sudo pkg upgrade 
Updating desktop repository catalogue...
Fetching meta.txz: 100%    260 B   0.3kB/s    00:01    
Fetching packagesite.txz: 100%  158 KiB 161.4kB/s    00:01    
Processing entries: 100%
desktop repository update completed. 716 packages processed.
Updating server repository catalogue...
Fetching meta.txz: 100%    260 B   0.3kB/s    00:01    
Fetching packagesite.txz: 100%  370 KiB 378.6kB/s    00:01    
Processing entries: 100%
server repository update completed. 1611 packages processed.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating desktop repository catalogue...
desktop repository is up-to-date.
Updating server repository catalogue...
server repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating desktop repository catalogue...
desktop repository is up-to-date.
Updating server repository catalogue...
server repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating desktop repository catalogue...
desktop repository is up-to-date.
Updating server repository catalogue...
server repository is up-to-date.
All repositories are up-to-date.
... etc. ad infinitum

Workaround was simply pkg install -r server pkg and then continue with pkg upgrade.

@dbarr

This comment has been minimized.

Show comment
Hide comment
@dbarr

dbarr Aug 14, 2015

Also experienced this, your workaround worked perfectly.

dbarr commented Aug 14, 2015

Also experienced this, your workaround worked perfectly.

@dlangille

This comment has been minimized.

Show comment
Hide comment
@dlangille

dlangille Aug 18, 2015

Worked for me today. Thanks.

Worked for me today. Thanks.

@problame

This comment has been minimized.

Show comment
Hide comment
@problame

problame Aug 19, 2015

I think I first encountered this bug around 1.5.4. Any pointers to the piece of code causing this issue?

I think I first encountered this bug around 1.5.4. Any pointers to the piece of code causing this issue?

@tomster

This comment has been minimized.

Show comment
Hide comment
@tomster

tomster Oct 30, 2015

not sure if i'm just adding noise, but FTR i have encountered this issue today with 1.6.1_1, so it's definitely not resolved yet.

and yes, the workaround still works, too, thanks!

tomster commented Oct 30, 2015

not sure if i'm just adding noise, but FTR i have encountered this issue today with 1.6.1_1, so it's definitely not resolved yet.

and yes, the workaround still works, too, thanks!

@dch

This comment has been minimized.

Show comment
Hide comment
@dch

dch Nov 4, 2015

ditto. However given there's no bugzilla report should I add one for this, and refer the workaround + github issue to make it a bit more discoverable?

dch commented Nov 4, 2015

ditto. However given there's no bugzilla report should I add one for this, and refer the workaround + github issue to make it a bit more discoverable?

@andrejzverev

This comment has been minimized.

Show comment
Hide comment
@andrejzverev

andrejzverev Dec 22, 2015

Contributor

any news about this? I see even someone provided patch..

Contributor

andrejzverev commented Dec 22, 2015

any news about this? I see even someone provided patch..

@dlangille

This comment has been minimized.

Show comment
Hide comment
@dlangille

dlangille Jan 16, 2016

andrejzverev: It appears to have been fixed on 2015.11.18, 26 days before your question.

andrejzverev: It appears to have been fixed on 2015.11.18, 26 days before your question.

@andrejzverev

This comment has been minimized.

Show comment
Hide comment
@andrejzverev

andrejzverev Jan 17, 2016

Contributor

@dlangille nope, one part of fix was committed on 24 Dec, yes. But loop still exist in case on multirepo.

Contributor

andrejzverev commented Jan 17, 2016

@dlangille nope, one part of fix was committed on 24 Dec, yes. But loop still exist in case on multirepo.

@dlangille

This comment has been minimized.

Show comment
Hide comment
@dlangille

dlangille Feb 14, 2016

This is happening to me now on 10.2-RELEASE-p9 with pkg 1.6.2

I have a snapshot of the system if anyone wishes it for debugging.

This is happening to me now on 10.2-RELEASE-p9 with pkg 1.6.2

I have a snapshot of the system if anyone wishes it for debugging.

@problame

This comment has been minimized.

Show comment
Hide comment
@problame

problame Apr 3, 2016

The bug just happened to me again. The infinite loop happens reproducibly on 1.6.2 when pkg upgrade to a repository with an updated version of pkg.

If it helps: when running pkg upgrade -y, pkg prompts to reinstall pkg repeatedly.

problame commented Apr 3, 2016

The bug just happened to me again. The infinite loop happens reproducibly on 1.6.2 when pkg upgrade to a repository with an updated version of pkg.

If it helps: when running pkg upgrade -y, pkg prompts to reinstall pkg repeatedly.

@bapt

This comment has been minimized.

Show comment
Hide comment
@bapt

bapt May 6, 2016

Member

should be totlaly fixed since pkg 1.7.2

Member

bapt commented May 6, 2016

should be totlaly fixed since pkg 1.7.2

@bapt bapt closed this May 6, 2016

@alxrem

This comment has been minimized.

Show comment
Hide comment
@alxrem

alxrem Jul 7, 2016

Bug is reproduced with pkg-1.8.6. FreeBSD repo and my local repo contains pkg-1.8.7. pkg upgrade causes an infinite loop, as shown in the first message.

# uname -a
FreeBSD freebsd1 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

# pkg info pkg
pkg-1.8.6
Name           : pkg
Version        : 1.8.6
Installed on   : Wed Jul  6 18:37:47 2016 MSK
Origin         : ports-mgmt/pkg
Architecture   : freebsd:10:x86:64
Prefix         : /usr/local
Categories     : ports-mgmt
Licenses       : BSD2CLAUSE
Maintainer     : pkg@FreeBSD.org
WWW            : http://wiki.freebsd.org/pkgng
Comment        : Package manager
Shared Libs provided:
    libpkg.so.3
Annotations    :
    repo_type      : binary
    repository     : FreeBSD
Flat size      : 10.2MiB
Description    :
Package management tool

WWW: http://wiki.freebsd.org/pkgng

# pkg search pkg
...
pkg-1.8.7                      Package manager
...
pkg-1.8.7                      Package manager
...

# pkg upgrade
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
Updating wallarm-node-test repository catalogue...
wallarm-node-test repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
Updating wallarm-node-test repository catalogue...
wallarm-node-test repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating FreeBSD repository catalogue...
...

alxrem commented Jul 7, 2016

Bug is reproduced with pkg-1.8.6. FreeBSD repo and my local repo contains pkg-1.8.7. pkg upgrade causes an infinite loop, as shown in the first message.

# uname -a
FreeBSD freebsd1 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

# pkg info pkg
pkg-1.8.6
Name           : pkg
Version        : 1.8.6
Installed on   : Wed Jul  6 18:37:47 2016 MSK
Origin         : ports-mgmt/pkg
Architecture   : freebsd:10:x86:64
Prefix         : /usr/local
Categories     : ports-mgmt
Licenses       : BSD2CLAUSE
Maintainer     : pkg@FreeBSD.org
WWW            : http://wiki.freebsd.org/pkgng
Comment        : Package manager
Shared Libs provided:
    libpkg.so.3
Annotations    :
    repo_type      : binary
    repository     : FreeBSD
Flat size      : 10.2MiB
Description    :
Package management tool

WWW: http://wiki.freebsd.org/pkgng

# pkg search pkg
...
pkg-1.8.7                      Package manager
...
pkg-1.8.7                      Package manager
...

# pkg upgrade
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
Updating wallarm-node-test repository catalogue...
wallarm-node-test repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
Updating wallarm-node-test repository catalogue...
wallarm-node-test repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating FreeBSD repository catalogue...
...
@amontalban

This comment has been minimized.

Show comment
Hide comment
@amontalban

amontalban Jul 12, 2016

As @alxrem said this is happening to me too, if you have pkg package on both repositories (Say official FreeBSD and custom Poudriere repository) when you do:

pkg upgrade

It will run on an infinite loop, the fix (In my case) was to specify the custom Poudriere repository:

pkg install -r Repository -y pkg and then pkg upgrade

Hope it helps someone!

As @alxrem said this is happening to me too, if you have pkg package on both repositories (Say official FreeBSD and custom Poudriere repository) when you do:

pkg upgrade

It will run on an infinite loop, the fix (In my case) was to specify the custom Poudriere repository:

pkg install -r Repository -y pkg and then pkg upgrade

Hope it helps someone!

@amontalban

This comment has been minimized.

Show comment
Hide comment
@amontalban

amontalban Dec 7, 2016

@bapt do you think this will get fixed?

I'm adding more info because we are stumbling with this every now and then (Generating tons of pkg processes and network activity when running in multiple machines/jails):

root@newmm:~ # pkg search pkg-1.9.3
pkg-1.9.3                      Package manager
pkg-1.9.3                      Package manager
pkg-1.9.3                      Package manager
pkg-1.9.3                      Package manager
root@newmm:~ # pkg info pkg
pkg-1.8.7_3
Name           : pkg
Version        : 1.8.7_3
Installed on   : Fri Sep 16 12:16:23 2016 UTC
Origin         : ports-mgmt/pkg
Architecture   : freebsd:10:x86:64
Prefix         : /usr/local
Categories     : ports-mgmt
Licenses       : BSD2CLAUSE
Maintainer     : pkg@FreeBSD.org
WWW            : http://wiki.freebsd.org/pkgng
Comment        : Package manager
Shared Libs provided:
       libpkg.so.3
Annotations    :
       repo_type      : binary
       repository     : FreeBSD
Flat size      : 10.2MiB
Description    :
Package management tool

WWW: http://wiki.freebsd.org/pkgng

Would love to provide more information/debug if needed.

Thanks!

@bapt do you think this will get fixed?

I'm adding more info because we are stumbling with this every now and then (Generating tons of pkg processes and network activity when running in multiple machines/jails):

root@newmm:~ # pkg search pkg-1.9.3
pkg-1.9.3                      Package manager
pkg-1.9.3                      Package manager
pkg-1.9.3                      Package manager
pkg-1.9.3                      Package manager
root@newmm:~ # pkg info pkg
pkg-1.8.7_3
Name           : pkg
Version        : 1.8.7_3
Installed on   : Fri Sep 16 12:16:23 2016 UTC
Origin         : ports-mgmt/pkg
Architecture   : freebsd:10:x86:64
Prefix         : /usr/local
Categories     : ports-mgmt
Licenses       : BSD2CLAUSE
Maintainer     : pkg@FreeBSD.org
WWW            : http://wiki.freebsd.org/pkgng
Comment        : Package manager
Shared Libs provided:
       libpkg.so.3
Annotations    :
       repo_type      : binary
       repository     : FreeBSD
Flat size      : 10.2MiB
Description    :
Package management tool

WWW: http://wiki.freebsd.org/pkgng

Would love to provide more information/debug if needed.

Thanks!

@amontalban

This comment has been minimized.

Show comment
Hide comment
@amontalban

amontalban Dec 7, 2016

Little more information, this is for every package we want to upgrade. If we don't use the --debug flag we get an infinite loop.

root@newmm:~ # pkg --debug upgrade pkg
DBG(1)[18363]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[18363]> PkgRepo: verifying update for FreeBSD
DBG(1)[18363]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
DBG(1)[18363]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/meta.txz with opts "i"
DBG(1)[18363]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/packagesite.txz with opts "i"
FreeBSD repository is up-to-date.
Updating company repository catalogue...
DBG(1)[18363]> PkgRepo: verifying update for company
DBG(1)[18363]> Pkgrepo, begin update of '/var/db/pkg/repo-company.sqlite'
DBG(1)[18363]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/meta.txz with opts "i"
DBG(1)[18363]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/packagesite.txz with opts "i"
company repository is up-to-date.
All repositories are up-to-date.
DBG(1)[18363]> want to get an advisory lock on a database
New version of pkg detected; it needs to be installed first.
DBG(1)[18363]> removing pkg from the request as it is the same as local
Checking integrity...DBG(1)[18363]> check integrity for 0 items added
 done (0 conflicting)
Your packages are up to date.
DBG(1)[18363]> release an advisory lock on a database
root@newmm:~ # pkg --debug upgrade bash
DBG(1)[18735]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[18735]> PkgRepo: verifying update for FreeBSD
DBG(1)[18735]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
DBG(1)[18735]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/meta.txz with opts "i"
DBG(1)[18735]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/packagesite.txz with opts "i"
FreeBSD repository is up-to-date.
Updating company repository catalogue...
DBG(1)[18735]> PkgRepo: verifying update for company
DBG(1)[18735]> Pkgrepo, begin update of '/var/db/pkg/repo-company.sqlite'
DBG(1)[18735]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/meta.txz with opts "i"
DBG(1)[18735]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/packagesite.txz with opts "i"
company repository is up-to-date.
All repositories are up-to-date.
DBG(1)[18735]> want to get an advisory lock on a database
New version of pkg detected; it needs to be installed first.
DBG(1)[18735]> removing pkg from the request as it is the same as local
Checking integrity...DBG(1)[18735]> check integrity for 0 items added
 done (0 conflicting)
Your packages are up to date.
DBG(1)[18735]> release an advisory lock on a database

Little more information, this is for every package we want to upgrade. If we don't use the --debug flag we get an infinite loop.

root@newmm:~ # pkg --debug upgrade pkg
DBG(1)[18363]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[18363]> PkgRepo: verifying update for FreeBSD
DBG(1)[18363]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
DBG(1)[18363]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/meta.txz with opts "i"
DBG(1)[18363]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/packagesite.txz with opts "i"
FreeBSD repository is up-to-date.
Updating company repository catalogue...
DBG(1)[18363]> PkgRepo: verifying update for company
DBG(1)[18363]> Pkgrepo, begin update of '/var/db/pkg/repo-company.sqlite'
DBG(1)[18363]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/meta.txz with opts "i"
DBG(1)[18363]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/packagesite.txz with opts "i"
company repository is up-to-date.
All repositories are up-to-date.
DBG(1)[18363]> want to get an advisory lock on a database
New version of pkg detected; it needs to be installed first.
DBG(1)[18363]> removing pkg from the request as it is the same as local
Checking integrity...DBG(1)[18363]> check integrity for 0 items added
 done (0 conflicting)
Your packages are up to date.
DBG(1)[18363]> release an advisory lock on a database
root@newmm:~ # pkg --debug upgrade bash
DBG(1)[18735]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[18735]> PkgRepo: verifying update for FreeBSD
DBG(1)[18735]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
DBG(1)[18735]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/meta.txz with opts "i"
DBG(1)[18735]> Fetch: fetching from: http://pkgmir.geo.freebsd.org/FreeBSD:10:amd64/latest/packagesite.txz with opts "i"
FreeBSD repository is up-to-date.
Updating company repository catalogue...
DBG(1)[18735]> PkgRepo: verifying update for company
DBG(1)[18735]> Pkgrepo, begin update of '/var/db/pkg/repo-company.sqlite'
DBG(1)[18735]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/meta.txz with opts "i"
DBG(1)[18735]> Fetch: fetching from: https://pkg.company.com/master/FreeBSD:10:amd64/packagesite.txz with opts "i"
company repository is up-to-date.
All repositories are up-to-date.
DBG(1)[18735]> want to get an advisory lock on a database
New version of pkg detected; it needs to be installed first.
DBG(1)[18735]> removing pkg from the request as it is the same as local
Checking integrity...DBG(1)[18735]> check integrity for 0 items added
 done (0 conflicting)
Your packages are up to date.
DBG(1)[18735]> release an advisory lock on a database
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment