pacsync: Floating point exception

[AladW]

On i686, trying to sync a local repository with pacsync fails:

[archie@vbox aurutils]$ sudo pacsync custom
Floating point exception
[archie@vbox aurutils]$ sudo pacsync custom
error: could not sync db 'custom' (unable to lock database)

This happens regardless of the database present in /var/lib/pacman/sync, or what packages either database contains. i.e. I can reproduce it with an empty database:

repo-add custom.db.tar

strace output: https://gist.githubusercontent.com/anonymous/328229526a58ca2c9fcf23ad737c377d/raw
pacconf output: https://gist.githubusercontent.com/anonymous/c17f4309d15fced13ab44c2ca267be76/raw

[andrewgregory]

Are you running a recent build of pacman-git? There was a change to the download callback arguments that may be causing this: andrewgregory/pacman@58140db

[AladW]

The error was with pacman v5.0.1 / libalpm v10.0.1, but it's the same with pacman-git from HEAD

[andrewgregory]

I can't reproduce the error, but the only possible issue that I can see is in the download callback if curl is reporting a total download size of 0 for the file for some reason. Try building pacutils from the dload_fpe branch (with pacman v5.0.1) and let me know if that fixes it.

[AladW]

The dload_fpe branch works for me with pacman 5.0.1, though the pacsync output remains empty. FYI, same with pacman HEAD (though it doesn't build with -Werror that way)

[andrewgregory]

If pacsync is not printing any output at all, that sounds like it's not getting a valid xfer size which is even stranger. Does pacman's download output look correct when syncing custom? I've pushed a new commit to dload_fpe with some additional debugging output. Can you build with that and give me pacsync's output with --debug (still using pacman 5.0.1)?

[AladW]

Yeah, pacman output looks normal:

[archie@vbox ~]$ sudo pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 custom                        63.0 KiB  6.15M/s 00:00 [############################] 100%
:: Starting full system upgrade...
warning: linux-lts: ignoring package upgrade (4.4.19-1 => 4.4.21-1)
warning: linux-lts-headers: ignoring package upgrade (4.4.19-1 => 4.4.21-1)
warning: pacutils-git: local (0.3.0.15.g01a5473-1) is newer than custom (0.3.0.13.gc8913a7-1)
 there is nothing to do

pacsync --debug output: https://gist.github.com/anonymous/9de4b7612aae5dc5fbcbf0ef6709f385/raw

Without --debug it now seems the output is there, albeit partially (see last line):

[archie@vbox ~]$ sudo pacsync custom
syncing custom...
callback called with 'custom.db' 0 0
callback called with 'custom.db' 16383 0
callback called with 'custom.db' 32766 0
callback called with 'custom.db' 49149 0
callback called with 'custom.db' 64512 0
finished syncingom.db (64512)

[andrewgregory]

The xfer size looks fine, so I'm not sure why pacsync wouldn't have output something before. Though, for some reason the callback is never being called with the correct total size, so it won't ever print the final trailing newline and it might get overwritten by whatever writes to the terminal next. Can you give me pacsync's output without specifying any repos?

[AladW]

Sure:
pacsync: https://gist.github.com/anonymous/32b7e39b42ce11d94d90fd1c5fdb8f5f/raw
pacsync --debug: https://gist.github.com/anonymous/973d2f27b0ca795f0d815fc21beb9271/raw

[AladW]

Sorry, forgot to include the actual syncing of core e.a:
https://gist.github.com/anonymous/f89e6478b725a07655f0ddd539cd7d9a/raw
https://gist.github.com/anonymous/6c281cbc00dde56e3e8c2dfb57364eae/raw

[andrewgregory]

I am out of ideas that make any kind of sense. Can you install pacman-git from my dlcb_debug branch (https://github.com/andrewgregory/pacman/tree/dlcb_debug) and give me the full output of pacman -Syy and pacsync --force?

[AladW]

Ok, pacman -Syy: https://gist.githubusercontent.com/anonymous/ed33addf06e23d0339f359971864a3cb/raw
pacsync --force:
https://gist.githubusercontent.com/anonymous/6508f5f5b5a444646b5faf75f85631b3/raw

(Note that before compiling, I removed -Werror due to a -Wformat error in dload.c)

[andrewgregory]

The curl callback is clearly getting the correct values, and, even though I messed up the printf (hence the -Wformat error), it looks like should be passing the correct values to the front-end callback. The message that pacsync is actually printing, though, should only happen if one of the arguments is invalid. All I can think of is that there's something wrong with a type conversion somewhere that I'm missing. Does the dlcb_debug pacman build show download progress correctly (redirect stderr to /dev/null to prevent it from clobbering the progress bars)? Did the FPE start with the recent gcc or curl updates?

[andrewgregory]

Okay, I think I got it. If you rebuild the dload_fpe branch of pacutils, I think you should get the correct download progress with percentages.

[AladW]

That indeed fixes it, thanks!

One more thing, I had to disable make check, since it fails on 10-config-basic.t:

make -C t/ check
make[1]: Entering directory '/home/archie/.cache/aursync/pacutils-git/src/pacutils-git/t'
LD_LIBRARY_PATH=../lib prove 10-basename.t 10-config-basic.t 10-filelist_contains_path.t 10-pathcmp.t 10-strreplace.t 20-config-includes.t 20-config-root-inheritance.t 99-pu_list_shift.t
10-basename.t ................. ok   
10-config-basic.t ............. Bailout called.  Further testing stopped:  error initializing reader (No such file or directory)
FAILED--Further testing stopped: error initializing reader (No such file or directory)
make[1]: *** [Makefile:21: check] Error 1
make[1]: Leaving directory '/home/archie/.cache/aursync/pacutils-git/src/pacutils-git/t'
make: *** [Makefile:10: check] Error 2

https://gist.github.com/anonymous/9c95af5c8612e291cd1a208d688bfad5/raw

[andrewgregory]

A few of the tests use an ugly late binding hack to open in-memory files. I'm not sure why it's not working correctly for you, but I'm not terribly surprised that it's broken either. I've pushed some updated commits to dload_fpe; if you can verify that they build cleanly and work correctly I'll push them to master and take a look at the broken tests later.

[AladW]

It builds correctly, but it seems that newlines are not printed and there are some duplicate entries:

[archie@vbox pacutils-git]$ sudo pacsync --force
downloading core.db (0/0) 123031%downloading core.db (123031/0) 123031%downloading extra.db (0/0) 1798879%downloading extra.db (474903/0) 1798879%downloading extra.db (1163031/0) 1798879%downloading extra.db (1798879/0) 1798879%downloading community.db (0/0) 3777408%downloading community.db (704279/0) 3777408%downloading community.db (1113879/0) 3777408%downloading community.db (2014999/0) 3777408%downloading community.db (2424599/0) 3777408%downloading community.db (3112727/0) 3777408%downloading community.db (3653399/0) 3777408%downloading community.db (3777408/0) 3777408%downloading custom.db (0/0) 72192%downloading custom.db (72192/0) 72192%[archie@vbox pacutils-git]$ echo

[archie@vbox pacutils-git]$ sudo pacsync --force custom
downloading custom.db (0/0) 72192%downloading custom.db (72192/0) 72192%[archie@vbox pacutils-git]$ echo

[archie@vbox pacutils-git]$ 

[andrewgregory]

Try it now.

[AladW]

Yep, it works:

[archie@vbox pacutils-git]$ sudo pacsync --force custom
downloading custom.db (72192/72192) 100%
[archie@vbox pacutils-git]$ sudo pacsync --force
downloading core.db (123031/123031) 100%
downloading extra.db (1798879/1798879) 100%
downloading community.db (3777408/3777408) 100%
downloading custom.db (72192/72192) 100%

Thanks for taking the time to figure out this issue!

[andrewgregory]

Merged. Thanks for reporting.