Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

segfault when encountering WAL error in /var/lib/dnf/history.sqlite #1603

Closed
Safari77 opened this issue May 20, 2023 · 9 comments
Closed

segfault when encountering WAL error in /var/lib/dnf/history.sqlite #1603

Safari77 opened this issue May 20, 2023 · 9 comments
Assignees
@jan-kolarik
Labels
Triaged Someone on the DNF 5 team has read the issue and determined the next steps to take

Comments

@Safari77
Copy link

Looks like sqlite 3.42.0 triggers some error which causes segfault when it is being logged. Running on Fedora 37 which comes with sqlite 3.40.0 by default.
libdnf 0.70.1, rpm 4.18.1, dnf 4.15.1.

Running command "dnf upgrade".

Program received signal SIGSEGV, Segmentation fault.
__strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:76
76		VPCMPEQ	(%rdi), %ymm0, %ymm1
(gdb) bt
#0  __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:76
#1  0x00007ffff7867828 in __vfprintf_internal
    (s=s@entry=0x7fffffff9aa0, format=format@entry=0x7fffe8fc9293 "%s: %s: %s\n", ap=ap@entry=0x7fffffff9c50, mode_flags=mode_flags@entry=2) at /usr/src/debug/glibc-2.36-9.fc37.x86_64/stdio-common/vfprintf-process-arg.c:397
#2  0x00007ffff788835c in __vsnprintf_internal (string=0x7fffffff9b90 "", string@entry=0x0, maxlen=<optimized out>, 
    maxlen@entry=0, format=format@entry=0x7fffe8fc9293 "%s: %s: %s\n", args=args@entry=0x7fffffff9c50, mode_flags=mode_flags@entry=2) at vsnprintf.c:114
#3  0x00007ffff7922068 in ___vsnprintf_chk
    (s=s@entry=0x0, maxlen=maxlen@entry=0, flag=flag@entry=1, slen=slen@entry=18446744073709551615, format=format@entry=0x7fffe8fc9293 "%s: %s: %s\n", ap=ap@entry=0x7fffffff9c50) at vsnprintf_chk.c:34
#4  0x00007fffe8f3bf07 in vsnprintf (__ap=0x7fffffff9c50, __fmt=0x7fffe8fc9293 "%s: %s: %s\n", __n=0, __s=0x0)
    at /usr/include/bits/stdio2.h:68
#5  rpmlog (code=4, fmt=0x7fffe8fc9293 "%s: %s: %s\n") at /usr/src/debug/rpm-4.18.1-2.fc37.x86_64/rpmio/rpmlog.c:446
#6  0x00007fffe8de1cda in renderLogMsg (iErrCode=283, zFormat=<optimized out>, ap=ap@entry=0x7fffffff9e80)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:31531
#7  0x00007fffe8de1dc2 in sqlite3_log (iErrCode=<optimized out>, zFormat=<optimized out>)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:31542
#8  0x00007fffe8df187e in walIndexRecover (pWal=0x555555f609e8)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:64936
#9  walIndexReadHdr (pWal=pWal@entry=0x555555f609e8, pChanged=pChanged@entry=0x7fffffffa12c)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:422
#10 0x00007fffe8df1ef3 in walTryBeginRead
    (pWal=pWal@entry=0x555555f609e8, pChanged=pChanged@entry=0x7fffffffa12c, useWal=useWal@entry=0, cnt=cnt@entry=1)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:66260
#11 0x00007fffe8df2fa2 in sqlite3WalBeginReadTransaction (pChanged=0x7fffffffa12c, pWal=0x555555f609e8)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:66554
#12 pagerBeginReadTransaction (pPager=0x555555d39878) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:58955
#13 sqlite3PagerSharedLock (pPager=0x555555d39878) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:61124
#14 0x00007fffe8df8855 in lockBtree (pBt=0x5555563c1fb8)
at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:71967
#15 sqlite3BtreeBeginTrans (p=0x5555562a1a08, wrflag=0, pSchemaVersion=0x0)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:6823
#16 0x00007fffe8e4c631 in sqlite3InitOne
(db=0x555557837a88, iDb=iDb@entry=0, pzErrMsg=pzErrMsg@entry=0x7fffffffb008, mFlags=mFlags@entry=0)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:138178
#17 0x00007fffe8e4ccdc in sqlite3Init (db=db@entry=0x555557837a88, pzErrMsg=pzErrMsg@entry=0x7fffffffb008)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:138372
#18 0x00007fffe8e4cd1f in sqlite3ReadSchema (pParse=pParse@entry=0x7fffffffb000)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:138398
#19 0x00007fffe8e4d342 in sqlite3Pragma
    (pParse=0x7fffffffb000, pId1=<optimized out>, pId2=0x7fffffffa670, pValue=<optimized out>, minusFlag=<optimized out>) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:135486
#20 0x00007fffe8ed7c89 in yy_reduce.constprop.0
    (yypParser=0x7fffffffa610, yyruleno=<optimized out>, yyLookaheadToken=..., pParse=0x7fffffffb000, yyLookahead=<optimized out>) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:172382
#21 0x00007fffe8e80a2f in sqlite3Parser
    (yyminor=<error reading variable: DWARF-2 expression error: DW_OP_GNU_uninit must always be the very last op.>, yymajor=<optimized out>, yyp=0x7fffffffa610) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:173016
#22 sqlite3RunParser (pParse=<optimized out>, zSql=<optimized out>)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:43244
#23 0x00007fffe8e4be03 in sqlite3Prepare
    (db=db@entry=0x555557837a88, zSql=zSql@entry=0x7fffe95ab606 "PRAGMA journal_mode = WAL; PRAGMA foreign_keys = ON;", nBytes=nBytes@entry=-1, prepFlags=prepFlags@entry=128, pReprepare=pReprepare@entry=0x0, ppStmt=ppStmt@entry=0x7fffffffb2b8, pzTail=0x7fffffffb2c0) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:138700
#24 0x00007fffe8e4ce33 in sqlite3LockAndPrepare
    (pzTail=0x7fffffffb2c0, ppStmt=0x7fffffffb2b8, pOld=0x0, prepFlags=128, nBytes=-1, zSql=0x7fffe95ab606 "PRAGMA journal_mode = WAL; PRAGMA foreign_keys = ON;", db=0x555557837a88)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:138775
#25 sqlite3LockAndPrepare
    (db=0x555557837a88, zSql=0x7fffe95ab606 "PRAGMA journal_mode = WAL; PRAGMA foreign_keys = ON;", nBytes=-1, prepFlags=prepFlags@entry=128, pOld=pOld@entry=0x0, ppStmt=0x7fffffffb2b8, pzTail=0x7fffffffb2c0)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:7678
#26 0x00007fffe8e51eea in sqlite3_prepare_v2
    (db=<optimized out>, zSql=<optimized out>, nBytes=<optimized out>, ppStmt=<optimized out>, pzTail=<optimized out>)
    at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:138861
#27 0x00007fffe8e440d1 in sqlite3_exec
(db=0x555557837a88, zSql=0x7fffe95ab606 "PRAGMA journal_mode = WAL; PRAGMA foreign_keys = ON;", xCallback=0x0, pArg=0x0, pzErrMsg=0x0) at /usr/src/debug/sqlite-3.42.0-666.fc37.x86_64/sqlite3.c:132633
#28 0x00007fffe9524e31 in SQLite3::exec(char const*) (this=0x5555560b0bd0, sql=<optimized out>)
    at /usr/src/debug/libdnf-0.70.1-1.fc37.x86_64/libdnf/transaction/../transaction/../utils/sqlite3/Sqlite3.hpp:347
#29 0x00007fffe955cc54 in SQLite3::open() (this=0x5555560b0bd0)
    at /usr/src/debug/libdnf-0.70.1-1.fc37.x86_64/libdnf/utils/sqlite3/Sqlite3.cpp:43
#30 0x00007fffe9525ce5 in SQLite3::SQLite3(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (dbPath="/var/lib/dnf/history.sqlite", this=0x5555560b0bd0)
    at /usr/src/debug/libdnf-0.70.1-1.fc37.x86_64/libdnf/transaction/../utils/sqlite3/Sqlite3.hpp:334
#31 std::_Construct<SQLite3, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(SQLite3*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__p=0x5555560b0bd0)
    at /usr/include/c++/12/bits/stl_construct.h:119
#32 std::allocator_traits<std::allocator<void> >::construct<SQLite3, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::allocator<void>&, SQLite3*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__p=0x5555560b0bd0) at /usr/include/c++/12/bits/alloc_traits.h:635
#33 std::_Sp_counted_ptr_inplace<SQLite3, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__a=..., this=0x5555560b0bc0)
    at /usr/include/c++/12/bits/shared_ptr_base.h:604
@Safari77
Copy link
Author

If I do sqlite3 /var/lib/dnf/history.sqlite VACUUM\; , I can run dnf at least once without segfaulting.

@mcurlej mcurlej added the Triaged Someone on the DNF 5 team has read the issue and determined the next steps to take label May 22, 2023
@achilleas-k
Copy link

achilleas-k commented May 22, 2023 •
edited
Loading

I ran into this while running tests in osbuild-composer with up-to-date packages (with sqlite 3.42.0). Is this an issue in libdnf or sqlite? If it's in sqlite, is there a ticket I can follow for that?

@Safari77
Copy link
Author

I am not sure is this libdnf, sqlite, or rpm (see frame #5) bug, as I spent only a couple of minutes troubleshooting this.

@jan-kolarik
Copy link
Member

I don't see any official sqlite 3.42 package for Fedora. Is it something manually built? I cannot reproduce it using the latest versions from Fedora 38: libdnf 0.70.1 rpm 4.18.1 dnf 4.15.1 sqlite 3.40.1.

@Safari77
Copy link
Author

It's manually built. After I upgrade a package (history.sqlite is modified), segfaulting starts again.

@jan-kolarik
Copy link
Member

If it's working for you with the current official version of sqlite for Fedora, I would suggest to wait until the new version is released. It might be already fixed by then.

@Safari77
Copy link
Author 

# dnf upgrade
enabling rpmfusion-free-debuginfo repository
enabling rpmfusion-free-updates-debuginfo repository
Last metadata expiration check: 0:03:33 ago on Wed 24 May 2023 09:52:54 EEST.
warning: (null): notification message: recovered 213 frames from WAL file /var/lib/dnf/history.sqlite-wal
Dependencies resolved.
Nothing to do.
Complete!
# dnf upgrade
enabling rpmfusion-free-debuginfo repository
enabling rpmfusion-free-updates-debuginfo repository
Last metadata expiration check: 0:03:51 ago on Wed 24 May 2023 09:52:54 EEST.
Segmentation fault (core dumped)
#

@jan-kolarik
Copy link
Member

I guess if there has been already some operation involved using the manually built sqlite, the db could be broken. Could you check the dnf check output please?

@andyone
Copy link

andyone commented Sep 29, 2023

Related issue in rpm — rpm-software-management/rpm#2553

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jan-kolarik jan-kolarik

Labels
Triaged Someone on the DNF 5 team has read the issue and determined the next steps to take
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@andyone @achilleas-k @Safari77 @mcurlej @jan-kolarik