Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yum_package: RPMdb read lock not released when there is wildcard in disablerepo defined in option #8515

Open
lytao opened this issue May 10, 2019 · 2 comments

Comments

Projects
None yet
2 participants
@lytao
Copy link

commented May 10, 2019

Description

In Chef 14, when we have such a yum_package resource to run:

yum_package 'mypackage' do
  options '--disablerepo=* --enablerepo=myrepo'
  action :upgrade
end

The wildcard is not processed well in yum_helper.py. However, the worst thing is, it will cause yum_helper.py exit not in a right way (not killed by any of SIGINT/SIGHUP/SIGPIPI/SIGQUIT) and no exception raised. So the exit_handler is not tiggered. That way, rpmdb got locked after running chef-client for several times.

Chef Version

14.12.9

Platform Version

CentOS 7.6

Client Output

Chef-client will not raise any exception, but once RPM db locked, the next chef-client run will raise:

error: rpmdb: BDB0113 Thread/process 186206/139995210921792 failed: BDB1507 Thread 
died in Berkeley DB library                                                        
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, 
run database recovery                                                              
error: cannot open Packages index using db5 -  (-30973)                            
error: cannot open Packages database in /var/lib/rpm                               
error: rpmdb: BDB0113 Thread/process 186206/139995210921792 failed: BDB1507 Thread 
died in Berkeley DB library                                                        
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, 
run database recovery                                                              
error: cannot open Packages database in /var/lib/rpm                               
@lytao

This comment has been minimized.

Copy link
Author

commented May 13, 2019

Today I got another case related to the same issue. When we have a broken yum repository defined in "/etc/yum.repos.d" (we can't guarantee all repository from internet are 100% healthy), we got same rpmdb lock issue.
Then I traced the processes, and found SIGINT were sent but the python process quit because of exception before it got SIGINT. There is no exceptions catched in method "query(command)" in lib/chef/provider/package/yum/yum_helper.py. That means, any backend yum failure of the calls can cause rpmdb locked:

  • get_base()
  • base.repos.listEnabled()
  • base.rpmdb
  • base.pkgSack

To make it simple, we'd better catch all exceptions in the code block in yum_helper.py (the bottom lines):

    if command['action'] == "whatinstalled":      
        query(command)                            
    elif command['action'] == "whatavailable":    
        query(command)                            
    elif command['action'] == "versioncompare":   
        versioncompare(command['versions'])       
    elif command['action'] == "installonlypkgs":  
         install_only_packages(command['package'])
    else:                                         
        raise RuntimeError("bad command")         

and release rpmdb lock, and then raise the error messages.

@lamont-granquist

This comment has been minimized.

Copy link
Contributor

commented May 13, 2019

Nice debugging, I'll take a look at both of these when I get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.