Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RHEL/7.2 setting repo_gpgcheck=1 causes yum update to fail #1596

Closed
openprivacy opened this issue Nov 29, 2016 · 26 comments
Closed

RHEL/7.2 setting repo_gpgcheck=1 causes yum update to fail #1596

openprivacy opened this issue Nov 29, 2016 · 26 comments
Labels
bugfix Fixes to reported bugs. enhancement General enhancements to the project.
Milestone
Backlog

Comments

@openprivacy
Copy link
Contributor

I believe this worked in RHEL/7.1 but now having repo_gpgcheck=1 in /etc/yum.conf is failing on our AWS instances with e.g.,
http://mirror.es.its.nyu.edu/epel/7/x86_64/repodata/repomd.xml.asc: [Errno 14] HTTP Error 404 - Not Found

This bug is reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1360939

My question is: is this a bug in RHEL or should the SSG no longer test for this value?
@shawndwells
Copy link
Member

Valid check. Reflects a bug to Red Hat corporate though.

repo_gpgcheck will evaluate the signatures of the yum repo metadata. Red Hat stopped publishing those, so the check will fail for redhat.com repos. It's a totally valid check though. Ensures those who create their own local yum repositories sign their content.

The BZ you cited calls for Red Hat to start publishing gpg content for repo metadata. BugZillas do not carry SLA's though -- customer support cases do. To help move Red Hat forward, signing repo data in the future, we need customer cases.

If this is important to you (broad community, not just @openprivacy ): Please open a Red Hat support case and send me the ticket number (shawn@redhat.com). I will administratively associate your ticket with the bugzilla -- effectively giving us means to document customer demand to have Red Hat start publishing signatures for repo data.

In the mean time, I'll make a patch to the XCCDF indicating this is a known issue with Red Hat repos.

@shawndwells shawndwells added the enhancement General enhancements to the project. label Nov 29, 2016
@openprivacy
Copy link
Contributor Author

https://access.redhat.com/support/cases/#/case/01752320

@shawndwells
Copy link
Member

shawndwells commented Dec 6, 2016 via email

@redhatrises redhatrises added the bugfix Fixes to reported bugs. label Feb 13, 2017
@shawndwells
Copy link
Member

update:

If you are interested in Red Hat providing gpg signed repo data then you need to follow the content delivery ticket

https://projects.engineering.redhat.com/browse/DELIVERY-2451

If you are interested in Satellite 6 functionality then you want to follow:

https://bugzilla.redhat.com/show_bug.cgi?id=1410638

for the capability to sign the repositories satellite serves.

@openprivacy
Copy link
Contributor Author

openprivacy commented Feb 27, 2017 via email

@jmnielsen7
Copy link

jmnielsen7 commented Jun 14, 2017 •
edited

@shawndwells I hope this is not too far out of scope, but I thought I'd try my luck here. After hardening a system with the latest RHEL 7 SSG datastream (1.33, last I checked) somehow by default now any repos we add expect to have the metadata signed (although 'man yum.conf' apparently indicates differently by saying that the default setting is repo_gpgcheck=0 if not explicitly set otherwise - so I'm not sure what is overriding that). Since for third party repos that we add it will complain about the absence of repomd.xml.asc on each of them now, we've had to resort to setting gpgcheck=1 but repo_gpgcheck=0 on our repos for the time being to install packages (but worry this will produce a STIG non-compliance finding).

A concrete example is for Jenkins. Their repo usage instructions for RHEL are very simple (3 steps):

sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum install jenkins

Source: https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Red+Hat+distributions

Yet, that key seems to only be used for validating the RPMs themselves (gpgcheck) and not the repo metadata (repo_gpgpcheck), so we have to set the latter to 0 to avoid the repomd.xml.asc not found error. Taken on analogy to your above response about the situation with official Redhat repos it sounds like the repo maintainer has to publish a specific GPG key for validating the metadata. To be clear, would that be published as a separate (second) key that has to be imported and not a reuse of the GPG key used for validating the RPMs?

Having RHEL check for signed metadata is new to me, so I'm not really sure if (taking my specific example) whether that one GPG key (jenkins-ci.org.key) should have been enough to validate both the RPMs and the metadata or if it is typical to publish a second GPG key just for metadata. Any insights there?

@redhatrises
Copy link
Contributor

somehow by default now any repos we add expect to have the metadata signed

That is correct. SSG is now setting repo_gpgcheck=1.

Having RHEL check for signed metadata is new to me, so I'm not really sure if (taking my specific example) whether that one GPG key (jenkins-ci.org.key) should have been enough to validate both the RPMs and the metadata or if it is typical to publish a second GPG key just for metadata. Any insights there?

Should only require the one gpg key. If you go here: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/, you can see that repomd.xml.asc does not exist. If repomd.xml.asc exists, the repo was signed with a GPG key much like an RPM is signed with a GPG key.

@xmtrcv
Copy link

xmtrcv commented Jun 15, 2017

We ran into the same issue, finding out when attempting to yum install from a repo created from the RHEL 7.3 DVD source tree. Let me look through my documentation and update with some examples and hopefully a potential path forward.

@jmnielsen7
Copy link

jmnielsen7 commented Jun 15, 2017 •
edited

@redhatrises

That is correct. SSG is now setting repo_gpgcheck=1.

Ah, I see it set in /etc/yum.conf now. I should have thought to look for a global setting there earlier.

Should only require the one gpg key. If you go here: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/, you can see that repomd.xml.asc does not exist. If repomd.xml.asc exists, the repo was signed with a GPG key much like an RPM is signed with a GPG key.

Okay, so basically regardless of whether the repo maintainer has signed their metadata/released repomd.xml.asc it would only require one key in either scenario. So most likely in my example with Jenkins since I cannot curl a repomd.xml.asc, and if I disable repo_gpgcheck on it it still validates signed RPMs (since I imported the key provided), then most likely Jenkins has just not signed their metadata. Boy, this is going to be fun...

@jan-cerny
Copy link
Collaborator

@redhatrises @shawndwells
I run into this issue again today when I run remediation script for DISA STIG profile on RHEL7.
Setting repo_gpgcheck to 1 breaks the the whole remediation for this profile, because then yum can't install the packages that the DISA STIG profile requires to be installed.

I think we should remove the remediation script for this rule. Or at least disable the remediation script temporarily, till Red Hat signes its repositories. Now it just breaks yum and doesn't improve security.
Moreover, after this "remediation" is done, you can't install security updates, so the system is left vulnerable. And from user's perspective, the error message and recommendation that yum throws is misleading, because you as an user can't figure out that "repo_gpgcheck=1" caused the problem.

@xmtrcv
Copy link

xmtrcv commented Jul 25, 2017

@jan-cerny I can completely sympathize. The system undergoing remediation requires a signed repo in order to install additional packages, however, there is no failure that is obviously indicated. In our case, it was attempting to install 'esc' as required by remediation and failing to do so. I have since added that package into the kickstart, but a number of systems were built and remediated without having the package installed and will need to be patched.

Initially, I was setting repo_gpgcheck=0 at the end of my remediation script (package installation failures would have already occurred), but moving forward, we decided to sign the repo used as source to build/patch systems (since we self-sign some other provided repos with custom built packages.)

@shawndwells
Copy link
Member

shawndwells commented Jul 25, 2017 via email

@jan-cerny
Copy link
Collaborator

@shawndwells We discussed this with @mpreisler . We suggested I will comment out the line of remediation that breaks the system and add a comment there or warning explaining why this is commented out and why it can't be remediated. I will provide a pull request.

@jan-cerny
Copy link
Collaborator

#2189

@shepdelacreme shepdelacreme mentioned this issue Aug 9, 2017
Closed
ragingpastry added a commit to ragingpastry/ansible-centos-7-security that referenced this issue Sep 26, 2017
@ragingpastry
Add variable to disable repo_gpgcheck
070f06b 
The DISA STIGs want to enforce repo_gpgcheck = 1 in /etc/yum.conf
This will not work on current RedHat due to a few reasons:
1. RedHat does not provide gpg signed repo data.
   There is a ticket opened id=1410638
2. Repo maintainers must use this on their repos. EPEL currently does not
3. Algorithms used to gpg sign yum repos break when FIPs is enabled
   This is fixed in 7.4

For more information see the following: ComplianceAsCode/content#1596
@jmnielsen7
Copy link

jmnielsen7 commented Oct 9, 2017 •
edited

@shawndwells

Thanks for the detailed update above. As you suggested, that bugzilla link is internal to Redhat Developers and I cannot view it even when logged in. Are there any recent remarks there indicating that this has been addressed?

From what I can tell RHEL 7.4 has been released now though I haven't attempted to update any of our systems to it. Do you know of any updates on the status of signed repos and the RHN satellite with 7.4?

Thanks in advance.

@shawndwells
Copy link
Member

shawndwells commented Oct 9, 2017 via email

@jmnielsen7
Copy link

@shawndwells Excellent. Thank you. So with this libgcrypt update though does that mean that Redhat has already signed all their (global/non-satellite) RHEL 7.4 package repos with the requisite .asc files to make this work and satisfy DISA STIG requirements, or is that still TBD?

I know you mentioned the issue with this not working on a host OS in previous releases when FIPS mode was enabled, but that only seems to affect the end user's OS in so far as they are signing their own custom repos, correct? Currently I'm just wondering if the official RHEL repos for 7.4 are themselves signed now.

@agilmore2
Copy link
Contributor

https://access.redhat.com/support/cases/#/case/01961866

Also commented on bugzilla.

Thanks for keeing track of this!

@openprivacy
Copy link
Contributor Author

openprivacy commented Oct 27, 2017 via email

@jmnielsen7
Copy link

@agilmore2 @openprivacy For some reason I can't see either of those cases when I click on the link (while logged in, of course). I get an ascii art "404" printed to the screen followed by the message: "Not Found. The page you are looking for is not here. It might have been moved, removed, or had its name and address changed. It might otherwise be temporarily unavailable for technical reasons."

@agilmore2
Copy link
Contributor

agilmore2 commented Oct 31, 2017 via email

@shawndwells
Copy link
Member

@agilmore2 appreciate the case link! I see it's been added as a customer request for that bugzilla (which is awesome, because there are tons of them now).

As indicated in the public bugzilla, patches were merged upstream a weekish ago:


This has been merged into upstream Pulp.  Relevant commits are:
https://github.com/pulp/pulp_rpm/commit/f73805f626d96596f9ae962ab6d787d2e001f02c
https://github.com/pulp/pulp_rpm/commit/5393773e361ed548b72696652f338b76908429d0
https://github.com/pulp/pulp_rpm/commit/b3e2dd8bd8c43abb4144cc9d7a121fec5e5f5899

With the exception of the documentation changes (which can be excluded), these commits apply cleanly to the Pulp v2.8.7.18 that is used by Satellite v6.2.12.

** AS JUST A PERSONAL OPINION ** it looks like this will be enabled in future satellite versions and the conversation is now about backporting to existing Satellite codebase. Definitely keep an eye on the bugzilla for updates.

@shawndwells
Copy link
Member

Oh snap. Just noticed the pm_ack flag was set on the bugzilla. Indicates product management has approved this to go into the next Satellite release.

Product Management and QE are two of many stakeholders.... still needs development approval (e.g. can the patches be merged successfully?). Safe to say this issue is progressing smoothly, but not far enough along to be a feature commitment yet. Keep the customer requests coming!

@matts-mpg matts-mpg mentioned this issue Nov 2, 2017
Closed
@PaulSD
Copy link

PaulSD commented Dec 23, 2017 •
edited

I have posted backported patches and instructions for enabling this on Satellite 6.2.12 here:
https://bugzilla.redhat.com/show_bug.cgi?id=1410638#c17

@kenyon kenyon mentioned this issue Apr 20, 2018
Closed
@openprivacy
Copy link
Contributor Author

https://access.redhat.com/support/cases/#/case/01752320 - slow... deserves to be backlogged...

From: Janorkar, Anuja on Sep 19 2018 at 05:52 PM -07:00

Unfortunately, we have not received the update on this. We will get back to you as soon as we get an update.
We appreciate your patience.

Best Regards,
Anuja J.
Global Support Services, Red Hat

@redhatrises redhatrises added this to the Backlog milestone Oct 1, 2018
@ghost ghost mentioned this issue Nov 2, 2018
Closed
@redhatrises
Copy link
Contributor

Closing this upstream ticket as this now is an issue with the products and not the content. Please keep all downstream tickets open to resolve this issue. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bugfix Fixes to reported bugs. enhancement General enhancements to the project.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
8 participants
@PaulSD @openprivacy @shawndwells @agilmore2 @redhatrises @jan-cerny @xmtrcv @jmnielsen7