Add dependency on pki-acme #5039

cipherboy · 2020-08-17T15:22:09Z

With the merging of #4723, pki-acme should be added as a dependency of
IPA. Note that this is only necessary on PKI >= 10.10 and shouldn't be
backported to RHEL 8.3 as the subpackage doesn't exist there.

Related: dogtagpki/pki#513

Signed-off-by: Alexander Scheel <ascheel@redhat.com>

With the merging of freeipa#4723, pki-acme should be added as a dependency of IPA. Note that this is only necessary on PKI >= 10.10 and shouldn't be backported to RHEL 8.3 as the subpackage doesn't exist there. Related: dogtagpki/pki#513 Signed-off-by: Alexander Scheel <ascheel@redhat.com>

abbra · 2020-08-17T15:35:39Z

Before merging this we need:

either put pki-core 10.10+ build to https://copr.fedorainfracloud.org/coprs/g/freeipa/freeipa-master/ or add a separate COPR repo with pki-core 10.10+ as discussed between FreeIPA and PKI developers earlier
add that COPR repo to Azure CI pipeline setup
add that COPR repo to PR CI (if it is @freeipa/freeipa-master COPR repo, it is already used by PR CI)

edewata · 2020-08-17T15:41:33Z

I think you should be able to add @pki/master as an external repo in @freeipa/freeipa-master.

abbra · 2020-08-17T16:24:03Z

I think you should be able to add @pki/master as an external repo in @freeipa/freeipa-master.

No, this would only work for packages built in COPR itself.

However, now COPR has support for runtime dependencies: https://fedora-copr.github.io/posts/runtime-dependencies. I added copr://@pki/master there. There is, however, a warning that @pki/master COPR repo does not provide following chroots: fedora-rawhide-aarch64, fedora-32-aarch64. This means we will not be able to test on aarch64. Could you please add aarch64 chroots to @pki/master?

abbra · 2020-08-17T16:41:53Z

It doesn't seem to help, though, because pki-core >= 10.9.0-0.4 cannot be found in http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/de66f5ec-e0a6-11ea-a8d2-fa163e19313f/runner.log.gz

2020-08-17 16:32:05,469    DEBUG  fatal: [master]: FAILED! => {"attempts": 3, "changed": false, "failures": [], "msg": "
Depsolve Error occured: 
 Problem 1: cannot install the best candidate for the job
  - nothing provides pki-acme >= 10.9.0-0.4 needed by freeipa-server-4.9.0.dev-0.fc32.x86_64
 Problem 2: package freeipa-server-trust-ad-4.9.0.dev-0.fc32.x86_64 requires freeipa-server = 4.9.0.dev-0.fc32, but none of the providers can be installed
  - cannot install the best candidate for the job
  - nothing provides pki-acme >= 10.9.0-0.4 needed by freeipa-server-4.9.0.dev-0.fc32.x86_64
 Problem 3: package freeipa-server-dns-4.9.0.dev-0.fc32.noarch requires freeipa-server = 4.9.0.dev-0.fc32, but none of the providers can be installed
  - cannot install the best candidate for the job
  - nothing provides pki-acme >= 10.9.0-0.4 needed by freeipa-server-4.9.0.dev-0.fc32.x86_64
 Problem 4: package freeipa-healthcheck-0.6-3.fc32.noarch requires freeipa-server, but none of the providers can be installed
  - package freeipa-server-4.8.7-1.fc32.x86_64 requires freeipa-client = 4.8.7-1.fc32, but none of the providers can be installed
  - package freeipa-server-4.8.6-1.fc32.x86_64 requires freeipa-client = 4.8.6-1.fc32, but none of the providers can be installed
  - package freeipa-client-4.8.7-1.fc32.x86_64 conflicts with ipa-admintools provided by freeipa-client-4.9.0.dev-0.fc32.x86_64
  - package freeipa-client-4.8.7-1.fc32.x86_64 conflicts with ipa-client provided by freeipa-client-4.9.0.dev-0.fc32.x86_64
  - package freeipa-client-4.9.0.dev-0.fc32.x86_64 conflicts with ipa-admintools provided by freeipa-client-4.8.7-1.fc32.x86_64
  - package freeipa-client-4.9.0.dev-0.fc32.x86_64 conflicts with ipa-client provided by freeipa-client-4.8.7-1.fc32.x86_64
  - cannot install both freeipa-client-4.9.0.dev-0.fc32.x86_64 and freeipa-client-4.8.7-1.fc32.x86_64
  - package freeipa-client-4.8.6-1.fc32.x86_64 conflicts with ipa-admintools provided by freeipa-client-4.9.0.dev-0.fc32.x86_64
  - package freeipa-client-4.8.6-1.fc32.x86_64 conflicts with ipa-client provided by freeipa-client-4.9.0.dev-0.fc32.x86_64
  - package freeipa-client-4.9.0.dev-0.fc32.x86_64 conflicts with ipa-admintools provided by freeipa-client-4.8.6-1.fc32.x86_64
  - package freeipa-client-4.9.0.dev-0.fc32.x86_64 conflicts with ipa-client provided by freeipa-client-4.8.6-1.fc32.x86_64
  - cannot install both freeipa-client-4.9.0.dev-0.fc32.x86_64 and freeipa-client-4.8.6-1.fc32.x86_64
  - cannot install the best candidate for the job
  - nothing provides pki-acme >= 10.9.0-0.4 needed by freeipa-server-4.9.0.dev-0.fc32.x86_64
", "rc": 1, "results": []}

@netoarmando @f-trivino could you please look at it, may be instead of enabling runtime dependency in COPR we should add

repo_pki_master_enabled: 1

to https://github.com/freeipa/freeipa-pr-ci/blob/master/ansible/vars/ipa_branches/master.yml ?

edewata · 2020-08-17T17:10:39Z

I added fedora-32-aarch64 and fedora-rawhide-aarch64 and tried to rebuild the packages, but it looks like there are some issues with f30 and rawhide right now:
https://copr.fedorainfracloud.org/coprs/g/pki/master/builds/

cipherboy · 2020-08-17T17:16:20Z

Rawhide is currently broken in COPR due to:

netoarmando · 2020-08-17T18:12:13Z

I've triggered a PR-CI run in the parallel infra (freeipa-pr-ci2#354) to test these changes with a template where @pki/master copr repo is enabled in Fedora 32.

Same could be applied here by using these lines:

freeipa/ipatests/prci_definitions/nightly_latest_pki.yaml

Lines 41 to 42 in 03a5e5f

    
           name: freeipa/pki-master-f32 
        
           version: 0.0.3

abbra · 2020-08-17T19:04:29Z

thanks @netoarmando. The packages get installed fine but then deployment fails due to exceptions in pkispawn:

Packages: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/328399be-e0b6-11ea-b730-fa163e4ddc69/installed_packages/installed_packages_master.log.gz

pki-acme-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.noarch
pki-base-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.noarch
pki-base-java-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.noarch
pki-ca-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.noarch
pki-kra-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.noarch
pki-server-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.noarch
pki-symkey-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.x86_64
pki-tools-10.10.0-0.1.alpha1.20200814013225UTC.13406396.fc32.x86_64

Failure: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/328399be-e0b6-11ea-b730-fa163e4ddc69/report.html

Failed to configure CA instance: CalledProcessError(Command ['/usr/sbin/pkispawn', '-s', 'CA', '-f',  '/tmp/tmpddfdis9k'] returned non-zero exit status 1: 
'Notice: Trust flag u is set automatically if the private key is present.
WARNING: Unable to modify o=ipaca: netscape.ldap.LDAPException: error result (20); Type or value exists
Job for pki-tomcatd@pki-tomcat.service failed because the control process exited with error code.
See "systemctl status pki-tomcatd@pki-tomcat.service" and "journalctl -xe" for details.
ERROR: CalledProcessError: Command \'[\'systemctl\', \'start\', \'pki-tomcatd@pki-tomcat.service\']\' returned non-zero exit status 1.
  File "/usr/lib/python3.8/site-packages/pki/server/pkispawn.py", line 575, in main
    scriptlet.spawn(deployer)
  File "/usr/lib/python3.8/site-packages/pki/server/deployment/scriptlets/configuration.py", line 870, in spawn
    instance.start()
  File "/usr/lib/python3.8/site-packages/pki/server/__init__.py", line 261, in start
    subprocess.check_call(cmd)
  File "/usr/lib64/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)

')
See the installation logs and the following files/directories for more information:
  /var/log/pki/pki-tomcat
CA configuration failed.

cipherboy · 2020-08-17T19:09:52Z

Seems like a weird permissions error:

Aug 17 18:40:10 replica0.ipa.test pki-server[29810]: FileNotFoundException: /etc/pki/pki-tomcat/alias/ca.crt (Permission denied)
Aug 17 18:40:11 replica0.ipa.test pki-server[29788]: ERROR: Command: pki -d /etc/pki/pki-tomcat/alias -C /tmp/tmpe297v97t/password.txt nss-cert-export --with-chain --format PEM Server-Cert cert-pki-ca /etc/pki/pki-tomcat/alias/ca.crt
Aug 17 18:40:11 replica0.ipa.test audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pki-tomcatd@pki-tomcat comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Aug 17 18:40:11 replica0.ipa.test systemd[1]: pki-tomcatd@pki-tomcat.service: Control process exited, code=exited, status=255/EXCEPTION
Aug 17 18:40:11 replica0.ipa.test systemd[1]: pki-tomcatd@pki-tomcat.service: Failed with result 'exit-code'.
Aug 17 18:40:11 replica0.ipa.test systemd[1]: Failed to start PKI Tomcat Server pki-tomcat.
Aug 17 18:40:11 replica0.ipa.test systemd[1]: pki-tomcatd@pki-tomcat.service: Consumed 1.953s CPU time.

@edewata -- any thoughts?

See: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/328399be-e0b6-11ea-b730-fa163e4ddc69/test_integration-test_simple_replication.py-TestSimpleReplication-install/replica0.ipa.test/journal.gz

And: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/328399be-e0b6-11ea-b730-fa163e4ddc69/test_integration-test_simple_replication.py-TestSimpleReplication-install/replica0.ipa.test/var/log/pki/pki-ca-spawn.20200817183852.log.gz

cipherboy · 2020-08-17T19:11:29Z

@f-trivino as an aside, how would we get /etc/pki packaged in PR-CI output? Would that be possible, or does PR-CI use sensitive values that could get written there? (HSM passwords, ...)

netoarmando · 2020-08-17T19:33:44Z

@f-trivino as an aside, how would we get /etc/pki packaged in PR-CI output? Would that be possible, or does PR-CI use sensitive values that could get written there? (HSM passwords, ...)

@cipherboy I can't talk about data sensitivity, but logs available in PR-CI output are defined here: https://github.com/freeipa/freeipa/blob/master/ipatests/pytest_ipa/integration/__init__.py#L43

abbra · 2020-08-17T19:40:54Z

Right now we don't have any hardware-specific secrets in PR CI. Also, VMs get destroyed before logs copied out to a storage location, so anything there is not sensitive anymore.

edewata · 2020-08-17T20:43:42Z

@cipherboy Not sure. The CI for commit 1340639 passed. Is pkiuser supposed to have a write access to /etc/pki/pki-tomcat/alias in this particular IPA installation scenario?

abbra · 2020-08-18T05:32:19Z

On my local F32 test deployment of git master I have:

ls -la /etc/pki/pki-tomcat/alias/
total 176
drwxrwx---. 2 pkiuser pkiuser  4096 Jul 30 08:27 .
drwxrwx---. 7 pkiuser pkiuser  4096 Jul 30 08:26 ..
-rw-------. 1 pkiuser pkiuser  3453 Jul 30 08:27 ca.crt
-rw-------. 1 pkiuser pkiuser 49152 Jul 30 08:26 cert9.db
-rw-------. 1 pkiuser pkiuser 94208 Jul 30 08:26 key4.db
-rw-------. 1 pkiuser pkiuser   432 Jul 30 08:20 pkcs11.txt
-rw-------. 1 pkiuser pkiuser    42 Jul 30 08:22 pwdfile.txt

So yes, it is supposed to be owned by pkiuser:pkiuser.

edewata · 2020-08-18T22:33:06Z

If the file permissions were correct, the last command shouldn't have failed since it's simply exporting a cert from NSS database into a file. Could someone retry the command on the test machine (with the --debug option)?

$ pki --debug -d /etc/pki/pki-tomcat/alias -c <NSS database password> nss-cert-export \
    --with-chain \
    --format PEM \
    "Server-Cert cert-pki-ca" /etc/pki/pki-tomcat/alias/ca.crt

abbra · 2020-08-19T06:17:32Z

It didn't work for me in pki 10.9.0-0.4, only worked with 10.10.0-0.1.alpha1. I used a different file in the same folder to save the CA certificate to:

[root@master ~]# su - pkiuser -s /bin/bash
Last login: Wed Aug 19 06:03:54 UTC 2020 on pts/0
[pkiuser@master ~]$ pki --debug -d /etc/pki/pki-tomcat/alias -C /etc/pki/pki-tomcat/alias/pwdfile.txt nss-cert-export --with-chain --format PEM "Server-Cert cert-pki-ca" /etc/pki/pki-tomcat/alias/ca1.crt
INFO: PKI options: --debug -d /etc/pki/pki-tomcat/alias -C /etc/pki/pki-tomcat/alias/pwdfile.txt
INFO: PKI command: nss-cert-export nss-cert-export --with-chain --format PEM Server-Cert cert-pki-ca /etc/pki/pki-tomcat/alias/ca1.crt
INFO: Java command: /usr/lib/jvm/jre-openjdk/bin/java -cp /usr/share/pki/lib/* -Djava.util.logging.config.file=/usr/share/pki/etc/logging.properties com.netscape.cmstools.cli.MainCLI -d /etc/pki/pki-tomcat/alias -C /etc/pki/pki-tomcat/alias/pwdfile.txt --debug nss-cert-export --with-chain --format PEM Server-Cert cert-pki-ca /etc/pki/pki-tomcat/alias/ca1.crt
INFO: Server URL: https://master.ipa.test:8443
INFO: Loading NSS password from /etc/pki/pki-tomcat/alias/pwdfile.txt
INFO: NSS database: /etc/pki/pki-tomcat/alias
INFO: Message format: null
INFO: Command: nss-cert-export --with-chain --format PEM "Server-Cert cert-pki-ca" /etc/pki/pki-tomcat/alias/ca1.crt
INFO: Module: nss
INFO: Module: cert
INFO: Module: export
INFO: Initializing NSS
INFO: Logging into internal token
INFO: Using internal token
[pkiuser@master ~]$ ls -la /etc/pki/pki-tomcat/alias/
total 180
drwxrwx---. 2 pkiuser pkiuser  4096 Aug 19 06:12 .
drwxrwx---. 7 pkiuser pkiuser  4096 Jul 30 08:26 ..
-rw-r--r--. 1 pkiuser pkiuser  3102 Aug 19 06:12 ca1.crt
-rw-------. 1 pkiuser pkiuser  3453 Jul 30 08:27 ca.crt
-rw-------. 1 pkiuser pkiuser 49152 Jul 30 08:26 cert9.db
-rw-------. 1 pkiuser pkiuser 94208 Jul 30 08:26 key4.db
-rw-------. 1 pkiuser pkiuser   432 Jul 30 08:20 pkcs11.txt
-rw-------. 1 pkiuser pkiuser    42 Jul 30 08:22 pwdfile.txt

cipherboy · 2020-08-24T19:20:55Z

The above permissions issue was also reported by @ssidhaye and fixed in Fedora and RHEL packages. A fix should be available in COPR repos for 10.9 and 10.10 packages, plus fixed in F33+. Note that all Fedoras are waiting for respin to fix upgrade issue + additional issue Endi reported. Only F33 compose has succeeded since packages were built; Rawhide compose is still broken apparently.

netoarmando · 2020-09-03T13:30:05Z

Before merging this we need:

* either put pki-core 10.10+ build to https://copr.fedorainfracloud.org/coprs/g/freeipa/freeipa-master/ or add a separate COPR repo with pki-core 10.10+ as discussed between FreeIPA and PKI developers earlier

* add that COPR repo to Azure CI pipeline setup

* add that COPR repo to PR CI (if it is @freeipa/freeipa-master COPR repo, it is already used by PR CI)

As of now, @pki/master is a runtime dependency for @freeipa/freeipa-master. This is blocking me to release new PR-CI boxes to fix https://pagure.io/freeipa/issue/8473 (problem can be seen here: freeipa-pr-ci2#371)

We can either enable @pki/master to ci-master-* templates or remove it as a runtime dependency for @freeipa/freeipa-master. I would rather pick the latter and, as listed by @abbra as one of the options, put pki-core 10.10+ build to @freeipa/freeipa-master.

Update: @pki/master removed as a dependency.

flo-renaud · 2020-09-10T16:30:46Z

@cipherboy
If you provide a build in @pki/10.10 repo, we will be able to add the rpms to our @freeipa/freeipa-master copr repo, and your PR adding the dependency on pki-acme should work.
In our upstream tests we have 2 different sets of tests: one with the @freeipa/freeipa-master copr and one with @pki/master additionally enabled, we don't want to enable @pki/master for all the tests.

rcritten · 2020-09-10T22:17:21Z

To throw in a monkey wrench I'll soon need this version of PKI for the ipa-4-8 branch too in order to test backporting ACME there.

frasertweedale · 2020-09-18T05:41:07Z

Here's an alternative PR where the pki-acme dependency only arsises if pki-ca >= 10.10.0 (i.e. the version at which the pki-acme package came into existence). #5117

rcritten · 2020-09-18T18:19:41Z

#5117 merged, closing this one.

abbra added the ipa-next Mark as master (4.12) only label Aug 17, 2020

abbra added the re-run Trigger a new run of PR-CI label Aug 17, 2020

freeipa-pr-ci removed the re-run Trigger a new run of PR-CI label Aug 17, 2020

flo-renaud added the re-run Trigger a new run of PR-CI label Aug 31, 2020

freeipa-pr-ci removed the re-run Trigger a new run of PR-CI label Aug 31, 2020

flo-renaud mentioned this pull request Sep 1, 2020

Troubleshoot Selenium error freeipa-pr-ci2/freeipa#371

Closed

frasertweedale mentioned this pull request Sep 18, 2020

spec: require pki-acme if pki-ca >= 10.10 #5117

Closed

rcritten closed this Sep 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add dependency on pki-acme #5039

Add dependency on pki-acme #5039

cipherboy commented Aug 17, 2020

abbra commented Aug 17, 2020

edewata commented Aug 17, 2020

abbra commented Aug 17, 2020

abbra commented Aug 17, 2020 •

edited

edewata commented Aug 17, 2020

cipherboy commented Aug 17, 2020

netoarmando commented Aug 17, 2020 •

edited

abbra commented Aug 17, 2020 •

edited

cipherboy commented Aug 17, 2020

cipherboy commented Aug 17, 2020

netoarmando commented Aug 17, 2020

abbra commented Aug 17, 2020

edewata commented Aug 17, 2020

abbra commented Aug 18, 2020

edewata commented Aug 18, 2020

abbra commented Aug 19, 2020

cipherboy commented Aug 24, 2020

netoarmando commented Sep 3, 2020 •

edited

flo-renaud commented Sep 10, 2020

rcritten commented Sep 10, 2020

frasertweedale commented Sep 18, 2020

rcritten commented Sep 18, 2020

Add dependency on pki-acme #5039

Add dependency on pki-acme #5039

Conversation

cipherboy commented Aug 17, 2020

abbra commented Aug 17, 2020

edewata commented Aug 17, 2020

abbra commented Aug 17, 2020

abbra commented Aug 17, 2020 • edited

edewata commented Aug 17, 2020

cipherboy commented Aug 17, 2020

netoarmando commented Aug 17, 2020 • edited

abbra commented Aug 17, 2020 • edited

cipherboy commented Aug 17, 2020

cipherboy commented Aug 17, 2020

netoarmando commented Aug 17, 2020

abbra commented Aug 17, 2020

edewata commented Aug 17, 2020

abbra commented Aug 18, 2020

edewata commented Aug 18, 2020

abbra commented Aug 19, 2020

cipherboy commented Aug 24, 2020

netoarmando commented Sep 3, 2020 • edited

flo-renaud commented Sep 10, 2020

rcritten commented Sep 10, 2020

frasertweedale commented Sep 18, 2020

rcritten commented Sep 18, 2020

abbra commented Aug 17, 2020 •

edited

netoarmando commented Aug 17, 2020 •

edited

abbra commented Aug 17, 2020 •

edited

netoarmando commented Sep 3, 2020 •

edited