Entrust Datacard - Support for "entrust" provider in openssl_certificate module

[ctrufan]

SUMMARY

Addition of support for a new 'provider' value for the openssl_certificate module, to connect to Entrust Datacards "ECS" API for submission of CSRs.

Currently returns (e.g. as the 'return value') all the data that is returned by the ECS API when placing a certificate request. While returned, this data is not documented as part of the shared return data of the module, as much of it is essentially metadata. Some customers may want to register or log it for auditing and tracking purposes, so there is value in it's availability, but it isn't intended to be actioned upon by other tasks in a playbook. I couldn't find any instructions in documentation about whether documenting return values was mandatory, so this seemed in line with existing behavior (e.g. some providers return various pieces of data that aren't a part of the module doc currently). Regardless, the releases of the actual ECS API are intended to be backwards compatible - there are no values in this metadata that will not be available on future playbook runs.

The connection to the ECS API requires both client certificate and HTTP basic auth, and in addition to the CSR there are requester_name, requester_phone, and requester_email fields. The "ecs" module_util creates a client based on the OpenAPI 2.0 specification file defined for our API. We have integration tests on our end to verify that we never break forwards/backwards compatibility with future specification releases. The specification file defaults to a downloadable location, but can also be manually specified. This location is currently not available, but will be as of the next release of the API.

I left the "ocsp" variables without an entrust_ prefix as the functionality could easily be expanded for general cert validation logic, but for now documented them as entrust only. The actual method is not entrust specific, but I didn't test with a variety of other CAs and OCSP responders, so wasn't comfortable making it supported for other providers myself.

Both the module utility and the OCSP check use the ansible 'urls' lib for network connections, and to_text/to_native and six for conversions of input and output.

Details about the integration tests that were run included below, with an attached text file (with sensitive information removed) showing the results of test run.

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

ansible/modules/crypto/openssl_certificate.py
ansible/module_utils/ecs/init.py

ADDITIONAL INFORMATION

We (will) run integration tests on a weekly basis against the "devel" branch of ansible, against QA test systems representing both the current "live" release of the ECS API, and the in-development next release, with the goal of catching any breaking changes early.
entrust_openssl_playbook_partial.txt
entrust_openssl_playbookoutput.txt

For integration tests, we test via a molecule playbook set up to run against a number of docker images:

• CentOS7 with python 2.7, 'cryptography' installed
• CentOS7 with python 2.7, 'pyopenssl' installed
• CentOS7 with python 3.6, 'cryptography' installed
• CentOS7 with python 3.6, 'pyopenssl' installed
  (while I tested with Python 3.5, I had some issues with the docker images and ansible run unrelated to these module changes, so the automation runs on 3.6)

The actual operations tested in the test playbook include, in order:

• openssl_certificate with a local copy of API specification (for customers that want to minimize network traffic)
• openssl_certificate with a specified entrust_notafter.
• openssl_certificate with no api specification path specified (uses default).
• openssl_certificate explicitly specifying the path of the spec for an HTTP endpoint
• openssl_certificate pointing to an existing entrust-issued cert, that is still valid and with a far expiration data
• openssl_certificate pointing to an existing entrust-issued certificate that is close to expiration - this performs a 'RENEW' operation
• openssl_certificate pointing to an existing cert, but with a request for a different certificate type or pointing to a different private key than used to issue cert - this requests a new certificate

The python tests run by testinfra at the end of the integration tests test, in general:

• New certificates were issued for the commands expected to issue new certificates
• Expiration date was correct for the test which included an entrust_notafter
• The 'renew and succeed' certificate resulted in a new certificate issuance

• name: Generate an Entrust certificate via the Entrust ECS API
  openssl_certificate:
  path: /etc/ssl/crt/ansible.com.crt
  csr_path: /etc/ssl/csr/ansible.com.csr
  provider: entrust
  entrust_requester_name: Jo Doe
  entrust_requester_email: jdoe@ansible.com
  entrust_requester_phone: 555-555-5555
  entrust_cert_type: STANDARD_SSL
  entrust_api_user: api_username
  entrust_api_key: api_password
  entrust_api_client_cert_path: /etc/ssl/entrust/ecs-client.crt
  entrust_api_client_cert_key_path: /etc/ssl/entrust/ecs-key.crt
  entrust_api_specification_path: /etc/ssl/entrust/api-docs/cms-api-2.1.0.yaml

[ansibot]

cc @MarkusTeufelberger @Shaps @Spredzy @Xyon @felixfontein @puiterwijk @resmo
click here for bot help

[ansibot]

The test ansible-test sanity --test compile --python 2.6 [explain] failed with 1 error:

lib/ansible/module_utils/ecs/__init__.py:126:72: SyntaxError: x.get('name'): kwargs.get(x.get('name', None), None) for x in self.parameters if x.get('in') == 'path' and kwargs.get(x.get('name', None), None)

The test ansible-test sanity --test empty-init [explain] failed with 1 error:

lib/ansible/module_utils/ecs/__init__.py:0:0: empty __init__.py required

The test ansible-test sanity --test import --python 2.6 [explain] failed with 2 errors:

lib/ansible/module_utils/ecs/__init__.py:126:72: SyntaxError: invalid syntax
lib/ansible/modules/crypto/openssl_certificate.py:704:0: SyntaxError: invalid syntax (__init__.py, line 126)

click here for bot help

[ctrufan]

Ah, whoops. Meant to squash before pushing, sorry about the commit flood. Anyone can feel free to let me know if you'd like me to squash, or if ansible policy is to handle squash during hypothetical PR merge.

Thanks,
Chris

[felixfontein]

There's no need to squash now (as long as the commit count isn't getting too huge); it will be squashed during merge anyway.

[ctrufan]

Closing. Ran into something with my integration tests - the behavior of the CSR/issued certificate is inconsistent with the openssl_certificate module's other providers.

Namely, the "Organization" field in the subject DN. For OV/EV certificate types we default to the organization that is tied to the requester details, which is a customers validated organization. We do support other organizations, but only if explicitly specified as a parameter.

I could simply document that, but I think it would be better to change the behavior so the organization in the CSR is explicitly called out when we make the API call, so that we will get a clean "Unapproved organization of x provided" if the CSR is one that we can't issue a certificate against as-is.

[felixfontein]

I guess you can later also add a entrust_certificate (or more) modules which allow to access the API more flexibly. The openssl_certificate module should offer the basic options which are easy to implement (depending on the backend). I don't know how the Entrust API works, but your change sounds good :)

[ctrufan]

I guess you can later also add a entrust_certificate (or more) modules which allow to access the API more flexibly. The openssl_certificate module should offer the basic options which are easy to implement (depending on the backend). I don't know how the Entrust API works, but your change sounds good :)

That was the goal - basic options (and to be more or less plug and play with the other providers). Actually working on another module that will support a greater breadth of our options/functionality now.

@felixfontein thanks for all your input, really appreciate your comments.

[felixfontein]

LGTM. @MarkusTeufelberger @Spredzy do you want to take a look as well?

[felixfontein]

@MarkusTeufelberger @Spredzy if you need more time to look at this, please tell me. Otherwise I'll probably merge it this weekend. @ctrufan is working on another module which (I think) needs the new module_utils added here, and also there are too many openssl_certificate PRs open right now ;)

[felixfontein]

@ctrufan thank you for implementing this!

[MarkusTeufelberger]

Did you mean 365 days here?

[ctrufan]

Yes, holdover from before I changed 30->365 as the code default.

[MarkusTeufelberger]

This should be RFC 3339, ISO standard 3339 doesn't look timestamp related. ;-)

[ctrufan]

Oops, you're right! Good catch.

[MarkusTeufelberger]

Thanks for adding the provider and for reviewing the code!

[ctrufan]

@MarkusTeufelberger @felixfontein
Thanks for the reviews and merge!

[felixfontein]

@MarkusTeufelberger thank for reviewing as well! Sorry I already merged this... Wanted to remove (potential) merge conflicts... (looks like I forgot to update the sanity PR though)
@ctrufan can you address @MarkusTeufelberger's review comments in a new PR?

[MarkusTeufelberger]

No problem, I was not very available in the last few weeks and didn't tell y'all.

Better merge and fix than being stuck in rebase purgatory.