Functional and Declarative BDD Testing Examples

[leefaus]

Feature Request

Description of Problem:

When writing tests for IaC solutions, I want to have the ability to use standard language conventions that already exist in other languages so I can reuse my knowledge and best practices as part of the solution.

Potential Solutions:

Using Pulumi would allow us to use common conventions that we already know with languages like Typescript and Go with Gherkin and BDD. There are already existing documented approaches to using industry standard frameworks with Pulumi like Typescript and Mocha.

This is an alternative to using specific testing frameworks that are IaC implementation specific and require learning a new framework.

[mcleo-d]

Hi @leefaus

Please note this issue was mentioned at #77 with the following investigation action taken ...

@eddie-knight to form an opinion and report the viability of Pulumi and BDD Tests #62 including @leefaus & @alfredtommy

Cheers 🚀 🚀 🚀

[eddie-knight]

1. Pulumi is very friendly for a developer's mindset (This from a Citihubber who prefers using Pulumi after using each technology on different projects)
2. Terraform is designed for a more "human readable" approach that infrastructure engineers tend to prefer (This from multiple Citihubbers who use Terraform)
3. Terraform is familiar for most (if not all) of our bank clients, who use it extensively
4. Most of our clients have a strong Java development base, so many of their contributors would need to learn a new language to use Pulumi- which reduces the benefit of not needing to learn Terraform
5. Terraform CDK provides a similar solution as Pulumi; it currently supports Python and Typescript
6. Terratest and other ecosystem projects help fill in many of the gaps in "vanilla" Terraform that Pulumi seeks to resolve

In sum, if your target audience is developers who prefer to use one of the four supported Pulumi languages, there is benefit in taking that route. If you have a more generalized audience including Java developers and cloud infrastructure engineers, it will be profitable to use Terraform instead.

As an aside to something that may not have been discussed yet... I would suggest that Pulumi and Terraform not be provided as "options" within the project, as the perceived complexity could cause "analysis paralysis" and become a deterrent to any newcomers who are exploring the project.

[mcleo-d]

Hi @leefaus and @eddie-knight,

During Cloud Service Certification tomorrow #79 we'll be pressing ahead with a review of the CSC Kanban Board, which means it would be great if you could attend to give an overview of work happening (or what you need to unblock and move forward) on the items below.

Please note the comment by @eddie-knight who has given feedback - #62 (comment)

Feel free to own the actions and call for support if needed.

Speak soon,

@mcleo-d

[adamj-codethink]

I was going to suggest a middle ground approach on the call, but i think we ran over.

But at Codethink we have used Jsonnet to abstract out the terraform modules, making it easier to read/test you can see some examples here.

[mcleo-d]

@adamj-codethink - Great suggestion. Feel free to coordinate with @leefaus who is the owner of this issue 👍

[mcleo-d]

@leefaus - Can you update the objectives and acceptance criteria of this issue so we know what to deliver back to the CSC project? The following are extracts from the 19th November CSC project meeting #79 to help.

1. Evaluate and describe the deltas between Terraform and Pulumi to provide Recommended and Alternative scenarios depending on cloud team capabilities.

2. Describe the differences between Functional versus Declarative BDD testing to enable internal team discussions for the purpose of providing feedback to Cloud Service Certification. Examples are also welcome.

[mcleo-d]

Group Discussion : CIS Benchmarks, BDD Testing and Tools Provisioning

• Spikes for defining how BDD scripts should be written and executed are being done through this issue ... Group Discussion : CIS Benchmarks, BDD Testing and Tools Provisioning #84 (comment)

[mcleo-d]

Hi @leefaus and @eddie-knight,

I have extended CSC Sprint 2 until 17th June '21 due to team member annual leave and have added this item to the #127 agenda to discuss in relation to completion and Sprint 3.

It would be great to talk through if we can close this issue and create individual issues for Probr and conftest on the #127 call?

Cheers,

James 🚀

[eddie-knight]

I suggest we close this issue as it is covered by individual action items #117, #124, and #128

[mcleo-d]

@leefaus - The Armory CCLA is now signed and you have been added to the FINOS CLA Bot. You are also free to move forward according to the following comment ... #119 (comment)

cc @eddie-knight

[mcleo-d]

@leefaus - The CSC project moved to backlog at the end of Sprint 3 #131

@leefaus to decide whether to bring into Sprint 4 https://github.com/orgs/finos/projects/1#column-15157368

[mcleo-d]

Closed issue as moved forward with OC-CDMC