Multiple Issues With Relative Paths #150
Comments
armenr
changed the title
|
I do the same thing when referencing the module I'm testing from the fixture path. Mine looks like this: |
|
Oh, crap. I hope that's not it, because that'd be stupid. ...aaaaaaaand that wasn't it. Changed my main.tf to: Didn't make a difference. The only thing that seems to resolve properly is if I hardcode the FULL path. Relative paths don't seem to work, at all. |
|
This is mostly okay with me, since I've already done the Git magic to break my modules out into their own repo (and preserved their commit history...sheesh), but what is problematic is that I may want to work in a local branch, and hack on local copies of my modules - and run them through integration and unit tests before promoting them to master...and it would be nice to be able to do that by referencing my modules through relative paths. Uncertain what the issue is :-\ |
Problem Pt. 2I got kitchen-terraform working by passing it the GitHub reference for "source =" for the modules. That's because of Problem Pt. 1 This is actually a bit more problematic than I'd originally thought. I've developed and have been testing a plugin for using Ansible as a sort of provisioner, and I expose a relative path variable for where the roles and the playbook to use which I have in my main.tf. The variable declaration looks like this: When I try to converge with the relative path passed in, here's what I get: BUT when I pass the FULL path in the variable, converge works, completely, and without error. E.g. - I think I could explore a way to work around it by having the ansible module that uses the ansible privisioner plugin that I've been hacking on provide the full path to the playbook and the the roles folder as an output...but this is undesirable and shouldn't be necessary. I wouldn't want to add that work or code. So, it seems to me, there's an inherent problem with the way kitchen-terraform is inferring relative paths - such that it is actually broken. I can try to dig through your source code to see where relative path inference/expansion is happening, but I'd really appreciate any help or insights the authors and maintainers might be able to provide. Thank you! |
armenr
changed the title
|
The actual, accurate, relative path to where the file exists in real life is Similarly, when I use this local reference So, it would seem, with relative paths, it's always 3 directory levels off - To make kitchen-terraform work, I'd have to break my actual terraform runs. |
|
This is very strange, let's figure this out. Can you help me out with the details, maybe sending the output of a Is this the module folder you are testing? Is this the fixtures directory? And your trying to bring in this ansible folder: Trying to determine if it looks similar to our example module with tests: |
|
Greetings @armenr! There is a driver attribute called directory where test fixtures can be placed (http://www.rubydoc.info/gems/kitchen-terraform/Kitchen/Driver/Terraform) The value of that attribute effects will effect your converge. Without it being set, you're entire codebase (as opposed to fixture directory) will be copied to .kitchen/kitchen-terraform/default-ubuntu and that is where terraform is running from. So that explains the extra levels of directory you are needing to "escape" out of your module and into other terraform projects. |
|
@armenr: thanks for your interest in the project! I'm sorry that you're having a frustrating experience. As @nictrix and @edwardbartholomew have alluded to, I believe the problem that you are seeing is caused by our use of I am currently working on changes to the workflow which will drop the source module copying in favour of using a Terraform workspace. I believe these changes will solve your problem as no directories will be moved prior to Terraform resolving relative paths. |
|
Wow. The response and care has been overwhelming. You guys are awesome! I'm going to address responses in order they were posted. Since I haven't used kitchen in a couple of years, I may be rusty when it comes to this idea of fixtures. @nictrix - to answer your question: So the way I've been approaching this so far has been that I have a folder inside "project-ops" called testbed. That testbed is just a place for me to play with infra configs safely. It's just a sandbox. It is set up to mimic my already-in-production pritunl/vpn setup. You could technically break that off into a directory called "dev" that would sit at the same level of nestedness as "project-ops", within the "environments" folder. It probably should go there anyway... Anyhow, within the testbed directory, I describe an infra similar to my actually deployed VPN infra, and have a .kitchen.yml file there. I thought it would be as easy as firing up kitchen and having it converge, without realizing from the updates that you posted, as well from @edwardbartholomew and @ncs-alane that I may be missing a critical piece which is that driver attribute for fixtures. This is purely ID10T error on my part - not a fault of kitchen-terraform. :) This is what my .kitchen.yml looks like: So, I guess I might need some guidance to understand how and where to put my fixtures, what they essentially represent, and how that translates into the directory structure and kitchen.yml config I've been setting up. @edwardbartholomew - Thank you for the reference there! I'm going to dive into the documentation, realizing now that I completely missed that WHOLE aspect of properly getting set up and configured. @ncs-alane - THAT sounds super useful. I think the logical switch to workspaces is actually a brilliant way to leverage terraform's underlying native way of doing that. As for the workaround you mentioned, do you mean that I need to copy my aws_ec2 module, aws_vpc module, etc to that fixture directory, or just the code for testbed instance module (which is a module built on top of the aws_ec2 module)? The clarification would be a great help, not just in terms of understanding the plumbing better, but also to understand just how much DRY-ness may be sacrificed. I'm happy to make the sacrifice anyhow, since I'd value accurate testing over code repetition - but the repetition would mean that when I've made changes to the actual infra code, I'd also have to always remember to duplicate those changes over to the copies in /fixtures. Or I may have completely confused the hell out of myself and misunderstood you. The way I see it in my head is that there should be integration tests in the my testbed directory, or any directory where I'm creating full "stacks" of infra (vpc + security groups + ec2's + s3 buckets, etc...) that will converge with kitchen-terraform. This is now just a follow-up, and now this is me just being greedy because you awesome project contributors and maintainers are paying attention to me --> As a clearly separated concern, there should also be unit tests for each of my individual modules in the /modules/ directory. I'm assuming there will need to fixtures/fixture directories for the unit tests in each of the modules folders, as well as fixtures/fixture directories for the integration-oriented tests in each of my deployed "environment" directories? With the unit tests @ the module level, I understand I have to pass variables and write tests that assess some set of hypothetical sample variables which flatly verify things do what they are supposed to. In the case of my integration approach in the "environments," my desire is to have kitchen converge with the exact variables and params that will be passed to terraform in real-life. That is to say, is it possible to converge the same vars and configs (intended for prod) in kitchen-terraform without having a separate set of fixtures, or a copy/paste of the exact vars and code from /testbed/ into some nested /fixtures folder? If my questions indicate ignorance on my part, I apologize. I am a bit confused as to what that should actually look like, both in terms of organizing things at the filesystem level, and also in terms of what the .kitchen.yml config should look like. AGAIN - please let me restate my gratitude for everyone's very kind and prompt attention. You guys ROCK! |
|
Hi @armenr thanks for the explanation and information about your working structure. I believe I understand what you're trying to accomplish with the The changes I would recommend are:
final tree output under .
├── Gemfile
├── Gemfile.lock
└── test
├── fixtures
│ └── project_ops
│ ├── main.tf
│ ├── outputs.tf
│ ├── secrets.tf
│ ├── test1_instances.tf
│ ├── variables.tf
│ └── vpc.tf
└── integration
└── default
├── controls
└── inspec.ymlThen # cd into testbed directory
bundle exec kitchen list && \
bundle exec kitchen converge && \
bundle exec kitchen verifyI like what you're doing, testing all the modules put together in your live repo. And as you mentioned you definitely want to start testing each module separately. The setup for those should be similar to how the |
|
@nictrix - Thanks for that! Appreciate you taking the time to map that out for me. So, if I understand correctly, There's no way around code duplication, right? That is to say, if what I want to test (as my fixture) is the actual code I've written for my testbed environment, I'd have to duplicate those .tf files into a fixtures directory to test against? |
|
Ahh I see I thought testbed was just for testing, not something that stayed running for a long period of time. I just built a skeleton module to mimic My .kitchen.yml: ---
driver:
name: terraform
directory: "./"
provisioner:
name: terraform
platforms:
- name: terraform
transport:
name: ssh
verifier:
name: terraform
format: doc
groups:
- name: default
controls:
- operating_system
suites:
- name: default |
|
So, I actually tried that yesterday after reading your post and exploring the RubyDocs for the Driver API! I'm glad we're thinking along the same lines. I tested that yesterday in the afternoon - it didn't help. :-\ In /testbed/: In .kitchen.yml: Result of converge: Even if I created a secondary variables file to use for resolving my ansible playbooks, you can't interpolate variables in module "source=" lines, so I can't parameterize that. I think the only way to make this work (for now) is actually going to have to involve code duplication...unless I've missed something. The reason I'm insistent is because I'd like to check out a branch of my ops repo, hack on my modules, and test their convergence both at the module's unit-test level, as well as the integration-test level of any systems that are built on top of those modules, but I'd want to do on local copies of my modules and infra. |
|
And the reason I'm persistent/surfacing the issue so verbosely and actively here is not because I think the contributors/developers/maintainers OWE me custom features or anything...It's because I think this is an exceptionally powerful tool for use in safety-critical, security-critical, and regulated/compliance-framed deployments. Examples:
In all of these cases, one must document and show systems compliance. The happy road to "continuous compliance and security" is through config management and infrastructure as code. That's now evolved to "The code you write for your config management and infra should serve as self-documenting specification and proof." The next logical leap from there would be to say "We can prove compliance because we've translated our security and compliance requirements matrix into tests that we've written and run on our configurations and environments as part of the test-driven process of developing all of our configs and infra as code." With a tool like kitchen-terraform, the world of testing the FULL compliance spec (in the form of AWSSpec + InSpec tests) as integration tests becomes a very happy and useful reality...but it would be a workable reality when you can virtually simulate a full integration test on the complete, "as-it-would-be-in-production" set of code and variables/parameters without code duplication or differences in the vars you set. I know it might sound like making a mountain out of a mole-hill, but I've spent my last 5 years in compliance-centric environments, and we've dreamt of ways to achieve this reality. In addition to the work I've been trying to do with your kitchen-terraform testing tool, I've also accidentally discovered a new paradigm for wielding and coupling Ansible and Terraform together...so I'd actually be able to test the infra, the security groups, and all the infrastructure specification in ADDITION to my configuration management spec at the services level, all from one run, from one converge, and from one set of tests. It's something I've been quietly polishing and testing in the field, and am preparing to release as a set of open source plugins, modules, and an intro blog + detailed documentation. I'm happy to discuss...even help, but I haven't written Ruby in some time, so I don't know whether you'd want me coming anywhere near your code lol. |
|
@armenr: you are certainly welcome to hack away on the project! We definitely appreciate users opening issues because they provide us with valuable information that informs the design going forward. Given that usage patterns for Terraform are plentiful, it can be challenging for us to consider and test all possibilities in isolation. Allow me to reiterate that the next release of kitchen-terraform, featuring workspaces, should solve the problems you are facing with relative paths. I hope to publish the release branch today or tomorrow and I would be very appreciative if you could test your project with that branch to see if there are any additional issues that need to be addressed. |
|
@ncs-alane - You are the MAN! I completely appreciate that different people have different ways of approaching and wielding their tools and their set up, and what you folks have to do, as a team, is make decisions that maximize utility for the greatest number of people, in the least "unique snowflake" kind of way. I will happily and gladly test the release branch for you! :) I'll keep an eye out to see when it's published. Just as an update:I've been able to sidestep code duplication and the ability to test branch-specific modules in the following way: test1_instance.tf looks like this: And then I added a test_variables.tfvars: And in my .kitchen.yml, I configured: The fix required 2 things:
...Plus, it's all temporary anyhow because the new branch is going to kick all kinds of ass. :) |
|
@armenr: you can try the new branch now! You will need to adjust your gemfile accordingly, like: gem(
"kitchen-terraform",
git: "https://github.com/newcontext-oss/kitchen-terraform.git",
branch: "ncs-alane-split-workflow"
)Note that there are some changes to the driver configuration as outlined in 63d619a. |
|
Thanks man! :) Going to test drive this now. |
|
@ncs-alane - > Ran into some issues with the converge. Steps Taken:
EVERYTHING went off without a hitch, including relative path inference/resolution, so that is excellent!! 👍 BUT, my converge fails with the following message: Our account is mostly new, and mostly barren, but the previous person managing our infrastructure left behind some cruft in the form of VPC's that were never used. So, from what I can see, this is just us hitting the default limit of 5 VPC's per region. I've already asked for that to be increased. I didn't realize kitchen, when converging, would try to request an actual new VPC. Is this always the case? In the previous iteration of the kitchen-terraform gem, I had no such problem when running converge. By switching to workspaces, how are we suddenly hitting this default limit when, if I switch back to the old gem without workspaces, that wasn't the case? |
|
Alrightyroo - This was a terribly misguided assumption on my part, but I will most definitely need a testing.tfvars file. All else, however, is well squared away. Hooray for looking like a complete and total newb. |
|
@armenr no problem! 😆 By the way, you can also declare variables as a driver configuration attribute if you prefer that to using the variables file.
I'm glad to read this is the case; this was a fairly significant problem. |
|
Thank you. So far, we're still cruising. Really happy about this - and very grateful! @ncs-alane - It may be useful to also include something in the readme/documentation about how someone can manually pull an already existing state/terraform-managed infrastructure into kitchen-terraform, non-destructively. :) |
|
@armenr: Can you open a new issue for that point with some more details? |
|
Will do! Also, I'm a pretty big fan of this idea of passing custom vars through the .kitchen.yml file. The idea that everything you need to see or know about the kitchen convergence is in one file (the .kitchen.yml) is very attractive. I'm glad that you exposed the ability to pass variables directly as k/v pairs, without necessitating that the user must provide a tfvars for testing is wonderful. |
|
@ncs-alane --> Happy to open a separate Issue if you tell me to do so, I did run into this: Error Output:How would I pass a list to a var in the kitchen.yml file? If that's not yet supported, I can just use a tfvars file instead. |
|
@armenr: I believe I have a solution for you, but please open a separate issue so that this issue can remain focused on a single topic. |
#Problem Pt. 1
I've got modules I use to create EC2's. Makes for easy templating and repeatability.
I am very, very excited about the fact that we can now use the kitchen testing paradigm to test and verify infrastructure with InSpec. EXCELLENT WORK, FOLKS!! :)
Unfortunately, I'm getting errors on a completely vanilla, first-pass run for converge. From what I can tell, I think it's because I'm using relative paths to reference my modules directories.
Error reads:
Error downloading modules: Error loading modules: module test_server: invalid source string: ./../../../modules/aws_ec2For the environment in which I want to try running some kitchen-terraform testing + InSpec verification, I've got the following in my main.tf
For now, modules are local to the repo - my next effort after getting testing set up will be to individualize and version-stamp my modules in a repository separate from the actual infrastructure code and reference them via git and release tag. For now, I reference my modules by relative paths:
With the minimal configuration suggested in the "Getting Started" documentation, I have the following in my .kitchen.yml
When I run kitchen converge, I get the following:
When I reference my modules by absolute path, the trouble goes away:
Any help or advice would be greatly appreciated. Thank you!
The text was updated successfully, but these errors were encountered: