Linux support (including new-user documentation)

[mike-zipit]

This isn't a bug-report per-se but a new-user feedback list and items that would be needed to complete the documentation:

• README needs installation section.
• Install script needs to detect and use unix line-endings where appropriate
• aws-vault needs to be fully functional prior to installation
• New project setup guidance would be helpful

• I'm still struggling with how to use geodesic with an existing infrastructure. For instance, I have a full terraform project. How would I start using geodesic with that project? I'm sure you mount it somehow when you start geodesic, but I don't really see that option.
• The cloudeposse/geodesic output script (which you pipe into bash to install, which isn't documented) is in windows line-endings. I had to execute docker run --rm -it $IMAGE | tr -d "\r" > $TMP && bash $TMP 0.9.17
• aws-vault linux support is half-baked. I'm still struggling to get it fully working. It seems doing a backend=file helps, but there's still some weirdness I'm trying to understand
• Why does geodesic need to run with --privileged? If it's a must, then this complicates using aws-vault in both geodesic and native (geodesic will create vault keys as root in your home directory)

[osterman]

This is priceless feedback! I update the docs (tomorrow) and work with you to get it sorted out.

[osterman]

I'm going to take a stab at answering some of these inline. Let me just say that we know the docs are woefully incomplete, but we're investing hundreds of hours into them right now.

README needs installation section.

I think we will be updating the README.md to instead link out to the docs. We have a "quickstart", but it's not ideal. In fact, we're working on a better process for getting started that involves providing "Geodesic module" (repos) reference architectures that you will be able to simply fork. The current scaffolding system was well-intentioned, but even we don't use it. We always fork from another repo and change what we need. Stay posted. The first of these should be made public this week.

https://docs.cloudposse.com/geodesic/module/quickstart/

[osterman]

The cloudeposse/geodesic output script (which you pipe into bash to install, which isn't documented) is in windows line-endings. I had to execute docker run --rm -it $IMAGE | tr -d "\r" > $TMP && bash $TMP 0.9.17

This is actually due to running docker run -it; the -it causes it to run in "interactive tty mode" which causes the \r new line endings. Per the docs, drop the -it =)

[osterman]

aws-vault linux support is half-baked. I'm still struggling to get it fully working. It seems doing a backend=file helps, but there's still some weirdness I'm trying to understand

We have a pretty good write up on how we use this. @goruha uses linux too, so it's working pretty well. The key is to use AWS_VAULT_BACKEND=file if you want it to work b/w linux and OSX.

Our docs for aws-vault are here: https://docs.cloudposse.com/tools/aws-vault/

[osterman]

Why does geodesic need to run with --privileged? If it's a must, then this complicates using aws-vault in both geodesic and native (geodesic will create vault keys as root in your home directory)

I think we can loosen this requirement so that it only applies to OSX. The original reason is for OSX so we can run hwclock -s to sync the hardware clock of the Docker VM. Otherwise, when laptops go to sleep, we encounter time-drift and all signed AWS requests fail.

[osterman]

New project setup guidance would be helpful

100% agree. This has been surprisingly difficult to get right from a documentation perspective. Hope by EOW this will be in better shape.

[osterman]

I'm still struggling with how to use geodesic with an existing infrastructure. For instance, I have a full terraform project.

Okay, so I think the first thing is to think about "infrastructure code" the same as any other application code that you would ship with docker. For example, if you were going to distribute a NodeJS app, you'd distribute the .js code with the container. We look as terraform code the same way.

How would I start using geodesic with that project?

For example, here's what one of our typical Dockerfiles look like. In this case, it's for root.cloudposse.co, our upcoming reference architecture for the "root" (apex) AWS organization.

FROM r.cfcr.io/cloudposse/terraform-root-modules:0.1.5 as terraform-root-modules

FROM cloudposse/geodesic:0.9.17

ENV DOCKER_IMAGE "cloudposse/root.cloudposse.co"
ENV DOCKER_TAG "latest"

ENV BANNER="root.cloudposse.co"

# Default AWS Profile name
ENV AWS_DEFAULT_PROFILE="cpco-root-admin"

# AWS Region
ENV AWS_REGION="us-west-2"

# Terraform State Bucket
ENV TF_BUCKET="cpco-root-terraform-state"
ENV TF_BUCKET_REGION="us-west-2"
ENV TF_DYNAMODB_TABLE="cpco-root-terraform-state-lock"

# Terraform Vars
ENV TF_VAR_domain_name=root.cloudposse.co
ENV TF_VAR_namespace=cpco
ENV TF_VAR_stage=root

# chamber KMS config
ENV CHAMBER_KMS_KEY_ALIAS="alias/cpco-root-chamber"

# Copy root modules
COPY --from=terraform-root-modules /aws/tfstate-backend/ /conf/tfstate-backend/
COPY --from=terraform-root-modules /aws/chamber/ /conf/chamber/
COPY --from=terraform-root-modules /aws/cloudtrail/ /conf/cloudtrail/

# Place configuration in 'conf/' directory
COPY conf/ /conf/

# Filesystem entry for tfstate
RUN s3 fstab '${TF_BUCKET}' '/' '/secrets/tf'

WORKDIR /conf/

I'm sure you mount it somehow when you start geodesic, but I don't really see that option.

So as you can see from the above examples, we don't "mount" (per say) the .tf code into the container. We distribute it with the container.

Now, since rebuilding the container during development would be extremely tedious, we do mount $HOME to /localhost in the container. This allows you to easily iterate on a local copy of the .tf code.

[osterman]

The bigger question is how to deal with terraform remote state, if you're not doing it our way.

We have some bad docs on how we do it. They are bad in that they are the "complicated" way that can be simplified. Not yet sure when I can get to simplifying it, but the best thing is if we do a quick call this week or if you ping @aknysh on #community in our slack team.

[osterman]

Also, want to link to https://github.com/cloudposse/terraform-aws-tfstate-backend which is our module for managing tfstate. We use this little helper script (init-terraform) to setup remote state. Unfortunately, terraform isn't very "12-factor" compatible (as in supporting ENVs) when it comes to state.

[osterman]

I know terragrunt has some convenience around managing remote state as well. What I don't like is that it doesn't use terraform for provisioning the state bucket. What I'd like to see is instead a patch against terraform to support ENVs natively for these parameters.

[mike-zipit]

Per what docs? Neither the geodesic Readme nor the docs site mentioned installation.

…

On Sat, Jun 2, 2018, 10:54 PM Erik Osterman ***@***.***> wrote: This is actually due to running docker run -it; the -it causes it to run in "interactive tty mode" which causes the \r new line endings. Per the docs, drop the -it =) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfNDKMaVir7vxVCAFj4DYmG9adM-euyYks5t40_tgaJpZM4UXoxQ> .

[osterman]

Per what docs? Neither the geodesic Readme nor the docs site mentioned
installation.

My apologies. It does appear that that got lost somewhere. I'll be updating the docs.cloudposse.com portal with some fresh docs today.

[osterman]

We've improved linux support but still have problems with permissions on mounted directories.

We've also added support for WSL (windows).

https://docs.cloudposse.com/documentation/getting-started/
https://docs.cloudposse.com/reference-architectures/