adding go-cli (#67)

* adding go-cli Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * sign commit Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * rename project Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * adding init/create steps Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * adding globals note Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * removing the templates Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * update globals Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * update security check Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * rebranding cli Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Add check tools and update outputs Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * simplify os.shell calls Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * minor notes tweaks Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * cleaning create steps Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * address zip slip Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * address zip slip Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * cleaning some steps Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * reshuffle create steps Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Redirecting out to logs Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * add create dry-run Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * add progress bars Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * fix pr comments Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * reorg funcs Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * reorg funcs Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * synching changes Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * synching changes Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * synching changes Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * synching changes Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * review create/init Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * review imports Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * fix dry-run create Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * add some skip steps Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Adding skip steps Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * feat: add docker compose to kubefirst-dev Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: update gitlab folder structure, and clean up Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: update vault calls Signed-off-by: João Vanzuita <joao@kubeshop.io> * Add destroy buckets Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * added new lines Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * refactor: move setup functions to configs, add more descriptive documentation Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: add better log data Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: add better log data Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: add initial single source of truth for config, re-structure Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move kubernetes client requests to internal Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move kubernetes client requests to internal Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move gitlab functions to gitlab internal Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: remove globals, add values to config Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move git to internal Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: remove unused commands, refactor argocd and helm calls Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move ssh and gitlab functions Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move vault and softserve to internal Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: update logs to use logs.panic Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: move command line flags to function callers Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: update init progress bar Signed-off-by: João Vanzuita <joao@kubeshop.io> * added cluster-name and gitops-version Signed-off-by: Thiago Pagotto <pagottoo@gmail.com> * blank space Signed-off-by: Thiago Pagotto <pagottoo@gmail.com> * chore: merge arcocdsync and clean commands Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: merge in progress, code is breaking, now we can start validation and fix Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: merge in progress, code is breaking at init command Signed-off-by: João Vanzuita <joao@kubeshop.io> * refactor: update progress bar Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: update step names Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: update installation steps Signed-off-by: João Vanzuita <joao@kubeshop.io> * Rename repo Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * docs: update readme with more general details Signed-off-by: João Vanzuita <joao@kubeshop.io> * docs: fix typo Signed-off-by: João Vanzuita <joao@kubeshop.io> * reapply branch code Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * revert terraform action Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * fix err Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * re-ordering the functions (#89) * fix change Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * fix change again Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * feat: expose argocd container port Signed-off-by: João Vanzuita <joao@kubeshop.io> * fix: fix aws profile config Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: expose gitlab and vault ports Signed-off-by: João Vanzuita <joao@kubeshop.io> * Adding new ways to call shell Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Migrate most of terraforms Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Fix Var Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Fix err Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * Fix err Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * feat: update files and folders to follow rebranding Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: update naming left overs Signed-off-by: João Vanzuita <joao@kubeshop.io> * add new progress lib Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * create go-release-binaries workflow * add additional os/arch * Explore a smaller create flow Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * restore dry-run create Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * feat: implement handoff screen for clean command Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: clean up Signed-off-by: João Vanzuita <joao@kubeshop.io> * clean portforward noise Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * adding retry spec to argocd registry (#102) * enable bucket version Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * refactor: update k1srt folder to k1 Signed-off-by: João Vanzuita <joao@kubeshop.io> * feat: add info styled screen Signed-off-by: João Vanzuita <joao@kubeshop.io> * remove more noise Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * fix vars Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * fix lint stuff Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * feat: add handoff for create command Signed-off-by: João Vanzuita <joao@kubeshop.io> * feat: add non styled info command Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: clean up Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: clean up, and add tests Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: enable docker colors, use 256 colors to avoid colors mismatch Signed-off-by: João Vanzuita <joao@kubeshop.io> * chore: update info screen to be non blocking screen Signed-off-by: João Vanzuita <joao@kubeshop.io> * static kubeconfig naming convention (#112) Signed-off-by: johndietz <john@kubefirst.com> * k-ray cluster create updates (#108) * merge conflicts * upstream merge conflict resolution * adopts new struct definition * new wait functionality for vault * remove unused vault code * fixes for vault unseal (#114) * Vault unseal (#117) * fixes for vault unseal * not my favorite commit * remove not my stuff * Vault unseal (#118) * fixes for vault unseal * not my favorite commit * remove not my stuff * add reports package back * Vault unseal (#119) * fixes for vault unseal * not my favorite commit * remove not my stuff * add reports package back * viper check adjustments * viper check adjustments * Vault unseal (#120) * fixes for vault unseal * not my favorite commit * remove not my stuff * add reports package back * viper check adjustments * viper check adjustments * vault local address * changes, mostly formatting * spaces * adjustments through latest provisioning * Vault unseal tweak 0718 (#121) * allow the retry to work as expected * re-enable dry-run create Signed-off-by: 6za <53096417+6za@users.noreply.github.com> * adjustments for gitlab takeover * unused ref * detokenization games * add slash to .git/ on detokenize denylist * adding registry sync after gitlab gitops registry * terraform lock removal * adding argocd app host check and recycling/resyncing * addressinging orchestration issues from last run * port-forward fix Co-authored-by: jarededwards <jared@kubefirst.com> Co-authored-by: Cesar Filho <53096417+6za@users.noreply.github.com> Co-authored-by: 6za <53096417+6za@users.noreply.github.com> Co-authored-by: João Paulo Vanzuita <joao.vanzuita@cobi.bike> Co-authored-by: Thiago Pagotto <pagottoo@gmail.com> Co-authored-by: Jared Edwards <jared@kubefirst.com> Co-authored-by: John Dietz <john@kubefirst.com>
kubefirst · Jul 19, 2022 · ebe5bfc · ebe5bfc
1 parent f4d364b
commit ebe5bfc
Show file tree

Hide file tree

Showing 208 changed files with 6,319 additions and 14,096 deletions.
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
@@ -32,7 +32,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        language: [ 'javascript' ]
+        language: [ 'go' ]
         # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
         # Learn more about CodeQL language support at https://git.io/codeql-language-support
 

diff --git a/.github/workflows/go-release-binaries.yaml b/.github/workflows/go-release-binaries.yaml
@@ -8,8 +8,24 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v3
+    - uses: wangyoucao577/go-release-action@v1.29
+      with:
+        github_token: ${{ secrets.RELEASE_TOKEN }}
+        goos: darwin
+        goarch: amd64
+    - uses: wangyoucao577/go-release-action@v1.29
+      with:
+        github_token: ${{ secrets.RELEASE_TOKEN }}
+        goos: darwin
+        goarch: arm64
     - uses: wangyoucao577/go-release-action@v1.29
       with:
         github_token: ${{ secrets.RELEASE_TOKEN }}
         goos: linux
         goarch: amd64
+    - uses: wangyoucao577/go-release-action@v1.29
+      with:
+        github_token: ${{ secrets.RELEASE_TOKEN }}
+        goos: linux
+        goarch: arm64
+
diff --git a/.gitignore b/.gitignore
@@ -10,4 +10,9 @@ kubeconfig_*
 */cypress/screenshots/
 */cypress/videos/
 .DS_Store
-/git
+/git
+bin
+.vscode/settings.json
+logs/
+/tmp
+lint_log.txt
diff --git a/Dockerfile b/Dockerfile
diff --git a/README.md b/README.md
@@ -1,128 +1,88 @@
-# nebulous
-The Kubefirst Open Source Platform
+# Kubefirst CLI
 
-![images/nebulous-arch.png](images/nebulous-arch.png)
+Kubefirst CLI is a cloud provisioning tool. With simple setup and few CLI calls, we spin up a full AWS cluster with full
+GitOps integration, secrets management, production and development Kubernetes environments ready to be consumed.
 
-## tl;dr:
-- step 1: establish a new aws account with a single hosted zone that's configured to receive traffic from your domain name registrar
-- step 2: add your 6 configuration values to kubefirst.env and run the nebulous container
-- step 3: get a fully-functioning application delivery ecosystem, complete with kubernetes, gitops, vault, terraform, atlantis, gitlab, gitlab-runner, and a sample app that demonstrates how it all works.
+- [Setup](#setup)
+- [Start the container](#start-the-container)
+- [Initialization](#initialization)
+- [Creation](#creation)
+- [Access ArgoCD](#access-argocd)
+- [Destroy](#destroy)
+- [Available Commands]()
 
----
+## Setup
 
-# user guide
+The setup is extremely simple, create a `.env` file in the root folder, and add the following variables:
 
-## docs
-- [introduction](https://docs.kubefirst.com/)
-- [installation](https://docs.kubefirst.com/nebulous/install.html)
-- [getting familiar](https://docs.kubefirst.com/kubefirst/getting-started.html)
-- [teardown](https://docs.kubefirst.com/nebulous/teardown.html)
-- [faq](https://docs.kubefirst.com/nebulous/faq.html)
-- [contact](https://docs.kubefirst.com/contact.html)
+| Variable           | example          |
+|--------------------|------------------|
+| AWS_PROFILE        | default          |
+| AWS_REGION         | us-east-1        |
+| CLOUD_PROVIDER=aws | aws              |
+| HOSTED_ZONE_NAME   | example.com      |
+| ADMIN_EMAIL        | john@example.com |
 
----
+## Start the container
 
-# contributor guide
+We run everything on isolation with Docker, for that, start the container with:
 
-The docs above are tailored to our end user's experience. However things are a little different if you're contributing to nebulous itself. The docs that follow are intended only for source contributors.
+```bash
+docker-compose up kubefirst-dev
+```
+
+## Initialization
 
-### step 1 - setup nebulous.env
+Some process requires previous initialization, for that, run:
 
-For a first run, this step is no different than the guidance to our end users, you need to set up a `kubefirst.env` in the nebulous repo's root directory. You can create the file template by running this from your terminal, editing with your values with the normal settings.
+```bash
+mkdir -p ~/.kubefirst
+go run . init --admin-email $ADMIN_EMAIL --cloud $CLOUD_PROVIDER --hosted-zone-name $HOSTED_ZONE_NAME --region $AWS_REGION
+```
 
-For subsequent executions, especially while debugging, it's sometimes helpful to use some additional environment variables that allow you to control the flow of execution. See the notes in each section for details on controlling your debugging.
+## Creation
 
-In addition to the flow controls, you'll also find some hack comments by the various terraform apply commands. This allows you to change apply commands to exiting deploy commands. This can also be valuable when you need a mulligan on a particular section.
+At this point, everything is ready to start provisioning the cloud services, and for that we can run:
 
 ```bash
-cat << EOF > kubefirst.env
-###############################
-# Access settings
-# The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are your credentials to 
-# log into your AWS account, you can often find these in `~/.aws/credentials`
-# The AWS_DEFAULT_REGION is the aws region that your new infrastructure will provision in - 
-# The AWS_HOSTED_ZONE_NAME is the domain name associated with your prerequesite hosted zone in route53 - it should look similar to yourdomain.com with no www. prefix and no . suffix
-
-AWS_ACCESS_KEY_ID=YOUR_ADMIN_AWS_ACCESS_KEY_ID
-AWS_SECRET_ACCESS_KEY=YOUR_ADMIN_AWS_SECRET_ACCESS_KEY
-AWS_HOSTED_ZONE_NAME=yourdomain.com
-AWS_DEFAULT_REGION=us-east-2
-
-
-###################
-# Admin settings
-# The EMAIL_ADDRESS is used for the ssh key that's generated and for certificate expiration notifications
-# The GITLAB_BOT_ROOT_PASSWORD is the password to use for the gitlab root user, change this to a value only you know
-
-EMAIL_ADDRESS=YOUR_EMAIL_ADDRESS@yourdomain.com
-GITLAB_BOT_ROOT_PASSWORD=123456ABCDEF!
-
-
-###############################
-# Users:
-# The BUCKET_RAND needs to be set and uncommented before destroy, see the teardown 
-# docs for details.
-# 
-# Contributors: 
-# The BUCKET_RAND has implications on bucket reuse when iterating
-# once you successfully get past base terraform apply, 
-# take the random suffix that was generated, apply it to the 
-# next line, and start reusing the bucket for subsequent runs.
-# if you don't set this value on subsequent runs, it will keep 
-# generating new buckets for you. You can find this value in the 
-# nebulous execution output.
-# 
-# BUCKET_RAND=abc123
-
-
-###############################
-# Note: Operational Flow Controls - uncomment the items below 
-# when you want to skip over various sections. Leaving them
-# all commented like they are here will execute everything.
-# 
-#
-# SKIP_HZ_CHECK=true
-# SKIP_DETOKENIZATION=true
-# SKIP_BASE_APPLY=true
-# SKIP_GITLAB_RECONFIG=true
-# SKIP_GITLAB_APPLY=true
-# SKIP_ARGOCD_APPLY=true
-# SKIP_VAULT_APPLY=true
-# SKIP_SSH_STORAGE=true
-# SKIP_USERS_APPLY=true
-# SKIP_OIDC_PATCHING=true
-
-EOF
+go run . create
 ```
 
-### step 2 - build nebulous locally
-
-Come up with local tag name for your nebulous image. We'll use `foo` as our example local tag name in these docs. To build the `foo` tag of nebulous run the following from your local nebulous repo root directory.
+## Access ArgoCD
 
 ```bash
-docker build . -t nebulous:foo
+aws eks update-kubeconfig --name kubefirst
+kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d
+kubectl -n argocd port-forward svc/argocd-server 8080:80
 ```
 
-### step 3 - running nebulous
+## Destroy
 
-Once you have built the `nebulous:foo` image as shown above, you can kickoff the automated init script by running the following. The difference between this guidance and the end user guidance is that this mounts the `gitops`, `scripts`, and `git` directories to your localhost volume so you can negotiate changes to the runtime environment on the fly.
+It will destroy the kubefirst management cluster, and clean up every change made in the cloud.
 
-This is how you run the container with the volume mounts. Run this from your nebulous directory:
-```
-docker run -it --env-file=kubefirst.env -v $PWD/gitops:/gitops -v $PWD/metaphor:/metaphor -v $PWD/scripts:/scripts -v $PWD/git:/git --entrypoint /scripts/nebulous/init.sh nebulous:foo
-```
+```bash
 
-### step 4 - teardown (once you're ready to tear it all back down, obviously)
+go run . destroy
+rm -rf ~/.kubefirst
+rm ~/.flare
+```
 
-There are a few things to note about teardown.
+## Available Commands
 
-Nebulous creates a VPC, some subnets, a gitlab server, a kubernetes cluster, some policies, roles, and a few other things (complete list in the teardown docs). Terraform knows about all of these things, and if you only created these resources, you'll be able to run teardown without thinking too hard.
+Kubefirst provides extra tooling for handling the provisioning work.
 
-However, terraform is only able to destroy resources that are managed in terraform. It doesn't know about things you do manually. Anything you may have added through non-terraform operations must be manually removed before running the teardown script. 
+| Command    | Description                                               |
+|------------|-----------------------------------------------------------|
+| argocdSync | Request ArgoCD to synchronize applications                |
+| checktools | use to check compatibility of .kubefirst/tools            |
+| clean      | removes all kubefirst resources locally for new execution |
+| create     | create a kubefirst management cluster                     |
+| destroy    | destroy the kubefirst management cluster                  |
+| info       | provides general Kubefirst setup data                     |
+| init       | initialize your local machine to execute `create`         |
+| version    | print the version number for kubefirst-cli"               |
 
-Let's consider, for example, a scenario where you manually `helm install`ed an app to your new cluster, and that app spins up a new load balancer in your VPC. If you don't remove that app and its load balancer before running destroy, you won't be able to complete the terraform destroy operation. This is because you can't remove a VPC that still has a live load balancer running in it.
+#### Notes:
 
-With that context in mind, once you've removed the manual things you may have added to this environment, you can kickoff the automated destroy script by running:
-```
-docker run -it --env-file=kubefirst.env -v $PWD/gitops:/gitops -v $PWD/metaphor:/metaphor -v $PWD/scripts:/scripts -v $PWD/git:/git --entrypoint /scripts/nebulous/destroy.sh  nebulous:foo  
-```
+added gitlab.yaml to registry  
+pushing local to soft origin