⚠️ PLEASE, DO NOT EDIT THIS FILE. IT IS AUTOGENERATE YOU SHOULD EDIT modules/docs/src/main/mdoc/README.md WITH THE COMMAND sbt docs/mdoc

---

Spark

---

Spark is a project with many proof of concept modules...

---

Simple Data Lake

Simple Data Lake using AWS Lake Formation

Loading Real-time Data Into Amazon Redshift Cluster

Useful commands

eks

# Install aws cli and configure
brew install awscli
aws configure
 
# configure kube config
 aws eks --profile awspro --region eu-west-2 update-kubeconfig --name eks-cluster-name

pyenv

Installing pyenv on Mac OS X is very simple with Homebrew.

brew install pyenv
brew install pyenv-virtualenv

After installation, add the following at the bottom of your profile (~/.bash_profile or ~/.zshenv)

if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi
if which pyenv-virtualenv-init > /dev/null; then eval "$(pyenv virtualenv-init -)"; fi

export PATH="$(pyenv root)/shims:$PATH"

Installing new Python versions is very straightforward. All Python versions are installed in the versions directory under the pyenv root.

pyenv install 3.9.5

The pyenv versions command will also show the virtual environments along with the installed versions of Python.

pyenv versions

Create a virtual environment for a specific Python version.

pyenv virtualenv 3.9.5 python-3.9.5

List existing virtual environments created by pyenv

pyenv virtualenvs

Activate a virtual environment.

pyenv activate python-3.9.5
pyenv local python-3.9.5

pip

pip --version

molecule

Install molecule

pip install 'molecule[docker]'

Create a role

molecule init role --driver-name docker bastion

Run test

molecule test

Run test

molecule destroy

Links

https://www.akbaribrahim.com/managing-multiple-python-versions-with-pyenv/ https://www.akbaribrahim.com/managing-python-virtual-environments-with-pyenv-virtualenv/ https://zacks.one/aws-cloud-architecting/