Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aryeh/rpc-auth #36

Merged
merged 58 commits into from
Nov 25, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
6fee28f
RPC Auth WIP
harryttd Oct 27, 2020
dec7345
WIP
harryttd Oct 27, 2020
20bc5ea
Delete nonce from redis after use
harryttd Oct 27, 2020
1b022e3
WIP Get python server running in k8s and expose through ingress
harryttd Oct 28, 2020
e438a41
Use (experimenting) devspace for hot reloading the python server
harryttd Oct 28, 2020
f65b18f
Validate url access token and passthrough to RPC endpoint
harryttd Oct 28, 2020
fb83b1b
Inject env vars into python server
harryttd Oct 29, 2020
8d2632c
Clean up client logs
harryttd Oct 29, 2020
f0d2f3b
Merge branch 'master' into aryeh/rpc-auth
harryttd Oct 29, 2020
e8687d3
tweaks
harryttd Oct 29, 2020
f1da427
Have devspace enable minikube nginx addon
harryttd Oct 29, 2020
8a604b2
Use host address that the client initiated with in the returned url
harryttd Oct 29, 2020
14b7034
Formatting using black and isort
harryttd Oct 30, 2020
c31ff23
Better naming for tezos rpc service url
harryttd Oct 30, 2020
8c3b8e1
Make /vending-machine http POST only
harryttd Oct 30, 2020
806870a
Remove create_nonce function
harryttd Oct 30, 2020
95ca56e
Use namespace fields in devspace.yaml
harryttd Nov 2, 2020
f7279aa
Update injected value for tezos rpc endpoint
harryttd Nov 2, 2020
8fe6e8c
tweak dockerfile to build smaller image
Nov 2, 2020
f9ce69f
Working nginx auth proxy
harryttd Nov 3, 2020
853cbd7
Better nginx auth configuration
harryttd Nov 4, 2020
f4ed012
Use uuid hex format for access token
harryttd Nov 4, 2020
224dc90
Use python version 3.9 in Dockerfile
harryttd Nov 4, 2020
1734ba1
Use @cache directive for chain id instead of global variable
harryttd Nov 4, 2020
de28627
pip install uwsgi
harryttd Nov 5, 2020
a1534f2
Rebuild docker image on changes + reload on k8s yaml changes
harryttd Nov 6, 2020
9f528dc
Run server in either dev or prod
harryttd Nov 10, 2020
be1272e
Merge branch 'rpc-auth-smaller-docker-image' into aryeh/rpc-auth
harryttd Nov 10, 2020
8a457be
Docker: Always run uwsgi and run it as non-root
harryttd Nov 11, 2020
0a8addc
Fix devspace not syncing files when running container as non-root
harryttd Nov 11, 2020
4b2dfff
Fix FLASK_ENV not being set in Dockerfile
harryttd Nov 12, 2020
cffc113
Store in redis public key hash instead of public key
harryttd Nov 12, 2020
cd78851
Save set of access tokens per tz address
harryttd Nov 12, 2020
6e82a3c
Validate nonce in redis transaction so that it can't be replayed
harryttd Nov 16, 2020
07a6340
Reload uwsgi after syntax error
harryttd Nov 16, 2020
074e7b5
Extract auth token in python instead of nginx
harryttd Nov 16, 2020
b416cb3
Merge branch 'master' into merge-auth-with-mkchain
harryttd Nov 16, 2020
7edf679
WIP merge rpc-auth with mkchain
harryttd Nov 16, 2020
9698c6a
Move lint.sh into scripts/
harryttd Nov 16, 2020
d509484
Merge branch 'master' into merge-auth-with-mkchain
harryttd Nov 16, 2020
ec6cbd8
WIP merge rpc authentication with mkchain
harryttd Nov 17, 2020
9ea611d
Don't expose tezos node RPC over zerotier
harryttd Nov 17, 2020
a5b8c4c
gitignore generated chain yaml files
harryttd Nov 17, 2020
d500c4a
isort + black
harryttd Nov 17, 2020
5357394
Use ip instead of localhost alias for rpc listening address
harryttd Nov 17, 2020
32c098f
Only run uwsgi touchreload in development
harryttd Nov 18, 2020
635b952
Run rpc-auth by default in prod
harryttd Nov 18, 2020
64e2de2
Merge branch 'master' into aryeh/rpc-auth
harryttd Nov 19, 2020
7fca9ae
Configure redis persistance
harryttd Nov 19, 2020
eec82f1
WIP
harryttd Nov 20, 2020
22c25d6
Configure redis persistance
harryttd Nov 19, 2020
3a0a70f
fix for devspace
harryttd Nov 20, 2020
e1361e8
Rename rpc-auth-service to just rpc-auth
harryttd Nov 20, 2020
c9a2ef8
Add FLASK_ENV var to devspace
harryttd Nov 20, 2020
38dde33
Properly handle falsey chain args
harryttd Nov 20, 2020
97d618a
Merge branch 'aryeh/rpc-auth' into devspace
harryttd Nov 23, 2020
3ca2c86
Devspace hook to increase fs.inotify.max_user_watches
harryttd Nov 24, 2020
bf55629
Merge branch 'master' into aryeh/rpc-auth
harryttd Nov 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ dist
*.pyc
*.egg-info
.venv*
.DS_Store
104 changes: 104 additions & 0 deletions rpc-auth/client/index.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
#!/bin/bash

while test $# -gt 0; do
case "$1" in
-h | --help)
echo "options:"
echo "-h, --help show brief help"
echo "--cluster-address specify ip or url for requesting nonce"
echo "--chain-id specify chain id of permission chain"
echo "--tz-alias specify tz alias of your key"
exit 0
;;
--cluster-address)
shift
if test $# -gt 0; then
export CLUSTER_ADDRESS=$1
else
echo "no address specified"
exit 1
fi
shift
;;
--chain-id)
shift
if test $# -gt 0; then
export CHAIN_ID=$1
else
echo "no chain id specified"
exit 1
fi
shift
;;
--tz-alias)
shift
if test $# -gt 0; then
export TZ_ALIAS=$1
else
echo "no tz alias specified"
exit 1
fi
shift
;;
*)
break
;;
esac
done

if [ -z "$CLUSTER_ADDRESS" ]; then
echo "--cluster-address flag is required"
exit 1
elif [ -z "$CHAIN_ID" ]; then
echo "--chain-id flag is required"
exit 1
elif [ -z "$TZ_ALIAS" ]; then
echo "--tz-alias flag is required"
exit 1
fi

if ! tezos-client show address $TZ_ALIAS >/dev/null 2>&1; then
echo "no public key hash alias named $TZ_ALIAS"
exit 1
fi

get_response_body() {
echo $1 | sed -e 's/ HTTPSTATUS\:.*//g'
}
get_response_status() {
printf '%q' $1 | sed -e 's/.*HTTPSTATUS://'
}

echo "Requesting data to sign..."
nonce_res=$(curl -s -X GET http://$CLUSTER_ADDRESS/vending-machine/$CHAIN_ID -w " HTTPSTATUS:%{http_code}")
NONCE=$(get_response_body "$nonce_res")
nonce_res_status=$(get_response_status "$nonce_res")

if [ "$nonce_res_status" != "200" ]; then
echo "Failed to get nonce. [HTTP status: $nonce_res_status]"
echo "$nonce_res"
exit 1
fi

# echo NONCE: "$NONCE"

echo "Signing data..."
SIGNATURE=$(tezos-client -p PsCARTHAGazK sign bytes 0x05${NONCE} for ${TZ_ALIAS} | cut -f 2 -d " ")
PUBLIC_KEY=$(tezos-client show address ${TZ_ALIAS} 2>/dev/null | grep "Public Key: " | awk '{print $3}')

# echo SIGNATURE: "$SIGNATURE"
# echo PUBLIC_KEY: "$PUBLIC_KEY"

echo "Sending request for RPC url..."
secret_url_res=$(curl -s -X GET -d "nonce=${NONCE}" -d "signature=${SIGNATURE}" -d "public_key=${PUBLIC_KEY}" http://$CLUSTER_ADDRESS/vending-machine -w " HTTPSTATUS:%{http_code}")
SECRET_URL=$(get_response_body "$secret_url_res")
secret_url_status=$(get_response_status "$secret_url_res")

if [ "$secret_url_status" != "200" ]; then
echo "Failed to get secret url. [HTTP status: $secret_url_status]"
echo "$secret_url_res"
exit 1
fi

echo "Your secret tezos node RPC url: $SECRET_URL"

4 changes: 4 additions & 0 deletions rpc-auth/server/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@


# Ignore devspace.yaml file to prevent image rebuilding after config changes
devspace.yaml
4 changes: 4 additions & 0 deletions rpc-auth/server/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@


# Ignore DevSpace cache and log folder
.devspace/
30 changes: 30 additions & 0 deletions rpc-auth/server/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
FROM python:3.8

ARG FLASK_ENV=development

# Creating Application Source Code Directory
RUN mkdir -p /var/rpc-auth/

# Setting Home Directory for containers
WORKDIR /var/rpc-auth/

# Install dependencies for pytezos
RUN apt-get update
RUN apt-get install -y libsodium-dev libsecp256k1-dev libgmp-dev

# Installing python dependencies
COPY requirements.txt /var/rpc-auth/
RUN pip install --no-cache-dir -r requirements.txt

# Copying src code to Container
COPY index.py /var/rpc-auth/

# Application Environment variables
ENV FLASK_ENV=$FLASK_ENV
ENV PYTHONUNBUFFERED=x

# Exposing Ports
EXPOSE 8080

# Running Python Application
CMD ["python3", "index.py"]
157 changes: 157 additions & 0 deletions rpc-auth/server/backend.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
apiVersion: v1
kind: Service
metadata:
name: redis-service
namespace: tqtezos
labels:
app: redis
spec:
ports:
- name: redis
port: 6379
selector:
app: redis
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: redis-pv-claim
namespace: tqtezos
labels:
app: redis
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: tqtezos
labels:
app: redis
spec:
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: master
image: redis
readinessProbe:
exec:
command:
- sh
- -c
- "redis-cli -h $(hostname) ping"
initialDelaySeconds: 5
timeoutSeconds: 5
livenessProbe:
exec:
command:
- sh
- -c
- "redis-cli -h $(hostname) ping"
initialDelaySeconds: 5
periodSeconds: 3
# resources:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k8s noob question: at what point in time do we configure resources for all of our different infrastructure objects? How do we determine what the resources should be?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's important to set resources once we are in production and have metrics collection in place. The main benefit is to have sensible autoscaling behaviour. In a minikube context it matters less since there is only one node anyway.

# requests:
# cpu: 100m
# memory: 100Mi
ports:
- containerPort: 6379
volumeMounts:
- mountPath: /data
name: redis-data
volumes:
- name: redis-data
persistentVolumeClaim:
claimName: redis-pv-claim
---
apiVersion: v1
kind: Service
metadata:
name: rpc-auth-service
namespace: tqtezos
spec:
selector:
app: rpc-auth
ports:
- port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rpc-auth
namespace: tqtezos
spec:
selector:
matchLabels:
app: rpc-auth
template:
metadata:
labels:
app: rpc-auth
spec:
containers:
- name: rpc-auth
image: rpc-auth
imagePullPolicy: Never
ports:
- containerPort: 8080
env:
- name: REDIS_HOST
value: redis-service
- name: REDIS_PORT
value: "6379"
- name: TEZOS_RPC
value: tezos-rpc
- name: TEZOS_RPC_PORT
value: "8732"
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rpc-auth-ingress
namespace: tqtezos
annotations:
# nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- http:
paths:
# Client provides chain id and gets back a nonce
- path: /vending-machine/(.*)
pathType: Exact
backend:
service:
name: rpc-auth-service
port:
number: 8080
# Client provides signed data and gets back a secret url
- path: /vending-machine
pathType: Exact
backend:
service:
name: rpc-auth-service
port:
number: 8080
# Client uses secret url to access the RPC endpoint
- path: /tezos-node-rpc/(.*)
pathType: Exact
backend:
service:
name: rpc-auth-service
port:
number: 8080
---

23 changes: 23 additions & 0 deletions rpc-auth/server/devspace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
version: v1beta9
deployments:
- name: backend
kubectl:
manifests:
- ./backend.yaml
images:
server:
image: rpc-auth
dev:
sync:
- imageName: server
labelSelector:
app: rpc-auth
hooks:
- command: minikube
args:
- addons
- enable
- ingress
when:
before:
deployments: all
Loading