-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Aryeh/rpc-auth #36
Merged
Merged
Aryeh/rpc-auth #36
Changes from 12 commits
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
6fee28f
RPC Auth WIP
harryttd dec7345
WIP
harryttd 20bc5ea
Delete nonce from redis after use
harryttd 1b022e3
WIP Get python server running in k8s and expose through ingress
harryttd e438a41
Use (experimenting) devspace for hot reloading the python server
harryttd f65b18f
Validate url access token and passthrough to RPC endpoint
harryttd fb83b1b
Inject env vars into python server
harryttd 8d2632c
Clean up client logs
harryttd f0d2f3b
Merge branch 'master' into aryeh/rpc-auth
harryttd e8687d3
tweaks
harryttd f1da427
Have devspace enable minikube nginx addon
harryttd 8a604b2
Use host address that the client initiated with in the returned url
harryttd 14b7034
Formatting using black and isort
harryttd c31ff23
Better naming for tezos rpc service url
harryttd 8c3b8e1
Make /vending-machine http POST only
harryttd 806870a
Remove create_nonce function
harryttd 95ca56e
Use namespace fields in devspace.yaml
harryttd f7279aa
Update injected value for tezos rpc endpoint
harryttd 8fe6e8c
tweak dockerfile to build smaller image
f9ce69f
Working nginx auth proxy
harryttd 853cbd7
Better nginx auth configuration
harryttd f4ed012
Use uuid hex format for access token
harryttd 224dc90
Use python version 3.9 in Dockerfile
harryttd 1734ba1
Use @cache directive for chain id instead of global variable
harryttd de28627
pip install uwsgi
harryttd a1534f2
Rebuild docker image on changes + reload on k8s yaml changes
harryttd 9f528dc
Run server in either dev or prod
harryttd be1272e
Merge branch 'rpc-auth-smaller-docker-image' into aryeh/rpc-auth
harryttd 8a457be
Docker: Always run uwsgi and run it as non-root
harryttd 0a8addc
Fix devspace not syncing files when running container as non-root
harryttd 4b2dfff
Fix FLASK_ENV not being set in Dockerfile
harryttd cffc113
Store in redis public key hash instead of public key
harryttd cd78851
Save set of access tokens per tz address
harryttd 6e82a3c
Validate nonce in redis transaction so that it can't be replayed
harryttd 07a6340
Reload uwsgi after syntax error
harryttd 074e7b5
Extract auth token in python instead of nginx
harryttd b416cb3
Merge branch 'master' into merge-auth-with-mkchain
harryttd 7edf679
WIP merge rpc-auth with mkchain
harryttd 9698c6a
Move lint.sh into scripts/
harryttd d509484
Merge branch 'master' into merge-auth-with-mkchain
harryttd ec6cbd8
WIP merge rpc authentication with mkchain
harryttd 9ea611d
Don't expose tezos node RPC over zerotier
harryttd a5b8c4c
gitignore generated chain yaml files
harryttd d500c4a
isort + black
harryttd 5357394
Use ip instead of localhost alias for rpc listening address
harryttd 32c098f
Only run uwsgi touchreload in development
harryttd 635b952
Run rpc-auth by default in prod
harryttd 64e2de2
Merge branch 'master' into aryeh/rpc-auth
harryttd 7fca9ae
Configure redis persistance
harryttd eec82f1
WIP
harryttd 22c25d6
Configure redis persistance
harryttd 3a0a70f
fix for devspace
harryttd e1361e8
Rename rpc-auth-service to just rpc-auth
harryttd c9a2ef8
Add FLASK_ENV var to devspace
harryttd 38dde33
Properly handle falsey chain args
harryttd 97d618a
Merge branch 'aryeh/rpc-auth' into devspace
harryttd 3ca2c86
Devspace hook to increase fs.inotify.max_user_watches
harryttd bf55629
Merge branch 'master' into aryeh/rpc-auth
harryttd File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,3 +4,4 @@ dist | |
*.pyc | ||
*.egg-info | ||
.venv* | ||
.DS_Store |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
#!/bin/bash | ||
|
||
while test $# -gt 0; do | ||
case "$1" in | ||
-h | --help) | ||
echo "options:" | ||
echo "-h, --help show brief help" | ||
echo "--cluster-address specify ip or url for requesting nonce" | ||
echo "--chain-id specify chain id of permission chain" | ||
echo "--tz-alias specify tz alias of your key" | ||
exit 0 | ||
;; | ||
--cluster-address) | ||
shift | ||
if test $# -gt 0; then | ||
export CLUSTER_ADDRESS=$1 | ||
else | ||
echo "no address specified" | ||
exit 1 | ||
fi | ||
shift | ||
;; | ||
--chain-id) | ||
shift | ||
if test $# -gt 0; then | ||
export CHAIN_ID=$1 | ||
else | ||
echo "no chain id specified" | ||
exit 1 | ||
fi | ||
shift | ||
;; | ||
--tz-alias) | ||
shift | ||
if test $# -gt 0; then | ||
export TZ_ALIAS=$1 | ||
else | ||
echo "no tz alias specified" | ||
exit 1 | ||
fi | ||
shift | ||
;; | ||
*) | ||
break | ||
;; | ||
esac | ||
done | ||
|
||
if [ -z "$CLUSTER_ADDRESS" ]; then | ||
echo "--cluster-address flag is required" | ||
exit 1 | ||
elif [ -z "$CHAIN_ID" ]; then | ||
echo "--chain-id flag is required" | ||
exit 1 | ||
elif [ -z "$TZ_ALIAS" ]; then | ||
echo "--tz-alias flag is required" | ||
exit 1 | ||
fi | ||
|
||
if ! tezos-client show address $TZ_ALIAS >/dev/null 2>&1; then | ||
echo "no public key hash alias named $TZ_ALIAS" | ||
exit 1 | ||
fi | ||
|
||
get_response_body() { | ||
echo $1 | sed -e 's/ HTTPSTATUS\:.*//g' | ||
} | ||
get_response_status() { | ||
printf '%q' $1 | sed -e 's/.*HTTPSTATUS://' | ||
} | ||
|
||
echo "Requesting data to sign..." | ||
nonce_res=$(curl -s -X GET http://$CLUSTER_ADDRESS/vending-machine/$CHAIN_ID -w " HTTPSTATUS:%{http_code}") | ||
NONCE=$(get_response_body "$nonce_res") | ||
nonce_res_status=$(get_response_status "$nonce_res") | ||
|
||
if [ "$nonce_res_status" != "200" ]; then | ||
echo "Failed to get nonce. [HTTP status: $nonce_res_status]" | ||
echo "$nonce_res" | ||
exit 1 | ||
fi | ||
|
||
# echo NONCE: "$NONCE" | ||
|
||
echo "Signing data..." | ||
SIGNATURE=$(tezos-client -p PsCARTHAGazK sign bytes 0x05${NONCE} for ${TZ_ALIAS} | cut -f 2 -d " ") | ||
PUBLIC_KEY=$(tezos-client show address ${TZ_ALIAS} 2>/dev/null | grep "Public Key: " | awk '{print $3}') | ||
|
||
# echo SIGNATURE: "$SIGNATURE" | ||
# echo PUBLIC_KEY: "$PUBLIC_KEY" | ||
|
||
echo "Sending request for RPC url..." | ||
secret_url_res=$(curl -s -X GET -d "nonce=${NONCE}" -d "signature=${SIGNATURE}" -d "public_key=${PUBLIC_KEY}" http://$CLUSTER_ADDRESS/vending-machine -w " HTTPSTATUS:%{http_code}") | ||
SECRET_URL=$(get_response_body "$secret_url_res") | ||
secret_url_status=$(get_response_status "$secret_url_res") | ||
|
||
if [ "$secret_url_status" != "200" ]; then | ||
echo "Failed to get secret url. [HTTP status: $secret_url_status]" | ||
echo "$secret_url_res" | ||
exit 1 | ||
fi | ||
|
||
echo "Your secret tezos node RPC url: $SECRET_URL" | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
|
||
|
||
# Ignore devspace.yaml file to prevent image rebuilding after config changes | ||
devspace.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
|
||
|
||
# Ignore DevSpace cache and log folder | ||
.devspace/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
FROM python:3.8 | ||
|
||
ARG FLASK_ENV=development | ||
|
||
# Creating Application Source Code Directory | ||
RUN mkdir -p /var/rpc-auth/ | ||
|
||
# Setting Home Directory for containers | ||
WORKDIR /var/rpc-auth/ | ||
|
||
# Install dependencies for pytezos | ||
RUN apt-get update | ||
RUN apt-get install -y libsodium-dev libsecp256k1-dev libgmp-dev | ||
|
||
# Installing python dependencies | ||
COPY requirements.txt /var/rpc-auth/ | ||
RUN pip install --no-cache-dir -r requirements.txt | ||
|
||
# Copying src code to Container | ||
COPY index.py /var/rpc-auth/ | ||
|
||
# Application Environment variables | ||
ENV FLASK_ENV=$FLASK_ENV | ||
ENV PYTHONUNBUFFERED=x | ||
|
||
# Exposing Ports | ||
EXPOSE 8080 | ||
|
||
# Running Python Application | ||
CMD ["python3", "index.py"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,157 @@ | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: redis-service | ||
namespace: tqtezos | ||
labels: | ||
app: redis | ||
spec: | ||
ports: | ||
- name: redis | ||
port: 6379 | ||
selector: | ||
app: redis | ||
--- | ||
kind: PersistentVolumeClaim | ||
apiVersion: v1 | ||
metadata: | ||
name: redis-pv-claim | ||
namespace: tqtezos | ||
labels: | ||
app: redis | ||
spec: | ||
accessModes: | ||
- ReadWriteMany | ||
resources: | ||
requests: | ||
storage: 5Gi | ||
--- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: redis | ||
namespace: tqtezos | ||
labels: | ||
app: redis | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: redis | ||
template: | ||
metadata: | ||
labels: | ||
app: redis | ||
spec: | ||
containers: | ||
- name: master | ||
image: redis | ||
readinessProbe: | ||
exec: | ||
command: | ||
- sh | ||
- -c | ||
- "redis-cli -h $(hostname) ping" | ||
initialDelaySeconds: 5 | ||
timeoutSeconds: 5 | ||
livenessProbe: | ||
exec: | ||
command: | ||
- sh | ||
- -c | ||
- "redis-cli -h $(hostname) ping" | ||
initialDelaySeconds: 5 | ||
periodSeconds: 3 | ||
# resources: | ||
# requests: | ||
# cpu: 100m | ||
# memory: 100Mi | ||
ports: | ||
- containerPort: 6379 | ||
volumeMounts: | ||
- mountPath: /data | ||
name: redis-data | ||
volumes: | ||
- name: redis-data | ||
persistentVolumeClaim: | ||
claimName: redis-pv-claim | ||
--- | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: rpc-auth-service | ||
namespace: tqtezos | ||
spec: | ||
selector: | ||
app: rpc-auth | ||
ports: | ||
- port: 8080 | ||
targetPort: 8080 | ||
--- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: rpc-auth | ||
namespace: tqtezos | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: rpc-auth | ||
template: | ||
metadata: | ||
labels: | ||
app: rpc-auth | ||
spec: | ||
containers: | ||
- name: rpc-auth | ||
image: rpc-auth | ||
imagePullPolicy: Never | ||
ports: | ||
- containerPort: 8080 | ||
env: | ||
- name: REDIS_HOST | ||
value: redis-service | ||
- name: REDIS_PORT | ||
value: "6379" | ||
- name: TEZOS_RPC | ||
value: tezos-rpc | ||
- name: TEZOS_RPC_PORT | ||
value: "8732" | ||
--- | ||
apiVersion: networking.k8s.io/v1 | ||
kind: Ingress | ||
metadata: | ||
name: rpc-auth-ingress | ||
namespace: tqtezos | ||
annotations: | ||
# nginx.ingress.kubernetes.io/rewrite-target: / | ||
nginx.ingress.kubernetes.io/use-regex: "true" | ||
spec: | ||
rules: | ||
- http: | ||
paths: | ||
# Client provides chain id and gets back a nonce | ||
- path: /vending-machine/(.*) | ||
pathType: Exact | ||
backend: | ||
service: | ||
name: rpc-auth-service | ||
port: | ||
number: 8080 | ||
# Client provides signed data and gets back a secret url | ||
- path: /vending-machine | ||
pathType: Exact | ||
backend: | ||
service: | ||
name: rpc-auth-service | ||
port: | ||
number: 8080 | ||
# Client uses secret url to access the RPC endpoint | ||
- path: /tezos-node-rpc/(.*) | ||
pathType: Exact | ||
backend: | ||
service: | ||
name: rpc-auth-service | ||
port: | ||
number: 8080 | ||
--- | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
version: v1beta9 | ||
deployments: | ||
- name: backend | ||
kubectl: | ||
manifests: | ||
- ./backend.yaml | ||
images: | ||
server: | ||
image: rpc-auth | ||
dev: | ||
sync: | ||
- imageName: server | ||
labelSelector: | ||
app: rpc-auth | ||
hooks: | ||
- command: minikube | ||
args: | ||
- addons | ||
- enable | ||
- ingress | ||
when: | ||
before: | ||
deployments: all |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
k8s noob question: at what point in time do we configure resources for all of our different infrastructure objects? How do we determine what the resources should be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's important to set resources once we are in production and have metrics collection in place. The main benefit is to have sensible autoscaling behaviour. In a minikube context it matters less since there is only one node anyway.