Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Public networks #117

Merged
merged 19 commits into from
Mar 2, 2021
Merged

Public networks #117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
733ecee
support for public networks
nicolasochem Feb 17, 2021
53ee815
Merge branch 'master' into public_networks
nicolasochem Feb 17, 2021
bba4b2a
Merge branch 'master' into public_networks
nicolasochem Feb 22, 2021
24f724f
run tezos-node config init and extract bootstrap ppers
nicolasochem Feb 23, 2021
5bc4c0f
add snapshot downloader
nicolasochem Feb 24, 2021
874e07c
switch to tezos container forces us to use sudo
nicolasochem Feb 26, 2021
e3201f1
I found a build which works well with edo2net; let's make it default
nicolasochem Feb 26, 2021
41d3cfa
remove exec so the snapshot file is actually deleted
nicolasochem Feb 26, 2021
80f62d0
re-add baker removed by mistake
nicolasochem Feb 27, 2021
776653b
add mkchain support
nicolasochem Feb 27, 2021
5007325
do not error when not public network
nicolasochem Feb 28, 2021
d6e33ef
move default protocol to edo2
nicolasochem Feb 28, 2021
4a098f1
don't put history mode in rolling if not public node
nicolasochem Feb 28, 2021
32b222e
point baker to correct data dir
nicolasochem Mar 1, 2021
bdd4f97
make config container slimmer
nicolasochem Mar 1, 2021
14a9297
revert custom version
nicolasochem Mar 1, 2021
7d851b2
lint
nicolasochem Mar 1, 2021
0ab21fc
restore default values.yaml to isolated chain
nicolasochem Mar 1, 2021
9fcb451
move wait-for-bootstrap back to the first init container
nicolasochem Mar 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,12 +130,22 @@ Set [unbuffered IO](https://docs.python.org/3.6/using/cmdline.html#envvar-PYTHON
export PYTHONUNBUFFERED=x
```

## Start your chain
## Chain types

There are 3 types of chains supported:

* **isolated**: self-sufficient private chain. Does not connect to the outside world,
* **private**: private chain on a zerotier network. Can span multiple clusters,
* **public**: public chain such as a public testnet or Tezos mainnet.

The `--chain-type` parameter lets you choose between these different types.

## Start your private chain

Run the following commands to create the Helm values, get the Helm chart repo, and install the Helm chart to start your chain.

```shell
mkchain $CHAIN_NAME --zerotier-network $ZT_NET --zerotier-token $ZT_TOKEN
mkchain $CHAIN_NAME --zerotier-network $ZT_NET --zerotier-token $ZT_TOKEN --chain-type private

helm repo add tqtezos https://tqtezos.github.io/tezos-helm-charts

Expand Down
3 changes: 2 additions & 1 deletion baker-endorser/entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ TEZ_VAR=/var/tezos
TEZ_BIN=/usr/local/bin
CLIENT_DIR="$TEZ_VAR/client"
NODE_DIR="$TEZ_VAR/node"
NODE_DATA_DIR="$TEZ_VAR/node/data"

proto_command=$(echo $CHAIN_PARAMS | jq -r '.proto_command')
if [ "${DAEMON}" == "baker" ]; then
extra_args="with local node $NODE_DIR"
extra_args="with local node $NODE_DATA_DIR"
fi
POD_INDEX=$(echo $POD_NAME | sed -e s/tezos-baking-node-//)
baker_account=$(echo $NODES | jq -r ".baking[${POD_INDEX}].bake_for")
Expand Down
32 changes: 16 additions & 16 deletions charts/tezos/templates/baker.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if not .Values.is_invitation }}
{{- if and (not .Values.is_invitation) (not (eq .Values.chain_type "public")) }}
apiVersion: batch/v1
kind: Job
metadata:
Expand Down Expand Up @@ -129,7 +129,7 @@ spec:
fieldPath: metadata.name
- name: DAEMON
value: endorser
{{- if .Values.zerotier_in_use }}
{{- if eq .Values.chain_type "private" }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should make the chain_types constants somehow. This way it is explicit what are acceptable values. And also so that we don’t have to keep referencing strings everywhere which can very easily introduce bugs. (e.g. hashs/hashes)

We can maybe have 3 (maybe just 2) vals: is_private, is_isolated, is_public. We can prob just go with the first 2. If NOT private AND NOT isolated, then it must be public. These vals get set by mkchain or whatever.

Or we can keep the eq checks in Helm templates but set “private” as a string Helm value. so private: "private" and check based off of that constant.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should have is_isolated and is_private because then they would not be mutually exclusive.

What do you recommend? Something like:

{{ $chain_type_private := "private }}
{{ $chain_type_isolated := "isolated" }}
{{ $chain_type_public := "public" }}

Then

{{- if eq .Values.chain_type $chain_type_private }}

?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes something like that although I don't think that exact thing will work. Those variables won't be global to all templates.

I think there are a couple of options:

  • Define global values: https://helm.sh/docs/chart_template_guide/subcharts_and_globals/#global-chart-values
  • Define helpers in a .tpl file in charts/tezos/templates. This will contain templates with something like this: 
    {{ define "tezos.public_chain" }}
{{- "public" -}}
{{ end }}
    Then we can compare on including the template. I think there is a way to treat the template like a function where you can pass a context in and let it spit back out a value like true/false that we can then just if on.

Either way, probably not necessary for getting the F-beta net out the door.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's iterate on this in another PR

- args:
- "-c"
- "echo 'starting zerotier' && zerotier-one /var/tezos/zerotier"
Expand All @@ -150,19 +150,6 @@ spec:
name: var-volume
{{- end }}
initContainers:
- args:
- "-c"
- "[ -f /var/tezos/node/identity.json ] || (mkdir -p /var/tezos/node && /usr/local/bin/tezos-node identity generate 0 --data-dir /var/tezos/node --config-file /etc/tezos/config.json)"
command:
- /bin/sh
image: {{ .Values.images.tezos | quote }}
name: identity-job
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /etc/tezos
name: config-volume
- mountPath: /var/tezos
name: var-volume
{{- if not .Values.is_invitation }}
- image: {{ .Values.tezos_k8s_images.wait_for_bootstrap }}
imagePullPolicy: IfNotPresent
Expand All @@ -171,7 +158,7 @@ spec:
- mountPath: /var/tezos
name: var-volume
{{- end }}
{{- if .Values.zerotier_in_use }}
{{- if eq .Values.chain_type "private" }}
- envFrom:
- configMapRef:
name: zerotier-config
Expand Down Expand Up @@ -216,6 +203,19 @@ spec:
name: config-volume
- mountPath: /var/tezos
name: var-volume
{{- if eq .Values.chain_type "public" }}
- image: "{{ .Values.tezos_k8s_images.snapshot_downloader }}"
imagePullPolicy: IfNotPresent
name: snapshot-downloader
volumeMounts:
- mountPath: /var/tezos
name: var-volume
- mountPath: /etc/tezos
name: config-volume
envFrom:
- configMapRef:
name: tezos-config
{{- end }}
securityContext:
fsGroup: 100
volumes:
Expand Down
4 changes: 3 additions & 1 deletion charts/tezos/templates/configs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ data:
"genesis_block": "{{ .Values.genesis.block }}",
"activation_account": "{{ (index .Values.accounts 0).name }}",
"timestamp": "{{ .Values.genesis.timestamp }}",
"zerotier_in_use": {{ .Values.zerotier_in_use }},
"chain_type": "{{ .Values.chain_type }}",
"network": "{{ .Values.network }}",
"snapshot_url": "{{ .Values.snapshot_url }}",
"proto_command": "{{ .Values.protocol.command }}",
"protocol_hash": "{{ .Values.protocol.hash }}"
}
Expand Down
30 changes: 15 additions & 15 deletions charts/tezos/templates/node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ spec:
name: config-volume
- mountPath: /var/tezos
name: var-volume
{{- if .Values.zerotier_in_use }}
{{- if eq .Values.chain_type "private" }}
- args:
- "-c"
- "echo 'starting zerotier' && zerotier-one /var/tezos/zerotier"
Expand All @@ -63,19 +63,6 @@ spec:
name: var-volume
{{- end }}
initContainers:
- args:
- "-c"
- "[ -f /var/tezos/node/identity.json ] || (mkdir -p /var/tezos/node && /usr/local/bin/tezos-node identity generate 0 --data-dir /var/tezos/node --config-file /etc/tezos/config.json)"
command:
- /bin/sh
image: {{ .Values.images.tezos | quote }}
name: identity-job
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /etc/tezos
name: config-volume
- mountPath: /var/tezos
name: var-volume
{{- if not .Values.is_invitation }}
- image: {{ .Values.tezos_k8s_images.wait_for_bootstrap }}
imagePullPolicy: IfNotPresent
Expand All @@ -84,7 +71,7 @@ spec:
- mountPath: /var/tezos
name: var-volume
{{- end }}
{{- if .Values.zerotier_in_use }}
{{- if eq .Values.chain_type "private" }}
- envFrom:
- configMapRef:
name: zerotier-config
Expand Down Expand Up @@ -129,6 +116,19 @@ spec:
name: config-volume
- mountPath: /var/tezos
name: var-volume
{{- if eq .Values.chain_type "public" }}
- image: "{{ .Values.tezos_k8s_images.snapshot_downloader }}"
imagePullPolicy: IfNotPresent
name: snapshot-downloader
volumeMounts:
- mountPath: /var/tezos
name: var-volume
- mountPath: /etc/tezos
name: config-volume
envFrom:
- configMapRef:
name: tezos-config
{{- end }}
securityContext:
fsGroup: 100
volumes:
Expand Down
8 changes: 5 additions & 3 deletions charts/tezos/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ baker: true
chain_name: YOUR_CHAIN_NAME_HERE
is_invitation: false
rpc_auth: false
zerotier_in_use: false
chain_type: isolated
zerotier_config:
zerotier_network: YOUR_NETWORK_ID_HERE
zerotier_token: YOUR_ZEROTIER_TOKEN_HERE
Expand All @@ -17,11 +17,13 @@ tezos_k8s_images:
config_generator: tezos-k8s-config-generator:dev
wait_for_bootstrap: tezos-k8s-wait-for-bootstrap:dev
zerotier: tezos-k8s-zerotier:dev
snapshot_downloader: tezos-k8s-snapshot-downloader:dev

snapshot_url: https://mainnet.xtz-shots.io/rolling
protocol:
bootstrap_mutez: "4000000000000"
hash: PsDELPH1Kxsxt8f9eWbxQeRxkjfbxoqM52jvs5Y5fBxWWh4ifpo
command: 007-PsDELPH1
hash: PtEdo2ZkT9oKpimTah6x2embF25oss54njMuPzkJTEi5RqfdZFA
command: 008-PtEdo2Zk
genesis:
block: "YOUR_GENESIS_CHAIN_ID_HERE"
timestamp: '2021-02-12T17:02:24.061398+00:00'
Expand Down
13 changes: 9 additions & 4 deletions config-generator/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
FROM python:alpine
FROM tezos/tezos:v8-release
ENV PYTHONUNBUFFERED=1
RUN apk add --no-cache --virtual .build-deps gcc musl-dev libffi-dev make \

# Note: we install build deps for pip, then remove everything after pip install.
# Then we reinstall runtime deps at the end.
RUN sudo apk add --no-cache --virtual .build-deps gcc py3-pip python3-dev musl-dev libffi-dev make \
&& pip install base58 pynacl \
&& apk del .build-deps gcc musl-dev libffi-dev make
&& sudo apk del .build-deps gcc musl-dev libffi-dev make \
&& sudo apk add --no-cache --virtual install jq gcc python3 py3-pip
COPY entrypoint.py /
ENTRYPOINT ["python", "/entrypoint.py"]
COPY entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD []
Loading