Skip to content

feat: updated infra privacy to support privacy set up #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mikelle merged 13 commits into from
May 15, 2024
Merged

feat: updated infra privacy to support privacy set up #56

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
66a821e
fix: updating infra for privacy
Mikelle May 13, 2024
a56bfe3
fix: deleted secrets
Mikelle May 13, 2024
e7b44aa
fix: added raw
Mikelle May 13, 2024
40af364
fix: updated nomad
Mikelle May 14, 2024
2bfb887
fix: fixed job env
Mikelle May 14, 2024
d0a2bae
fix: fixing oracle nomad job
Mikelle May 14, 2024
b2b4b88
fix: updated order
Mikelle May 14, 2024
01bd862
fix: updated infra
Mikelle May 14, 2024
242a571
fix: deleted comment for datadog logs
Mikelle May 14, 2024
8e70903
chore: merge branch 'main' into infra-privacy
Mikelle May 14, 2024
1ffd114
fix: fixes after merge
Mikelle May 14, 2024
290f0f8
Merge branch 'main' into infra-privacy
Mikelle May 15, 2024
47e8d3a
chore: get rid of contracts hardcode
Mikelle May 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions contracts-abi/config/testnet.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ type Contracts struct {
}

var TestnetContracts = Contracts{
BidderRegistry: "0xded9029fC3789ED393D62686c0c0f9dfA92aA2f6",
ProviderRegistry: "0xFA19327bDBf2632aAB7C77e61DC69DbC872d5AC1",
PreconfCommitmentStore: "0x1F8989fAd5f0538D794Fd9fa15d50942F305f367",
Oracle: "0x1cB85eC90320Ef25FB4F991E41392f518980e53a",
BlockTracker: "0xCB4AA84C916BB891cBF43320e0c97C3d4329Cec7",
BidderRegistry: "0x1E218818D409E0f00dfeBE8A960F7585d4fDff70",
ProviderRegistry: "0x0332388390d9df01cA3d26269f2B1Fc314deD9c0",
PreconfCommitmentStore: "0x2Aff805aBdF1Fe79AfcF8B3a9B4B45ECcD6b6D6e",
Oracle: "0x77A4FE615de28fdf0bF68D9B9ba773A32b5C7630",
BlockTracker: "0x042744D8cF66d8455350D43F9e09CA73b5C0CB94",
}
56 changes: 28 additions & 28 deletions infrastructure/nomad/deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,33 +8,33 @@
profiles:
devnet:
job_names:
- 'datadog-agent-logs-collector'
- 'mev-commit-geth-bootnode1'
- 'mev-commit-geth-signer-node1'
- 'mev-commit-geth-member-node'
- 'deploy-contracts'
- 'mev-commit-bootnode1'
- 'mev-commit-provider-node1'
- 'mev-commit-provider-node1-funder'
- 'mev-commit-bidder-node1'
- 'mev-commit-bidder-node1-funder'
- 'mev-commit-provider-emulator-node1'
- 'mev-commit-bidder-emulator-node1'
- 'mev-commit-oracle'
- 'datadog-agent-metrics-collector'
- "datadog-agent-logs-collector"
- "mev-commit-geth-bootnode1"
- "mev-commit-geth-signer-node1"
- "mev-commit-geth-member-node"
- "deploy-contracts"
- "mev-commit-bootnode1"
- "mev-commit-provider-node1"
- "mev-commit-provider-node1-funder"
- "mev-commit-oracle"
- "mev-commit-bidder-node1"
- "mev-commit-bidder-node1-funder"
- "mev-commit-provider-emulator-node1"
- "mev-commit-bidder-emulator-node1"
- "datadog-agent-metrics-collector"
testnet:
job_names:
- 'datadog-agent-logs-collector'
- 'mev-commit-geth-bootnode1'
- 'mev-commit-geth-signer-node1'
- 'mev-commit-geth-member-node'
- 'deploy-contracts'
- 'mev-commit-bootnode1'
- 'mev-commit-provider-node1'
- 'mev-commit-provider-node1-funder'
- 'mev-commit-provider-emulator-node1'
- 'mev-commit-oracle'
- 'datadog-agent-metrics-collector'
- "datadog-agent-logs-collector"
- "mev-commit-geth-bootnode1"
- "mev-commit-geth-signer-node1"
- "mev-commit-geth-member-node"
- "deploy-contracts"
- "mev-commit-bootnode1"
- "mev-commit-provider-node1"
- "mev-commit-provider-node1-funder"
- "mev-commit-provider-emulator-node1"
- "mev-commit-oracle"
- "datadog-agent-metrics-collector"

vars_prompt:
- name: version
Expand Down Expand Up @@ -99,7 +99,7 @@
ansible.builtin.file:
path: "{{ ansible_env.HOME }}/{{ env }}"
state: directory
mode: '0744'
mode: "0744"
recurse: yes

- name: Build | Nomad artifacts
Expand All @@ -120,7 +120,7 @@
loop:
- infrastructure/tools/keystore-generator
- p2p/integrationtest/provider
- p2p/integrationtest/bidder
- p2p/integrationtest/real-bidder
- bridge/standard/bridge-v1
- external/geth
- contracts
Expand Down Expand Up @@ -174,7 +174,7 @@
- name: Deploy | Nomad jobs
ansible.builtin.shell: |
nomad run {{ ansible_env.HOME }}/{{ env }}/{{ job.name }}.nomad
[ "{{ job.name }}" = "deploy-contracts" ] && sleep 180 || true
[ "{{ job.name }}" = "deploy-contracts" ] && sleep 240 || true
Copy link
Contributor

@shaspitz shaspitz May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why this sleep time needed to be increased?

Copy link
Contributor Author

@Mikelle Mikelle May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

180s were not enough, but with 240s it's more reliable; contracts have enough time to be deployed with that timing.

args:
executable: bash
loop: "{{ jobs }}"
Expand Down
12 changes: 2 additions & 10 deletions infrastructure/nomad/templates/nomad/mev-commit-emulator.nomad.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,7 @@ job "{{ job.name }}" {
{{- end }}
{% endraw %}
{% if job.target_type == 'bidder' %}
{%- raw %}
{{ range nomadService "mev-commit-geth-bootnode1" }}
{{- if contains "http" .Tags }}
EMULATOR_SETTLEMENT_RPC_ENDPOINT="http://{{ .Address }}:{{ .Port }}"
{{ end }}
{{ end }}
{% endraw %}
EMULATOR_L1_RPC_URL="{{ job.env['l1_rpc_url'] }}"
{% endif %}
EOH
destination = "secrets/.env"
Expand All @@ -86,10 +80,8 @@ job "{{ job.name }}" {
${EMULATOR_BINARY} \
-server-addr "${EMULATOR_IP_PORT}" \
{% if job.target_type == 'bidder' %}
-rpc-addr "${EMULATOR_SETTLEMENT_RPC_ENDPOINT}" \
Copy link
Contributor

@shaspitz shaspitz May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the emulator was previously (incorrectly) querying the mev-commit chain instead of L1?

Copy link
Contributor Author

@Mikelle Mikelle May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we found out this with Alok

-rpc-addr "${EMULATOR_L1_RPC_URL}" \
{% endif %}
-log-tags "${EMULATOR_LOG_TAGS}" \
-log-fmt "${EMULATOR_LOG_FMT}"
EOH
destination = "local/run.sh"
perms = "0755"
Expand Down
2 changes: 1 addition & 1 deletion infrastructure/nomad/templates/nomad/mev-commit-funder.nomad.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ job "{{ job.name }}" {
local/cast send \
--rpc-url http://{{ .Address }}:{{ .Port }} \
--private-key 0x7c9bf0f015874594d321c1c01ada3166c3509bbd91f76f9e4d7380c2df269c55 ${ETHEREUM_ADDRESS} \
--value 100ether
--value 1000ether
{{- end }}
{{- end }}
{% endraw %}
Expand Down
22 changes: 19 additions & 3 deletions infrastructure/nomad/templates/nomad/mev-commit-oracle.nomad.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,12 @@ job "{{ job.name }}" {
else 'service:' + job.name + '-{{ env "NOMAD_ALLOC_INDEX" }}'
}}"
MEV_ORACLE_LOG_LEVEL="{{ job.env.get('log-level', 'info') }}"
MEV_ORACLE_LAGGERD_MODE="{{ job.env.get('laggerd-mode', '64') }}"
MEV_ORACLE_LAGGERD_MODE="{{ job.env.get('laggerd-mode', '20') }}"
MEV_ORACLE_L1_RPC_URL="{{ job.env['l1_rpc_url'] }}"
{%- raw %}
MEV_ORACLE_KEYSTORE_PATH="/local/data-{{env "NOMAD_ALLOC_INDEX"}}/keystore"
MEV_ORACLE_KEYSTORE_PASSWORD="{{ with secret "secret/data/mev-commit" }}{{ .Data.data.oracle_keystore_password }}{{ end }}"
# MEV_ORACLE_KEYSTORE_PATH="/local/data-{{env "NOMAD_ALLOC_INDEX"}}/keystore"
# MEV_ORACLE_KEYSTORE_PASSWORD="{{ with secret "secret/data/mev-commit" }}{{ .Data.data.oracle_keystore_password }}{{ end }}"
MEV_ORACLE_PRIV_KEY_FILE = "/local/data-{{env "NOMAD_ALLOC_INDEX"}}/priv-key"
{{- range nomadService "mev-commit-oracle" }}
{{- if contains "http" .Tags }}
MEV_ORACLE_HTTP_PORT="{{ .Port }}"
Expand All @@ -148,6 +149,9 @@ job "{{ job.name }}" {
{{- if contains "http" .Tags }}
MEV_ORACLE_SETTLEMENT_RPC_URL="http://{{ .Address }}:{{ .Port }}"
{{- end }}
{{- if contains "ws" .Tags }}
MEV_ORACLE_SETTLEMENT_WS_RPC_ENDPOINT="ws://{{ .Address}}:{{ .Port }}"
{{- end }}
{{- end }}
{{- range nomadService "{% endraw %}{{ job.name }}{% raw %}" }}
{{- if contains "db" .Tags }}
Expand All @@ -167,6 +171,15 @@ job "{{ job.name }}" {
template {
data = <<-EOH
#!/usr/bin/env bash
{%- raw %}
{{- range nomadService "mev-commit-provider-node1" }}
{{ if contains "http" .Tags }}
TOPOLOGY=$(curl https://{{ .Address}}:{{ .Port }}/topology)
PROVIDER_ETHEREUM_ADDRESS=$(echo ${TOPOLOGY} | jq -r '.self["Ethereum Address"]')
export MEV_ORACLE_OVERRIDE_WINNERS=${PROVIDER_ETHEREUM_ADDRESS}
{{ end }}
{{- end }}
{% endraw %}

{%- raw %}
{{- range nomadService "datadog-agent-logs-collector" }}
Expand All @@ -185,6 +198,9 @@ job "{{ job.name }}" {
fi

mkdir -p $(dirname "${MEV_ORACLE_PRIV_KEY_FILE}") > /dev/null 2>&1
{%- raw %}
echo {{ with secret "secret/data/mev-commit" }}{{ .Data.data.oracle_private_key }}{{ end }} > ${MEV_ORACLE_PRIV_KEY_FILE}
{% endraw %}
chmod +x local/mev-commit-oracle
local/mev-commit-oracle start
EOH
Expand Down
3 changes: 3 additions & 0 deletions infrastructure/nomad/templates/nomad/mev-commit.nomad.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ job "{{ job.name }}" {
{{- if contains "http" .Tags }}
MEV_COMMIT_SETTLEMENT_RPC_ENDPOINT="http://{{ .Address }}:{{ .Port }}"
{{- end }}
{{- if contains "ws" .Tags }}
MEV_COMMIT_SETTLEMENT_WS_RPC_ENDPOINT="ws://{{ .Address}}:{{ .Port }}"
{{- end }}
{{- end }}
{% endraw %}
MEV_COMMIT_PEER_TYPE="{{ job.env['type'] }}"
Expand Down
43 changes: 17 additions & 26 deletions infrastructure/nomad/vars.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ jobs:
http:
static: 8545
to: 8545
ws:
static: 8546
to: 8546
p2p:
to: 30301
env:
Expand All @@ -49,21 +52,6 @@ jobs:
sync_mode: snap
block_address: "0xd9cd8E5DE6d55f796D980B818D350C0746C25b97"

- name: mev-commit-geth-signer-node2
template: mev-commit-geth.nomad.j2
count: 1
ports:
- metrics:
to: 6060
p2p:
to: 30311
env:
ip: 0.0.0.0
net_restrict: 0.0.0.0/0
type: signer
sync_mode: snap
block_address: "0x788EBABe5c3dD422Ef92Ca6714A69e2eabcE1Ee4"

- name: mev-commit-geth-member-node
template: mev-commit-geth.nomad.j2
count: 1
Expand Down Expand Up @@ -137,6 +125,19 @@ jobs:
target_type: provider
target_name: mev-commit-provider-node1

- name: mev-commit-oracle
template: mev-commit-oracle.nomad.j2
count: 1
ports:
- db:
static: 5432
to: 5432
http:
static: 8080
to: 8080
env:
l1_rpc_url: https://ethereum-holesky-rpc.publicnode.com

- name: mev-commit-bidder-node1
template: mev-commit.nomad.j2
count: 1
Expand Down Expand Up @@ -181,18 +182,8 @@ jobs:
ports:
- metrics:
to: 8080

- name: mev-commit-oracle
template: mev-commit-oracle.nomad.j2
count: 1
ports:
- db:
static: 5432
to: 5432
http:
to: 8080
env:
l1_rpc_url: https://rpc.sepolia.org
l1_rpc_url: https://ethereum-holesky-rpc.publicnode.com

- name: mev-commit-bridge
template: mev-commit-bridge.nomad.j2
Expand Down