Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark single transfer per node #267

Merged
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
d490675
228_first_commit
apoorv-2204 Mar 28, 2022
fedd5fb
added Static faucet provider
apoorv-2204 Mar 30, 2022
aa234f6
Moves aliases to comments
apoorv-2204 Mar 30, 2022
a7ab2d0
Addition of Mix aliases Tasks mix dev._
apoorv-2204 Mar 30, 2022
0935710
Augmented with Internal Faucet
apoorv-2204 Mar 30, 2022
c9d729f
Described more aliases
apoorv-2204 Mar 30, 2022
ef03b3c
end of day 30march22 , nochecks
apoorv-2204 Mar 30, 2022
ad15d48
building faucet txn
apoorv-2204 Mar 31, 2022
135a695
Retreive Funds from faucetendpoint, thisnode, via host
apoorv-2204 Mar 31, 2022
bb550c9
Txn creation
apoorv-2204 Mar 31, 2022
a0b343d
improve PBkey Prkey generation
apoorv-2204 Mar 31, 2022
093a445
Refactoring, TOdo: Redesign Play.sendtxn
apoorv-2204 Apr 1, 2022
84dac63
code improvement
apoorv-2204 Apr 1, 2022
e50fb89
tps scenario benchee
apoorv-2204 Apr 1, 2022
7844518
benchmarking
apoorv-2204 Apr 1, 2022
524c6ed
each New(A to b) tarnsaction
apoorv-2204 Apr 4, 2022
ffc3b9e
eod commit 6-04-22
apoorv-2204 Apr 6, 2022
7e713eb
Starting state of branch 228TPS
apoorv-2204 Apr 7, 2022
295e8f8
RAndom A to b o/p s
apoorv-2204 Apr 7, 2022
5abf2e0
fix mix format
apoorv-2204 Apr 7, 2022
411a197
removed benchmark outputs
apoorv-2204 Apr 8, 2022
a870e66
exposed api implementation
apoorv-2204 Apr 11, 2022
9f7f78d
Merge branch 'develop' of https://github.com/archethic-foundation/arc…
apoorv-2204 Apr 11, 2022
3273d38
integeration_works_fine
apoorv-2204 Apr 11, 2022
a5d3538
verify-replciation not tested
apoorv-2204 Apr 11, 2022
b6f894d
query timeout
apoorv-2204 Apr 12, 2022
27c4912
updated port : fixed timeout error
apoorv-2204 Apr 12, 2022
cde4e23
eod 120422
apoorv-2204 Apr 12, 2022
cda8f10
eod 140422
apoorv-2204 Apr 14, 2022
b659311
finding ws lib
apoorv-2204 Apr 15, 2022
8f654e1
mix.lock mint 1.0 to 1.1 , remaining libs in mix lock back to default
apoorv-2204 Apr 15, 2022
efcab9a
eod
apoorv-2204 Apr 15, 2022
6fa003e
webclient error
apoorv-2204 Apr 16, 2022
f8938c8
ws
apoorv-2204 Apr 18, 2022
09df9ec
___
apoorv-2204 Apr 19, 2022
f3d0aae
Add GraphqlClient
blackode Apr 22, 2022
d5de7f8
Update the graphql client calling
blackode Apr 22, 2022
c5de6ce
removed from supervison tree
apoorv-2204 Apr 25, 2022
785c2be
rename the modules
apoorv-2204 Apr 25, 2022
e7f53ea
correct typos
apoorv-2204 Apr 25, 2022
be15715
via playbook error no timeout thrown
apoorv-2204 Apr 25, 2022
3aaf0e2
moved Graphqlserver api to benchmark
apoorv-2204 Apr 25, 2022
a6da6dc
ioinspect to logger debug
apoorv-2204 Apr 25, 2022
d5870c4
new
apoorv-2204 Apr 26, 2022
4e04161
rebase
apoorv-2204 Apr 26, 2022
8c0230c
rebase2
apoorv-2204 Apr 26, 2022
4e35d97
Merge branch 'temp' into 228_benchmark_tps_per_node
apoorv-2204 Apr 26, 2022
3020789
rebase errors
apoorv-2204 Apr 26, 2022
409b9fc
fix errors
apoorv-2204 Apr 26, 2022
8d0d530
finding ws lib
apoorv-2204 Apr 27, 2022
4465312
added GQL SUB CLIENT || EOD(27/04/22)
apoorv-2204 Apr 27, 2022
cb5cc84
eod 27.04.22
apoorv-2204 Apr 27, 2022
3d8bed4
WS integeration with playbook
apoorv-2204 Apr 27, 2022
d64f34f
init: integeration with via helpers
apoorv-2204 Apr 27, 2022
3c0b08d
close: Via Tps helper | Drop: via Tps helper | refactor ws_client
apoorv-2204 Apr 28, 2022
e752f25
resolved git hooks | removed Poison
apoorv-2204 Apr 28, 2022
aeb691c
added untested ingestion and completion benchmark code
apoorv-2204 Apr 29, 2022
b6b17a8
added before scenario for ingestion and completeion
apoorv-2204 Apr 29, 2022
c87f1aa
Refactor to simplify and fix websocket handling
May 2, 2022
b850451
Fix ledger movements resolution to add bburn movement
May 2, 2022
680088a
Reset formatting
May 3, 2022
8e99f8f
Unlock unused deps
May 3, 2022
42ea6a7
Merge branch 'develop' into 228_benchmark_tps_per_node
May 3, 2022
861cec5
Fix renaming
May 3, 2022
7355461
Resolve transaction recipients by time
May 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/archethic/utils/regression.ex
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ defmodule ArchEthic.Utils.Regression do
alias ArchEthic.Utils.WebClient

@playbooks [UCO, SmartContract]
@benchmarks [P2PMessage,TPS]
@benchmarks [P2PMessage, TPS]

def run_playbooks(nodes, opts \\ []) do
Logger.debug("Running playbooks on #{inspect(nodes)} with #{inspect(opts)}")
Expand Down
67 changes: 0 additions & 67 deletions lib/archethic/utils/regression/benchmarks/helpers.ex

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
defmodule ArchEthic.Utils.Regression.Benchmarks.Helpers.FaucetEndpoint do
@moduledoc """
Contacts Testnet Faucet For UCOs
"""

def main(recipient_address) do
withdraw_uco({:https, "testnet.archethic.net", 443}, recipient_address)
apoorv-2204 marked this conversation as resolved.
Show resolved Hide resolved
end

def withdraw_uco({type, url, http_port}, wallet_address) do
with {:ok, conn_ref} <- establish_connection(type, url, http_port),
{:ok, conn, _req_ref} <- request(conn_ref, wallet_address) do
stream_responses(conn)
end
end

defp establish_connection(type, url, http_port) do
apoorv-2204 marked this conversation as resolved.
Show resolved Hide resolved
Mint.HTTP.connect(type, url, http_port)
end

defp request(conn_ref, wallet_address) do
{body, content_length} = get_request_body(wallet_address)
header = get_request_header(content_length)

Mint.HTTP.request(
conn_ref,
_method_type = "POST",
_path = "/faucet",
_header = header,
_body = body
)
end

def get_request_header(content_length) do
[
{"Content-Type", "application/x-www-form-urlencoded"},
{"Content-Length", "#{content_length}"},
{"cookie",
"_archethic_key=SFMyNTY.g3QAAAABbQAAAAtfY3NyZl90b2tlbm0AAAAYbnFtT0lVWE9ZZmVIVHZHQVdHenJKTzh2.AuLNklbvM6lHiGY2yGN6mmoDBNrcGqSL5v206ghydFs"}
]
end

def get_request_body(wallet_address) do
body =
"_csrf_token=DCcfJAc6Bz1sVl0ebAx-NhIlSgMMIH8DbVrkNo_r508V8z9wEb0qFoGu&address=#{wallet_address}"

{body, body |> :erlang.byte_size()}
end

def stream_responses(conn) do
receive do
message ->
with {:ok, conn, [{:status, _, 200}, {:headers, _, _}, {:data, _, _}, {:done, _}]} <-
Mint.HTTP.stream(conn, message),
{:ok, _} <- Mint.HTTP.close(conn) do
{:ok, :transferred}
end
after
5_000 ->
{:error, :timeout}
end
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
defmodule ArchEthic.Utils.Regression.Benchmarks.Helpers.InternalFaucet do
@moduledoc """
Executes UCO withrawl internally to the Tx-blockchain.
"""

alias ArchEthicWeb.TransactionSubscriber
alias ArchEthic.Crypto

alias ArchEthic.TransactionChain.{
Transaction,
TransactionData,
TransactionData.Ledger,
TransactionData.UCOLedger
}

alias ArchEthicWeb.TransactionSubscriber

@pool_seed Application.compile_env(:archethic, [ArchEthicWeb.FaucetController, :seed])

def main(recipient_address) do
apoorv-2204 marked this conversation as resolved.
Show resolved Hide resolved
initiate_transfer(recipient_address)
end

defp initiate_transfer(wallet_address) do
# if Application.get_env(:archethic, ArchEthicWeb.FaucetController)
# |> Keyword.get(:enabled, false) do
create_transfer(wallet_address)
# else
# {:error, :faucet_disabled}
# end
end

def create_transfer(address) do
with {:ok, recipient_address} <- Base.decode16(address, case: :mixed),
true <- Crypto.valid_address?(recipient_address),
{:ok, tx_address} <- transfer(recipient_address) do
TransactionSubscriber.register(tx_address, System.monotonic_time())
{:ok, :transferred}
else
_ -> {:error, :failure}
end
end

defp transfer(
recipient_address,
curve \\ Crypto.default_curve()
)
when is_bitstring(recipient_address) do
{gen_pub, _} = Crypto.derive_keypair(@pool_seed, 0, curve)

pool_gen_address = Crypto.derive_address(gen_pub)

with {:ok, last_address} <-
ArchEthic.get_last_transaction_address(pool_gen_address),
{:ok, last_index} <- ArchEthic.get_transaction_chain_length(last_address) do
create_transaction(last_index, curve, recipient_address)
else
{:error, _} = e ->
e
end
end

defp create_transaction(transaction_index, curve, recipient_address) do
tx =
Transaction.new(
:transfer,
%TransactionData{
ledger: %Ledger{
uco: %UCOLedger{
transfers: [
%UCOLedger.Transfer{
to: recipient_address,
amount: 10_000_000_000
}
]
}
}
},
@pool_seed,
transaction_index,
curve
)

case ArchEthic.send_new_transaction(tx) do
:ok ->
{:ok, tx.address}

{:error, _} = e ->
e
end
end
end
25 changes: 25 additions & 0 deletions lib/archethic/utils/regression/benchmarks/helpers/tps_helper.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
defmodule ArchEthic.Utils.Regression.Benchmarks.Helpers.TPSHelper do
@moduledoc """
Helpers Methods to carry out the TPS benchmarks
"""

def main() do
recipient_address = "0000c084b09c60e3bde2d0a81df08b20d82d8b6dfc1d39bc3dfa5e41b731718f09e1"
withdraw_uco_via_this_node(recipient_address)
end

def withdraw_uco_from_testnet_endpoint(recipient_address) do
apoorv-2204 marked this conversation as resolved.
Show resolved Hide resolved
alias ArchEthic.Utils.Regression.Benchmarks.Helpers.FaucetEndpoint
FaucetEndpoint.main(recipient_address)
end

def withdraw_uco_via_this_node(recipient_address) do
alias ArchEthic.Utils.Regression.Benchmarks.Helpers.InternalFaucet
InternalFaucet.main(recipient_address)
end

def withdraw_uco_via_host(recipient_address, host, port) do
apoorv-2204 marked this conversation as resolved.
Show resolved Hide resolved
alias ArchEthic.Utils.Regression.Playbook
Playbook.send_funds_to(recipient_address, host, port)
end
end
9 changes: 4 additions & 5 deletions lib/archethic/utils/regression/benchmarks/tps.ex
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
defmodule ArchEthic.Utils.Regression.Benchmark.TaskSupervisor
@moduledoc """
defmodule ArchEthic.Utils.Regression.Benchmark.TPS do
@moduledoc """
Module for regession testing the paralleing processing of transactions and
benchmarking the parallel transaction processing capability of a node
"""

require Logger

alias ArchEthic.Utils.Regression.Benchmark
# alias ArchEthic.Utils.Regression.Benchmarks.Helpers.TPSHelper

@behaviour Benchmark

def plan([host | _nodes], _opts) do
:rip
def plan([_host | _nodes], _opts) do
end

end
4 changes: 3 additions & 1 deletion mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ defmodule ArchEthic.MixProject do
{:gen_stage, "~> 1.1"},
{:flow, "~> 1.0"},
{:broadway, "~> 1.0"},
{:knigge, "~> 1.4"}
{:knigge, "~> 1.4"},
# required for mint https requests
{:castore, "~> 0.1.0"}
]
end

Expand Down
1 change: 1 addition & 0 deletions mix.lock
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
"blankable": {:hex, :blankable, "1.0.0", "89ab564a63c55af117e115144e3b3b57eb53ad43ba0f15553357eb283e0ed425", [:mix], [], "hexpm", "7cf11aac0e44f4eedbee0c15c1d37d94c090cb72a8d9fddf9f7aec30f9278899"},
"broadway": {:hex, :broadway, "1.0.1", "7b4ca0b439a425730b5fc1bf06aae350df6171434fd4f29bdbbe50d2d9d518fb", [:mix], [{:gen_stage, "~> 1.0", [hex: :gen_stage, repo: "hexpm", optional: false]}, {:nimble_options, "~> 0.3.7", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.3 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "6d1c7fdc100ba484a477c42881553dfa1730f31c461308286132efbfeb85568f"},
"bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm", "7af5c7e09fe1d40f76c8e4f9dd2be7cebd83909f31fee7cd0e9eadc567da8353"},
"castore": {:hex, :castore, "0.1.16", "2675f717adc700475345c5512c381ef9273eb5df26bdd3f8c13e2636cf4cc175", [:mix], [], "hexpm", "28ed2c43d83b5c25d35c51bc0abf229ac51359c170cba76171a462ced2e4b651"},
"connection": {:hex, :connection, "1.1.0", "ff2a49c4b75b6fb3e674bfc5536451607270aac754ffd1bdfe175abe4a6d7a68", [:mix], [], "hexpm", "722c1eb0a418fbe91ba7bd59a47e28008a189d47e37e0e7bb85585a016b2869c"},
"cors_plug": {:hex, :cors_plug, "1.5.2", "72df63c87e4f94112f458ce9d25800900cc88608c1078f0e4faddf20933eda6e", [:mix], [{:plug, "~> 1.3 or ~> 1.4 or ~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "9af027d20dc12dd0c4345a6b87247e0c62965871feea0bfecf9764648b02cc69"},
"cowboy": {:hex, :cowboy, "2.9.0", "865dd8b6607e14cf03282e10e934023a1bd8be6f6bacf921a7e2a96d800cd452", [:make, :rebar3], [{:cowlib, "2.11.0", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "1.8.0", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm", "2c729f934b4e1aa149aff882f57c6372c15399a20d54f65c8d67bef583021bde"},
Expand Down