Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Throughput test #61

Closed
3 tasks
konradkonrad opened this issue Jun 27, 2016 · 7 comments
Closed
3 tasks

Throughput test #61

konradkonrad opened this issue Jun 27, 2016 · 7 comments
Labels
Flag / Testing

Comments

@konradkonrad
Copy link
Contributor

konradkonrad commented Jun 27, 2016 •
edited by hackaugusto
Loading

Tasklist

  • Measure throughput of a node sending mediated transactions through N channels without mediation.
  • The benchmark script must accept an argument for the minimum value for the TPS, and if the given value is not reached it should exit with a status of 1
    • The benchmark must be added to the build with the minimum value for TPS set.
@konradkonrad konradkonrad added this to the MVP milestone Jun 27, 2016
@konradkonrad konradkonrad self-assigned this Jun 27, 2016
This was referenced Jun 27, 2016
Closed
Closed
Closed
Closed
@konradkonrad konradkonrad modified the milestones: Scalability PoC, MVP Jun 27, 2016
konradkonrad added a commit that referenced this issue Aug 26, 2016
@konradkonrad
Add simple deployment script in order to get rid of truffle
6e5d63f 
Relates to #61
konradkonrad added a commit that referenced this issue Aug 30, 2016
@konradkonrad
Include contract creation in full_genesis command #61
ea3e80b
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Ignore private keys #61
7875e51
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Add initial ansible stub for orchestration #61
80c58fe
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Disable inventory caching #61
d9f95e8
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Switch to ubuntu image and add ppa (wip) #61
6a9470e
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Working geth, solc installation #61
27c7ed5
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Finish up installation on AWS #61
87a49c3 
The playbook now installs everything necessary on a new image.
TODO:
- create DAG
- store as a new AMI (so launching multiple instances can be quicker)
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Add DAG creation #61
a40c03d
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Update README #61
2b3918c
konradkonrad added a commit that referenced this issue Aug 31, 2016
@konradkonrad
Add ami creation to build playbook #61
2dfa653
konradkonrad added a commit that referenced this issue Sep 1, 2016
@konradkonrad
Update py27 #61
19087c9 
The default system python was 2.7.6 on the base AMI.
konradkonrad added a commit that referenced this issue Sep 1, 2016
@konradkonrad
Fix raiden installation on AWS through ansible #61
615a830 
There seems to be a bug in ansible, where the `editable` argument is not properly
inserted in the pip command. Explicitely adding it solves this.
konradkonrad added a commit that referenced this issue Sep 1, 2016
@konradkonrad
Update scenario vars and README #61
49e1033
konradkonrad added a commit that referenced this issue Sep 1, 2016
@konradkonrad
WIP: Add raiden configuration in scenario-folder #61 #63
36959ea 
WIP-note: dependencies on master are currently broken, so this is not
debuggable atm.
konradkonrad added a commit that referenced this issue Sep 1, 2016
@konradkonrad
Fix syntax issue #61 #63
ac358ba
konradkonrad added a commit that referenced this issue Sep 1, 2016
@konradkonrad
Better conditionals for reading scenario params #63 #61
869706c
konradkonrad added a commit that referenced this issue Sep 2, 2016
@konradkonrad
Fix ansible installation #61
b8085f1 
Installing ansible with dependencies leads to broken dependencies
for pydevp2p. By explicitely installing with `--no-deps` and excluding
ansibles cryptography modules, this can be avoided. Hopefully ansible
still works as expected...
konradkonrad added a commit that referenced this issue Sep 2, 2016
@konradkonrad
Unpretty json for use in automation #61
8edcace 
This adds a flag `--pretty` (default False) to `config_builder.py`
commands.
konradkonrad added a commit that referenced this issue Sep 2, 2016
@konradkonrad
Make sure keystore directory exists #61
29d1c95
konradkonrad added a commit that referenced this issue Sep 2, 2016
@konradkonrad
Generify jsonification #61
32e88ad
utzig pushed a commit to utzig/raiden that referenced this issue Oct 31, 2016
@konradkonrad @utzig
Directly invoke scenario_runner.py raiden-network#61
ec19cbd 
Also use `shell` module so we can resolve `$()` subcommands without
needing to read in new variables.
If a newer version of raiden should be tested, the variable `-e force_pull` can be set.
utzig pushed a commit to utzig/raiden that referenced this issue Oct 31, 2016
@konradkonrad @utzig
Fix ansible syntax raiden-network#61
d9ba52b
utzig pushed a commit to utzig/raiden that referenced this issue Nov 1, 2016
@konradkonrad @utzig
Use our static solc in build-ami raiden-network#61
10e7cdd
utzig pushed a commit to utzig/raiden that referenced this issue Nov 1, 2016
@konradkonrad @utzig
Directly invoke scenario_runner.py raiden-network#61
493c7eb 
Also use `shell` module so we can resolve `$()` subcommands without
needing to read in new variables.
If a newer version of raiden should be tested, the variable `-e force_pull` can be set.
utzig pushed a commit to utzig/raiden that referenced this issue Nov 1, 2016
@konradkonrad @utzig
Fix ansible syntax raiden-network#61
9a6a84b
utzig pushed a commit to utzig/raiden that referenced this issue Nov 3, 2016
@konradkonrad @utzig
Use our static solc in build-ami raiden-network#61
a8f1cf5
utzig pushed a commit to utzig/raiden that referenced this issue Nov 3, 2016
@konradkonrad @utzig
Directly invoke scenario_runner.py raiden-network#61
f94c8e9 
Also use `shell` module so we can resolve `$()` subcommands without
needing to read in new variables.
If a newer version of raiden should be tested, the variable `-e force_pull` can be set.
utzig pushed a commit to utzig/raiden that referenced this issue Nov 3, 2016
@konradkonrad @utzig
Fix ansible syntax raiden-network#61
134f831
utzig pushed a commit to utzig/raiden that referenced this issue Nov 4, 2016
@konradkonrad @utzig
Use our static solc in build-ami raiden-network#61
26bc6a3
utzig pushed a commit to utzig/raiden that referenced this issue Nov 4, 2016
@konradkonrad @utzig
Directly invoke scenario_runner.py raiden-network#61
c0a4913 
Also use `shell` module so we can resolve `$()` subcommands without
needing to read in new variables.
If a newer version of raiden should be tested, the variable `-e force_pull` can be set.
utzig pushed a commit to utzig/raiden that referenced this issue Nov 4, 2016
@konradkonrad @utzig
Fix ansible syntax raiden-network#61
aeb9f6e
utzig pushed a commit to utzig/raiden that referenced this issue Nov 17, 2016
@konradkonrad @utzig
Use our static solc in build-ami raiden-network#61
64d80ef
utzig pushed a commit to utzig/raiden that referenced this issue Nov 17, 2016
@konradkonrad @utzig
Directly invoke scenario_runner.py raiden-network#61
d3238c0 
Also use `shell` module so we can resolve `$()` subcommands without
needing to read in new variables.
If a newer version of raiden should be tested, the variable `-e force_pull` can be set.
utzig pushed a commit to utzig/raiden that referenced this issue Nov 17, 2016
@konradkonrad @utzig
Fix ansible syntax raiden-network#61
b592126
@LefterisJP LefterisJP removed this from the Scalability Demo milestone Jan 12, 2017
@LefterisJP
Copy link
Contributor

@konradkonrad Is the throughput test viable in any way or form as of this moment? Should we try to revive it to measure throughput of our product before release so we can have proper benchmarks to support our claims?

@hackaugusto hackaugusto mentioned this issue Jun 20, 2018
Closed
@ulope
Copy link
Collaborator

ulope commented Jun 20, 2018

This has a lot of overlap with #652

@hackaugusto
Copy link
Contributor

hackaugusto commented Jun 20, 2018 •
edited
Loading

@ulope yep, but I assume that #652 will not run on the build but on a testnet, correct?

@andrevmatos andrevmatos mentioned this issue Jul 31, 2018
Closed
@konradkonrad konradkonrad removed their assignment Oct 1, 2018
@LefterisJP
Copy link
Contributor

@ulope Can we use the scenario player to measure throughput now?

@ulope
Copy link
Collaborator

ulope commented Oct 8, 2018

Yes, we just have to think about how we want to gather and export that data.

For gathering I see two options that can be added without much work:

  • Sum up the runtime of all transfer tasks and then divide by number of transfers
  • Add a benchmark parameter to the transfer task it so we have better control over what we measure

I'd prefer variant 2.

As for exporting, writing some form of json report should be enough.

@hackaugusto hackaugusto mentioned this issue Jan 10, 2019
Closed
@Dominik1999
Copy link
Contributor

@konradkonrad can we close this issue or should we somehow try to revive it?

hackaugusto pushed a commit to hackaugusto/raiden that referenced this issue Dec 5, 2019
@LefterisJP
Merge pull request raiden-network#61 from palango/fee-format
e9ebb1e 
Define fee format
@karlb
Copy link
Contributor

karlb commented Apr 7, 2022

Some scenarios can be considered to be throughput tests.

@karlb karlb closed this as completed Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Flag / Testing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@ulope @palango @utzig @karlb @hackaugusto @LefterisJP @konradkonrad @Dominik1999