Skip to content
This repository has been archived by the owner on Sep 8, 2020. It is now read-only.

Commit

Permalink
Test framework (#13)
Browse files Browse the repository at this point in the history
Pytest framework and Travis CI added.

* Use internal copy of python-bitcoinlib for bitcoin RawProxy RPC calls
to minimise dependencies

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed python-bitcoinlib dependency as its now internal:
lnd_grpc.bitcoin

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Re-added params required to communicate with bitcoin RPC

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Initial commit of working test framework

Framework extracted from Christian Deckers Lightning Integration Test
project found at https://github.com/cdecker/lightning-integration

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed BtcD references

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Working with implementation of lnd_grpc utilising macaroon connection

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Moved lightning and wallet stubs to below the credentials-builder.
Code of stub generators otherwise unchanged.

Added an insecure_stub() which can be used without macaroons

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Passing tests added up to send_coins()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Cleaned up directory. Added test_subscribe_transactions()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added get_addresses() helper function.
Added test_new_address()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* All non-interactive tests implemented and passing

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Fixed teardown of fixtures

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Reduced sleep time

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added send_many test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added connection test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added open_channel() test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Fixed lambda function to wait for the empty list

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Linting

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Remove bad test files

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Revert parameter names

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Amend parameters according to rebase off Pierre's PR#9

Signed-off-by: willcl-ark <will8clark@gmail.com>

* LND requires a block generation to activate the services correctly.

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test_open_channel()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Modified sleep timers

Signed-off-by: willcl-ark <will8clark@gmail.com>

* More efficient sleep timers. Improved wait_for logic

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added send_payment_sync test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added a setup_nodes() helper function

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Linting

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added alice, bob, carol and dave

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Refactored to use alice, bob, carol and dave fixtures

Signed-off-by: willcl-ark <will8clark@gmail.com>

* add_funds() will consider fixture use across tests

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test for manually specified send_payment_sync

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test for send_payment

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed python-bitcoinlib dependency as its now internal:
lnd_grpc.bitcoin

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Initial commit of working test framework

Framework extracted from Christian Deckers Lightning Integration Test
project found at https://github.com/cdecker/lightning-integration

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed BtcD references

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Working with implementation of lnd_grpc utilising macaroon connection

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Passing tests added up to send_coins()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Cleaned up directory. Added test_subscribe_transactions()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added get_addresses() helper function.
Added test_new_address()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* All non-interactive tests implemented and passing

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Fixed teardown of fixtures

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Reduced sleep time

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added send_many test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added connection test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added open_channel() test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Fixed lambda function to wait for the empty list

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Linting

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Remove bad test files

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Revert parameter names

Signed-off-by: willcl-ark <will8clark@gmail.com>

* LND requires a block generation to activate the services correctly.

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added setup_dave() which can add a 3rd node Dave to the network.

Dave is only connected with a channel to Carol currently

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test_send_to_route() which is passing on a simple 3 node network

Signed-off-by: willcl-ark <will8clark@gmail.com>

* modify send_to_route_sync to accept working `routes` argument

fixes broken `route` argument even though API docs state `routes` is
deprecated and `route` should be used instead.

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Return the routes iterable directly from query_routes()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed debugging code from test_send_to_route_sync()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Revised setup_nodes() to accept a list of n nodes as argument

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Linting

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Use new setup_nodes() format

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test_send_to_route()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Modified the send_to_route_generator so that it works (Magic!)

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Aligned send_to_route arguments with send_to_route_sync()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test_subscribe_channel_events()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed un-usable kwargs from subscribe_channel_events()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test_subscribe_channel_graph()

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added test_update_channel_policy()

All tests passing

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added break_down_nodes()

Interactive tests can now share instances of bob, carol and dave
to significantly speed up test times. Clients are broken down following
each test.

Signed-off-by: willcl-ark <will8clark@gmail.com>

* bob, carol and dave persist across classes

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed old tests

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed unnecessary cli.py

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Remove unnecessary import

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Initial write-up of test instructions

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Trimmed test-requirements.txt for unnecessary dependencies

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Amend add_funds() to consider a wallet with pre-existing balance

...now that fixtures are shared between non-interactive tests

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Break down nodes as part of setup_nodes()

Avoids cases where a previous test fails and skips break_down() leading
to error in subsequent test

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed old code

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Remove references to break_down_nodes() from test.py

Related to d08f496

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Add single break_down_nodes() to test_open_channel()

Avoids already connected error

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Add _update_channel_policy() arguments as per PR#12

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Rebase lnd_grpc.py imports onto master

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Import and linting fixes

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed redundant docs

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Re-generated with correct import for PyPi packaging

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Import and linting fixes

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Added package name to __init__.py

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Import and linting fixes

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Exclude any hardlinks/binaries from bin/ using MANIFEST.in

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Increment version. Remove tests from exclude=[]

Signed-off-by: willcl-ark <will8clark@gmail.com>

* remove pycaches

* remove DS_Store

* initialize bin directory

* Reverted lnd_dir to data_dir

Signed-off-by: willcl-ark <will8clark@gmail.com>

* fully remove __pycache__ from branch

* fully remove __pycache__ from branch

* add gitignore to ensure pycaches are ignored in future

* remote pycaches

* amend .gitignore to fully ignore pycaches

* Travis setup changes!

Move tests into top level directory to seperate from main package

Signed-off-by: willcl-ark <will8clark@gmail.com>

Remove python-bitcoinlib from codebase

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add python-bitcoinlib as a test-requirements.txt requirement

Also update grpcio version to match main requirements.txt version

Signed-off-by: willcl-ark <will8clark@gmail.com>

add __init__.py to test_utils

Don't require symlinking lnd into /bin folder

Just read from PATH

Signed-off-by: willcl-ark <will8clark@gmail.com>

Make test-requirements.txt include all requirements!

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add travis configurations

Signed-off-by: willcl-ark <will8clark@gmail.com>

Moved travis.yml so it's detected

Signed-off-by: willcl-ark <will8clark@gmail.com>

Made script executable for travis

Signed-off-by: willcl-ark <will8clark@gmail.com>

Made script executable for travis

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add universe repo for xenial

Signed-off-by: willcl-ark <will8clark@gmail.com>

Don't build bitcoin, just grab binary

Signed-off-by: willcl-ark <will8clark@gmail.com>

Use Travis $HOME variable

Signed-off-by: willcl-ark <will8clark@gmail.com>

Reconfigure travis

Signed-off-by: willcl-ark <will8clark@gmail.com>

Typo in test command

Signed-off-by: willcl-ark <will8clark@gmail.com>

Typo in test command

Signed-off-by: willcl-ark <will8clark@gmail.com>

Typo in install command

Signed-off-by: willcl-ark <will8clark@gmail.com>

Typo in grpcio

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add cheroot dependency

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add cheroot dependency

Signed-off-by: willcl-ark <will8clark@gmail.com>

Troubleshooting travis environment

Signed-off-by: willcl-ark <will8clark@gmail.com>

Troubleshooting travis environment

Signed-off-by: willcl-ark <will8clark@gmail.com>

Fixed travis exports?

Signed-off-by: willcl-ark <will8clark@gmail.com>

Try manually copying binaries to /usr/local/bin

Signed-off-by: willcl-ark <will8clark@gmail.com>

Try manually copying binaries to /usr/local/bin

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add TEST_DEBUG=1 to test script

Signed-off-by: willcl-ark <will8clark@gmail.com>

Add travis environment variable check for tls paths

Signed-off-by: willcl-ark <will8clark@gmail.com>

Reduce log verbosity to satisfy travis

Signed-off-by: willcl-ark <will8clark@gmail.com>

Removed superfluous print statements to further reduce travis log output

Signed-off-by: willcl-ark <will8clark@gmail.com>

Revert breaking print statement removals

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Remove unnecessary travis scripts now that we grab binaries

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Amend test.md to reflect PATH search for both binaries

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Removed unnecessary bin folder from tests

Signed-off-by: willcl-ark <will8clark@gmail.com>

* Try to pass node_id to the logger

Signed-off-by: willcl-ark <will8clark@gmail.com>
  • Loading branch information
willcl-ark committed Apr 8, 2019
1 parent 5bb437b commit 5e7936f
Show file tree
Hide file tree
Showing 27 changed files with 2,109 additions and 1,147 deletions.
126 changes: 126 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don’t work, or not
# install all needed dependencies.
#Pipfile.lock

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Pycharm workspace settings
.idea

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/
9 changes: 9 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
dist: xenial
language: python
python: 3.7.2
before_install: ./.travis/travis_before_install.sh
install:
- pip install -r test-requirements.txt
- pip install -e .
script:
- py.test -v -s tests/test.py
24 changes: 24 additions & 0 deletions .travis/travis_before_install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/sh

# Exit immediately at non-zero exit code
set -ev

#######################
## Install Bitcoin Core
#######################

wget https://bitcoincore.org/bin/bitcoin-core-0.17.0/bitcoin-0.17.0-x86_64-linux-gnu.tar.gz
tar -xzf bitcoin-0.17.0-x86_64-linux-gnu.tar.gz
sudo cp /home/travis/build/willcl-ark/lnd_grpc/bitcoin-0.17.0/bin/bitcoind /usr/local/bin/bitcoind
sudo cp /home/travis/build/willcl-ark/lnd_grpc/bitcoin-0.17.0/bin/bitcoin-cli /usr/local/bin/bitcoin-cli


######################
# Install LND v-0.6-beta
######################

# Install LND
wget https://github.com/lightningnetwork/lnd/releases/download/v0.6-beta-rc3/lnd-linux-amd64-v0.6-beta-rc3.tar.gz
tar -xzf lnd-linux-amd64-v0.6-beta-rc3.tar.gz
sudo cp /home/travis/build/willcl-ark/lnd_grpc/lnd-linux-amd64-v0.6-beta-rc3/lnd /usr/local/bin/lnd
sudo cp /home/travis/build/willcl-ark/lnd_grpc/lnd-linux-amd64-v0.6-beta-rc3/lncli /usr/local/bin/lncli
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
exclude lnd_grpc/bin/lnd
1 change: 1 addition & 0 deletions lnd_grpc/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
name = "lnd_grpc"
33 changes: 18 additions & 15 deletions lnd_grpc/lnd_grpc.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
import codecs
import sys
import time
from os import environ

import grpc

from lnd_grpc import utilities as u
from os import environ

from lnd_grpc.utilities import get_lnd_dir
from lnd_grpc.protos import rpc_pb2 as ln, rpc_pb2_grpc as lnrpc


# tell gRPC which cypher suite to use
environ["GRPC_SSL_CIPHER_SUITES"] = 'HIGH+ECDSA'

Expand Down Expand Up @@ -42,7 +43,7 @@ def lnd_dir(self):
if self._lnd_dir:
return self._lnd_dir
else:
self._lnd_dir = u.get_lnd_dir()
self._lnd_dir = get_lnd_dir()
return self._lnd_dir

@lnd_dir.setter
Expand Down Expand Up @@ -369,20 +370,22 @@ def pay_invoice(self, payment_request: str):
return response

@staticmethod
def send_to_route_generator(invoice, route):
i = 0
while i < 1:
request = ln.SendToRouteRequest(payment_hash=invoice.r_hash, routes=route)
def send_to_route_generator(invoice, routes):
# Commented out to complement the magic sleep below...
# while True:
request = ln.SendToRouteRequest(payment_hash=invoice.r_hash, routes=routes)
yield request
i += 1
# Magic sleep which tricks the response to the send_to_route() method to actually
# contain data...
time.sleep(5)

# Bi-directional streaming RPC
def send_to_route(self, invoice, route):
request_iterable = self.send_to_route_generator(invoice=invoice, route=route)
def send_to_route(self, invoice, routes):
request_iterable = self.send_to_route_generator(invoice=invoice, routes=routes)
return self.lightning_stub.SendToRoute(request_iterable)

# Synchronous non-streaming RPC
def send_to_route_sync(self, routes: ln.Route, **kwargs):
def send_to_route_sync(self, routes, **kwargs):
request = ln.SendToRouteRequest(routes=routes, **kwargs)
response = self.lightning_stub.SendToRouteSync(request)
return response
Expand Down Expand Up @@ -438,8 +441,8 @@ def get_chan_info(self, chan_id: int):
return response

# Uni-directional stream
def subscribe_channel_events(self, **kwargs):
request = ln.ChannelEventSubscription(**kwargs)
def subscribe_channel_events(self):
request = ln.ChannelEventSubscription()
return self.lightning_stub.SubscribeChannelEvents(request)

def get_node_info(self, pub_key: str):
Expand All @@ -458,7 +461,7 @@ def query_routes(self,
num_routes=num_routes,
**kwargs)
response = self.lightning_stub.QueryRoutes(request)
return response
return response.routes

def get_network_info(self):
request = ln.NetworkInfoRequest()
Expand Down
1 change: 1 addition & 0 deletions lnd_grpc/protos/googleapis
Submodule googleapis added at ca6189
749 changes: 375 additions & 374 deletions lnd_grpc/protos/rpc_pb2.py

Large diffs are not rendered by default.

Loading

0 comments on commit 5e7936f

Please sign in to comment.