indy-node.service: Failed with result 'exit-code'

[esrefatak]

Hi dear Indy Node community,

We plan to use Indy Node for a large-scale project. We are in R&D stage.

We could not start the indy-node service. Error occurs. Here are systemctl status indy-node output:

Here is validator.py status:

(Kindly ignore the reachable host list as I unplugged the cable)

Ubuntu version is 16.04.12 in VirtualBox.

I installed a network of 3 hosts. Each host contains 1 node.

Here is how I installed it.

sudo apt-get update -y && sudo apt-get install -y \
indy-node=1.13.0~dev1213 libindy-crypto=0.4.5 python3-indy-crypto=0.4.5 python3-orderedset=2.0 python3-psutil=5.4.3 python3-pympler=0.5 indy-plenum=1.13.0~dev1021 libindy=1.15.0~1536-xenial indy-cli=1.15.0~1536-xenial

sudo pip3 install python3-indy==1.15.0

sudo python3 init_indy_node.py \
--name Node1 --seed 4000F000u00000D0000000g0000Node1


sudo python3 create_domain_ledger_genesis_file.py \
--stewardDids \
"RnXJMDxifkBaceAkPizz8F, AmJHyEZ9ofrPz1DUfx97Qo, HBgMBsgaEytzE38djLfNe7" \
--stewardVerkeys \
"~9F4P9aQysR1YATnARMuhxt, ~CVv3mmR7QPanynNMJmTG9f, ~H9LbyNQEZPR1oKzBaNMrDW" \
--trusteeDids \
"L4vaBJDbZiaX68mVHLuZ2E" \
--trusteeVerkeys \
"~QKV8UWR7LHB7APvrWc88xK" \
--network sandbox


sudo python3 create_pool_ledger_genesis_file.py \
--nodeVerkeys \
"85f5c5a0de4236dc01a97e20e1ddf33699c4ddfe5431ced2c8f09e03634edda7, 499ae101d85af6e3093663d98afa631a258518a2ccb2dc5985d5ca248b39ee0e, 41f152fe61923ba93012e1a2309c07a5dc62d1772ec715a44fbb6afef18c8aa7" \
--nodeBlskeys \
"tP6NHFxgWayNLXWrAu6jSdkauHRAmViF11BhELfNenMJxC9vbg7GQwqh1W9w3bzbR6kCSjZuqwfktvtdzzx8Bx1NZuWzUN9a7nEehRWjH4ozKPH7P6tw6Et9W1yyX7wj33nVWAotZ2r9zoYouGV7ybg9ijmQHhYiqAYqXerQ6n7zD2, CkXUu3Sqrm8212ahXf2G6fpQctMW86rPR5o4AFi5LJytw3NsDPzba83m5xCNuvSS353NYEdDg8EZtsWEDMMpgUpAJAB1hYzaxAybcozwkYgoSi941wi6HqJHiNLxWgHzizwXGRY2KymcTqC385WkGnMLew6SzhajcP8wVniyZRz3pi, ftbhBQme51s3UZPavohqouyQkGNWf43fDn9ddey1Jm1vGpoquWh3suR9MCEj4BVvfnwnd7BiLGwSb6FfHF3wwPxYqKdzi3qq4MrxRkTDFCm7ZmbTPBkuSUy5Yr41cFeGFDP21TrZrYZb8oNjYCbJe9VxskrJyXpmbY2s2hWWVCsTNe" \
--nodeBlsProofs \
"QwxHLEJFt1HXuE7P6UdpXztNaNVAoFdiTbhMGXtQgPFCC7GbSG5UAXPFHdkmfsSV9jjyVataX2ujKCEqZn2hpxcnfeBEYioYdqehPGH2VLh1Qw3XZKC3vFW1kc3uMaRtHqWWR6m4Enu2qLuSdFcLtYCJreDpHghB8ds42QHXwoZ8T1, QmrjxheeyDAFknchT4AEBLD6fKVLS9ZMuXDmhstXXnecB64zBtzgmUFHQgoHtrvfPU6A4RS1uWb3AjBVBZh8QBquB1DMfGTGpYszvCQDhgLJtnvrJjVX7whtVT4r33KyCvicPyHXk1ku3H3MYJSbKgC8a9NUv19q2BugsZn1nLSgHm, Qy4DvpUTGiegNyaNhhPXv2oURCciHASsWV9YPB44wsopipJd8Yknu6pJ2Nm4m6BoFGmAnRMHNhw6LupYyNY1UGQkwrSZkFWERExrKMYS7xyi33K1PDpzoM3Wfag7ivLCBHRqfsUpbSBep3VgyqogEGCwmwSkVVSe7kJDBsNW9JYVnw" \
--nodeName \
"Node1, Node2, Node3" \
--nodePort \
"9701, 9703, 9705" \
--clientPort \
"9702, 9704, 9706" \
--stewardDids \
"RnXJMDxifkBaceAkPizz8F, AmJHyEZ9ofrPz1DUfx97Qo, PeVR5pdFizbK8WN9THd3mq" \
--nodeNum 1 \
--network sandbox \
--ips '10.0.2.5, 10.0.2.6, 10.0.2.7'

And then:

sudo python3 start_indy_node.py Node1 0.0.0.0 9701 0.0.0.0 9702

When I execute the start_indy_node.py, nothing happens as you can see in the following screenshow. In third host I press Ctrl+C and executed the validator to check if hosts can see eachother:

"Unknown state" turns into "stopped" status time to time.

I attached /var/log/indy content:

https://transfer.sh/a5f36Y/logs2.tar.gz

We struggle since last week.
Thanks in advance for your helps.

[WadeBarnes]

@esrefatak, I'm seeing a few things with your configuration.

1. The recommended way to setup a new production level network is documented here; Setting up a New Network. The docs refer to installing the sovrin package. If you want to have a pure indy-node network, you can replace sovrin with indy-node.
2. You need a minimum of 4 nodes in order to form a functional network.
3. The 1.13.x builds of indy-plenum and indy-node are meant for Ubuntu 20.04, they are not meant to be compatible with Ubuntu 16.04. The latest production release for Ubuntu 16.04 is version 1.12.4.

If you would prefer to use Ubuntu 20.04, here is the v1.13.1-rc2 release you could use. You'll need to register the Hyperledger deb repo to install the packages, as the indy packages are now being published there;

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9692C00E657DDE61
echo "deb  https://hyperledger.jfrog.io/artifactory/indy focal rc"  >> /etc/apt/sources.list

https://hyperledger.jfrog.io/ui/repos/tree/General/indy/pool/focal/rc/i/indy-node/indy-node_1.13.1~rc2_amd64.deb

[WadeBarnes]

@esrefatak, There is also the question of build your own vs use an existing service. There are couple service offerings available for indy-node networks, https://sovrin.org/, and https://indicio.tech/, that may meet your needs.

[esrefatak]

In our case, third party services are not affordable as we have really high number of users. Because of high volume transactions, we plan to host our own network. Thanks for your advice.

[esrefatak]

@WadeBarnes
I started to install all the network from scratch according to your recommendations.

For Ubuntu 20.04 ("Focal") what no I need to specify? No info in doc about this detail. I tried "focal" but failed.

A stackoverflow post says "use bionic" but i wanted to be sure.

[WadeBarnes]

I provided the repo spec above; #1766 (comment)

Here is a more complete example. It's a docker file for simplicity, but gives you the basic steps you'll need to perform:

FROM ubuntu:20.04
RUN apt-get update -y && apt-get install -y \
    gnupg \
    ca-certificates

# ========================================================================================================
# Update repository signing keys
# --------------------------------------------------------------------------------------------------------
# Hyperledger
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9692C00E657DDE61 && \
    # Sovrin
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88
# ========================================================================================================

RUN echo "deb https://hyperledger.jfrog.io/artifactory/indy focal rc"  >> /etc/apt/sources.list && \
    echo "deb http://security.ubuntu.com/ubuntu bionic-security main"  >> /etc/apt/sources.list && \
    echo "deb https://repo.sovrin.org/deb bionic master" >> /etc/apt/sources.list
RUN apt-get update -y && apt-get install -y \
    # rocksdb python wrapper
    rocksdb=5.8.8 \
    libgflags-dev \
    libsnappy-dev \
    zlib1g-dev \
    libbz2-dev \
    liblz4-dev \
    libgflags-dev \
    python3-libnacl=1.6.1 \
    python3-sortedcontainers=1.5.7 \
    python3-ujson=1.33 \
    python3-pyzmq=22.3.0 \
    indy-plenum=1.13.1~rc2 \
    indy-node=1.13.1~rc2 \
    libssl1.0.0 \
    ursa=0.3.2-1 \
    && mv /usr/lib/ursa/* /usr/lib && rm -rf /usr/lib/ursa

[esrefatak]

I provided the repo spec above; #1766 (comment)

Actually, problem is installing indy-cli. Docs you linked, requires to install indy-cli first and generate some keys/seed (before indy-node installation).

As you can see in following screenshot, it's not exist in sources:

Where I can install indy-cli from? For Ubuntu 20.04/focal?

[WadeBarnes]

The better option is to use the containerized indy-cli in von-network as recommended in the Setting up a New Network; https://github.com/bcgov/von-network/blob/main/docs/Indy-CLI.md

[WadeBarnes]

Closing for now. Please reopen if the issue continues.