Skip to content
Permalink
master
Switch branches/tags
Go to file
 
 
Cannot retrieve contributors at this time
1774 lines (1307 sloc) 112 KB

Sovrin Release Notes

image alt text

Disclosure

Although every attempt has been made to make this information as accurate as possible, please know there may be things that are omitted, not fully developed yet, or updates since this publication that were not included in the information below. Only the most pressing or significant items have been listed. For the entire list of tickets and or specific information about any given item, please visit the list at Hyperleder Indy's Jira. Once logged in, simply navigate to Projects > Indy.

1.1.89

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.12.4
indy-node 1.12.4
sovrin 1.1.89
sovtoken 1.0.9
sovtokefees 1.0.9

Major Changes

  • NYM dynamic validation check transaction w/out verkey or role

Changes and Additions

Description Additional Information Ticket Number
NYM dynamic validation check transaction w/out verkey or role

1.1.81

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.12.3
indy-node 1.12.3
sovrin 1.1.81
sovtoken 1.0.8
sovtokefees 1.0.8

Major Changes

  • Stability fixes

Major Fixes

Description Additional Information Ticket Number
TAA signature's validation fix (milliseconds sending broke the primary)

1.1.71

Notices for Stewards

Stop indy-node service on demoted nodes to avoid a minor issue with client's requests processing (see Known Issues for details).

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.12.2
indy-node 1.12.2
sovrin 1.1.71
sovtoken 1.0.7
sovtokefees 1.0.7

Major Changes

  • Stability fixes

Major Fixes

Description Additional Information Ticket Number
WARNING messages incorrectly logged if tokens are not used INDY-2221
REV_REG_DEF tag field is not validated INDY-2314
A node may re-send messages in a loop in case of connection issues INDY-2318
Up to F Nodes may not be able to finish View Change if there are uncommitted NODE txns INDY-2319
A node lagging behind may not be able to finish view change if nodes have been added/demoted INDY-2308
A lagging node may use wrong N and F quorum values and never finish view change if there are NODE txns being processed INDY-2320
A lagging node may be the only one who started view change in case of F Nodes added/promoted in 1 batch INDY-2322
Debug View Change when nodes added/demoted/promoted INDY-2326

Known Issues

Description Additional Information Ticket Number
Demoted Node should not process client's requests INDY-2334

1.1.67

Notices for Stewards

Use forced simultaneous pool upgrade.

Please be careful with demoting/promoting/adding nodes (see Known Issues for details).

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.12.1
indy-node 1.12.1
sovrin 1.1.67
sovtoken 1.0.6
sovtokefees 1.0.6

Major Changes

  • Multiple active TAAs implementation
  • Stability fixes

Major Fixes

Description Additional Information Ticket Number
Sovrin upgrade fails due to incorrect package dependencies INDY-2303
GET_CRED_DEF for a Schema with a lot of attributes may fail with Timeout INDY-2306
Only Trustee or Node owner can be the author of NODE demotion txn regardless of endorsement or auth constraint rules set INDY-2024

Changes and Additions

Description Additional Information Ticket Number
Allow multiple active TAAs INDY-2302
Allow multiple active TAAs: Debug INDY-2316
Improve TAA acceptance date validation INDY-2313
Get TAA should return the hash INDY-2297
Auth_Rules documentation should explain how endorsers work INDY-2304
Document PBFT view change protocol INDY-2138
Backups should start ordering in new view only after master instance ordered till prepared cert from NewView INDY-2299
Get rid of transport batches INDY-2294
Enable zeroMQ auto-reconnection INDY-2289
Improve simulation tests to include NODE txns INDY-2286
Improve BLS signature performance INDY-2280
Improve simulation tests to include processing of InstanceChanges INDY-2263

Known Issues

Description Additional Information Ticket Number
A node lagging behind may not be able to finish view change if nodes have been added/demoted INDY-2308
Up to F Nodes may not be able to finish View Change if there are uncommitted NODE txns INDY-2319
A node may start re-sending messages in a loop in case of connection issues INDY-2318
A lagging node may use wrong N and F quorum values and never finish view change if there are NODE txns being processed INDY-2320

1.1.63

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.12.0
indy-node 1.12.0
sovrin 1.1.63
sovtoken 1.0.5
sovtokefees 1.0.5

Major Changes

  • Improve primary selection algorithm
  • Take into account transaction history when recovering state for new nodes
  • Fix new nodes adding when there are old AUTH_RULE or plugin transactions

Major Fixes

Description Additional Information Ticket Number
The problem with a config state INDY-2283
Node loses consensus and unreachable by clients INDY-2287
A new added node failed to reach consensus INDY-2254

Changes and Additions

Description Additional Information Ticket Number
All nodes need to select the same primary during view change INDY-2262
Take into account txn history when recovering state from the ledger for new nodes INDY-2292
Do not restore Primaries from the audit ledger INDY-2298
Start View change on receiving a quorum of ViewChange messages INDY-2236
PBFT View change: cleanup and debug Part 3 INDY-2267
A Node missing a View Change may not be able to finish it if NODE txns have been sent INDY-2275
PrePrepare's Digest need to take into account all PrePrepare's fields INDY-2285
Investigate reasons of hundreds VCs during 15 txns per sec production load INDY-2295
Support historical req handlers for non-config ledgers INDY-2307

Known Issues

Description Additional Information Ticket Number
A node lagging behind may not be able to finish view change if nodes have been added/demoted INDY-2308
GET_CRED_DEF for a Schema with a lot of attributes may fail with Timeout INDY-2306
Only Trustee or Node owner can be the author of NODE demotion txn regardless of endorsement or auth constraint rules set INDY-2024

1.1.60

Notices for Stewards

Please be careful with demoting/promoting/adding nodes (see Known Issues for details).

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.11.0
indy-node 1.11.0
sovrin 1.1.60
sovtoken 1.0.4
sovtokefees 1.0.4

Major Changes

  • Switch to PBFT View Change protocol
  • Stability fixes

Major Fixes

Description Additional Information Ticket Number
One node doesn't catch up after promotion INDY-2222
A Replica may process messages from other Replicas INDY-2248
Up to F nodes are out of consensus after >3 hours of load INDY-2268
A Node may not connect to another Node after restart INDY-2274
Two View Changes happen during master or backup primary demotion INDY-2247

Changes and Additions

Description Additional Information Ticket Number
Debug: Integrate PBFT viewchanger service into current codebase INDY-2140
Request missing ViewChange messages when receiving NewView INDY-2178
Basic integration tests with a new View Change protocol need to pass INDY-2223
Recover from a situation when View Change is finished on >= N-F of other nodes INDY-2224
A Primary lagging behind a stable chedkpoints should not send NewView INDY-2230
Do not stabilize checkpoint after the view change if a Replica doesn't have this checkpoint INDY-2231
Save PrePrepare's BatchID in audit ledger and restore list of preprepares and prepares on node startup INDY-2235
PBFT View Change Debug: Part 2 INDY-2244
Optimize Propagate logic INDY-2257

Known Issues

Description Additional Information Ticket Number
All nodes need to select the same primary during view change INDY-2262
A Node missing a View Change may not be able to finish it if NODE txns have been sent INDY-2275
A new node joining the pool during the view change may not be able to start ordering immediately INDY-2276
Summary: If there are NODE txns for adding/removing nodes interleaved with View Changes (not any view changes, but a specific subset), then either up to F or all Nodes may not be able to finish view change. Please see the details and conditions when it may happen in INDY-2262.

1.1.58

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended. PBFT View Change was implemented but not enabled so old View Change is active now.

Component Version Information

Components Version Numbers
indy-plenum 1.10.0
indy-node 1.10.0
sovrin 1.1.58
sovtoken 1.0.3
sovtokefees 1.0.3

Major Changes

  • PBFT View Change implementation (not enabled yet) and corresponding code improvements
  • BLS multi-signature fixes and improvements
  • The latest version of ZMQ library support
  • Stability fixes

Major Fixes

Description Additional Information Ticket Number
GET_TXN doesn't work with old libindy INDY-2233
Need to improve error message with invalid signature INDY-2103
A node may not be able to connect to another node if another node was able to connect INDY-2183
ZMQError: Address already in use when restarting client stack INDY-2212
State proofs and audit proofs are not working for token ledger after transaction with FEEs is written ST-623
Unexpected change to file access permissions ST-621

Changes and Additions

Description Additional Information Ticket Number
All ledgers in a batch need to be BLS multi-signed INDY-2228
Drop ppSeqNo on Backups after View Change INDY-2226
Move 3PC Message Request logic into a separate service INDY-2220
Bump pyzmq to the latest version INDY-2213
Integration of Services: Cleanup INDY-2208
Integrate Checkpointer Service into Replica INDY-2179
Use audit ledger in Checkpoints INDY-2177
Integrate OrderingService into Replica INDY-2169
Integrate PrimarySelector into View Change Service INDY-2167
Integrate view change property-based tests into CI INDY-2150
Integrate and run PBFT View Changer simulation tests with a real implementation INDY-2149
Implement PBFT viewchanger service with most basic functionality INDY-2147
Extract and integrate ConsensusDataProvider from Replica INDY-2139
Extract Checkpointer service from Replica INDY-2137
Extract Orderer service from Replica INDY-2136
Simulation tests for View Changer (no integration) INDY-2135
Implementation: Make PBFT view change working INDY-1340
Implement network, executor, orderer and checkpointer as adaptors for existing codebase INDY-1339
Define Interfaces needed for View Change Service INDY-1338
Modify WriteReqManager to meet Executor interface needs INDY-1337
Stop resetting ppSeqNo (and relying on this) in new view INDY-1336
Enable full ordering of batches from last view that have been already ordered, make execution on replicas that executed them no-op INDY-1335
Balance checker script ST-620

Known Issues

Description Additional Information Ticket Number
One node doesn't catch up after promotion INDY-2222

1.1.56

Notices for Stewards

Migration script will be applied for buildernet only and will return error message in python shell for any other pools (if manual migration will be performed). There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.9.2
indy-node 1.9.2
sovrin 1.1.56
sovtoken 1.0.2
sovtokefees 1.0.2

Major Changes

  • Stability fixes
  • Endorser support fixes and improvements
  • Improving GET_TXN to be able to query just one node the same way as for other GET requests

Major Fixes

Description Additional Information Ticket Number
New nodes added after last upgrade (1.9.1) are not in consensus INDY-2211
indy-node broken by indy-plenum and python-dateutil INDY-2176
Issue with non utf-8 decoding INDY-2218
Endorsers must be specified within the transaction INDY-2199
One node doesn't catch up INDY-2215

Changes and Additions

Description Additional Information Ticket Number
As a user, I need to be able to know what was the last update time of the ledger when querying a txn via GET_TXN request INDY-1954
Endorser field can contain a DID with a known role only INDY-2198
TokenAuthNr needs to extend LedgerBasedAuthNr, not CoreAuthNr ST-618

1.1.52

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Component Version Information

Components Version Numbers
indy-plenum 1.9.1
indy-node 1.9.1
sovrin 1.1.52
sovtoken 1.0.1
sovtokefees 1.0.1

Major Changes

  • New DIDs can be created without endorsers
  • Transaction authors don't need to be endorsers
  • TAA acceptance should use date, not time
  • Bug fixes

Major Fixes

Description Additional Information Ticket Number
Incorrect request validation INDY-2164
Need to make "reask_ledger_status" repeatable INDY-2112
When view change takes too long instance change should be sent periodically INDY-2143
Hotfix: Request GET_UTXO needs to take into account max limit for a message ST-600

Changes and Additions

Description Additional Information Ticket Number
New DIDs can be created without endorsers INDY-2171
Transaction authors don't need to be endorsers INDY-2173
Grab pool data for failed system tests INDY-2141
Memory profiling needs to be removed from GET_VALIDATOR_INFO output INDY-2182
Implement PBFT viewchanger service with most basic functionality INDY-2147
Extract Orderer service from Replica INDY-2136
Extract and integrate ConsensusDataProvider from Replica INDY-2139
TAA acceptance should use date, not time INDY-2157
Clean-up Pluggable Request Handlers INDY-2154
GET_UTXO needs to use pagination to be able to return all unspent UTXOs regardless of max message size ST-602
Extended State Proof verification for GET_UTXO ST-604

1.1.50

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Some nodes can fail to send a REJECT or REPLY to client under specific network conditions. See Know Issues for more details.

Component Version Information

Components Version Numbers
indy-plenum 1.9.0
indy-node 1.9.0
sovrin 1.1.50
sovtoken 1.0.0
sovtokefees 1.0.0

Major Changes

  • Pluggable Request Handlers have been implemented

Major Fixes

Description Additional Information Ticket Number
Propagates with invalid requests can lead to node crashes INDY-2144
There is no validation of the ISSUANCE_TYPE field for the transaction REVOC_REG_DEF INDY-2142
Reduce CONS_PROOF timeout to speed up catchup under the load INDY-2083

Changes and Additions

Description Additional Information Ticket Number
As a Trustee(s), I need to have a way to set multiple AUTH_RULES by one command INDY-2087
Make more system tests to be ready for Indy Node CD pipeline INDY-2127
Integrate new handlers into the codebase INDY-1861
Define Interfaces needed for View Change Service INDY-1338
Rename TRUST_ANCHOR to ENDORSER INDY-1950
Update PBFT view change plan of attack INDY-2134
Apply a new Docker-in-docker approach for system tests INDY-2131
More tests for pluggable request handlers INDY-2108
Remove ANYONE_CAN_WRITE INDY-1956
[Design] ViewChange protocol must be as defined in PBFT INDY-1290
Batch containing some already executed requests should be applied correctly INDY-1405
Update Pluggable Req Handlers INDY-2097
As a Network Admin, I need to be able to forbid an action in AUTH_RULE, so that no changes in code are needed INDY-2077
Create Builders for handlers INDY-1860
Apply new request handlers approach to Token Plugins ST-523
Implementation: Apply new request handlers approach to Token Req Handlers ST-510
Implementation: Apply new request handlers approach to sovtoken FEEs Handlers ST-578

Known Issues

Description Additional Information Ticket Number
Incorrect request validation INDY-2164
PoA: Request GET_UTXO needs to take into account max limit for a message ST-575

1.1.46

Component Version Information

Components Version Numbers
indy-plenum 1.8.1
indy-node 1.8.1
sovrin 1.1.46

Major Fixes

Description Additional Information Ticket Number
All BuilderNet nodes are restarting every 30-50 seconds INDY-2128
Primaries are not updated in audit ledger if one of the primaries is demoted INDY-2129
A client may receive NACK for a payment transfer request, but the transaction will be eventually ordered (payment transferred) INDY-2122

1.1.45

Notices for Stewards

Payment transaction can return NACK from the pool but in fact it will be eventually ordered (see more details below).

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Pool upgrade to sovrin 1.1.32 and above should be performed simultaneously for all nodes due to txn format changes.

Pool upgrade to indy-node 1.8.0 should be performed simultaneously for all nodes due to audit ledger.

Component Version Information

Components Version Numbers
indy-plenum 1.8.0
indy-node 1.8.0
sovrin 1.1.45

Major Changes

  • Add Transaction Author Agreement Acceptance Mechanisms and Transaction Author Agreement support
  • Configurable Auth rules improvements
  • Stability fixes

Major Fixes

Description Additional Information Ticket Number
Issues with catch up and ordering under the load INDY-2064
Editing of CLAIM_DEF uses auth rules for Adding a Claim Def INDY-2078
Faulty primary can order and write already ordered and written request INDY-1709

Changes and Additions

Description Additional Information Ticket Number
As a Network Admin, I would like to use GET_AUTH_RULE output as an input for AUTH_RULE INDY-2102
Get Transaction Author Agreement Acceptance Mechanisms from the Config Ledger INDY-2071
Support Transaction Author Agreement in Write Requests INDY-2072
Validate transaction author agreement as part of consensus INDY-2073
Write Transaction Author Agreement to Config Ledger INDY-2066
Get Transaction Author Agreement from the config ledger INDY-2067
Write Transaction Author Agreement Acceptance Mechanisms to the Config Ledger INDY-2068
Catch-up should take into account state of other nodes when sending requests INDY-2053
Added FeesAuthorizer ST-535 ST-529
Update SET_FEES and GET_FEES logic for Auth Rules ST-531
Update GET_FEES to use aliases GET_FEE request was added ST-559
Support TAA for XFER txn ST-557
Register auth rule for MINT, XFER_PUBLIC and SET_FEES requests in auth_map ST-536

Known Issues

Description Additional Information Ticket Number
A client may receive NACK for a payment transfer request, but the transaction will be eventually ordered (payment transferred) INDY-2122
Incorrect auth constraint for node demotion INDY-2024
Request GET_UTXO may fail if payment address contains too many UTXOs ST-566

1.1.41

Notices for Stewards

(!) Don't use GET_AUTH_RULE txn - it will cause the service to crash.

(!) There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

(!) Pool upgrade to sovrin 1.1.32 should be performed simultaneously for all nodes due to txn format changes.

(!) Pool upgrade to indy-node 1.7.1 should be performed simultaneously for all nodes due to audit ledger. There should be no fees set up.

Component Version Information

Components Version Numbers
indy-plenum 1.7.1
indy-node 1.7.1
sovrin 1.1.41

Major changes

  • Audit Ledger
  • helps keep all other ledgers in sync
  • helps recovery of pool state by new or restarted nodes
  • can be used for external audit
  • Correct support of multi-signatures
  • Configurable Auth Rules in config state
  • Stability fixes

Major Fixes

Description Ticket Number
Validator-info doesn't show view change information and sometimes shows node info as unknown INDY-2008
Schema can't be written with error "'Version' object has no attribute 'dev'" INDY-2020
Node fails to start after the load INDY-2018
POA: Sovrin TestNet lost consensus INDY-2022
Nodes can fail on first start after upgrading from version without audit ledger to version with audit ledger INDY-2047
Pool is getting out of consensus after a forced view change and writes to all the ledgers INDY-2035
View Change processing - replica ends up with incorrect primaries INDY-1720
Validator node shows False for consensus INDY-2031
Watermarks may not be updated correctly after view change by a lagging node INDY-2060
ATTRIB doesn't have auth rules in auth map INDY-2061
Some nodes are stalled and throw an error under load INDY-2050
Some nodes failed to join consensus after upgrade INDY-2055

Changes and Additions

Description Ticket Number
Implementation: Restore current 3PC state from audit ledger INDY-1946
Implementation (not active): As a user/steward I want to have better understanding of release version and changelog INDY-1992
Implement auth rule maps in config ledger INDY-2001
Add audit ledger INDY-1944
INSTANCE_CHANGE messages should be persisted between restarts INDY-1984
Add updateState method for ConfigReqHandler INDY-2006
Use auth constraints from config ledger for validation INDY-2002
Implementation: Improve catch-up to use audit ledger for consistency INDY-1945
Implement a command to set auth constraints INDY-2003
Debug and validation: Move the auth_map structure to the config ledger INDY-1995
Need to enhance write permissions for Revocation transactions INDY-1554
Implement a command to get auth constraint INDY-2010
Integrate testinfra-based system tests to Indy CD INDY-2016
Debug and Validation: As a user/steward I want to have better understanding of release version and changelog INDY-2019
As a QA I want system tests to be run in parallel in CD pipeline INDY-2028
Debug and Validation: Audit Ledger and improving catch-up to use audit ledger for consistency INDY-1993
Need to track same transactions with different multi-signatures INDY-1757
Debug and Validation: Restore current 3PC state from audit ledger - Phase 1 INDY-2025
A Node need to be able to order stashed requests after long catch-ups INDY-1983
Need to account fields from PLUGIN_CLIENT_REQUEST_FIELDS when calculating digest INDY-1674
Debug and validation: Multi-signature support INDY-2046
Debug and Validation: Restore current 3PC state from audit ledger - Phase 2 INDY-2051

Known issues

Description Ticket Number
Get auth rule txn breaks the indy-node service if plugins are on ST-566

1.1.35

Notices for Stewards

(!) There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

(!) Pool upgrade to sovrin 1.1.32 should be performed simultaneously for all nodes due to txn format changes.

Component Version Information

Components Version Numbers
indy-plenum 1.6.58
indy-node 1.6.83
sovrin 1.1.35

Major Fixes

Description Ticket Number
validator-info to client times out if there are many upgrade attempts by node INDY-1922
Node on Sovrin TestNet did not upgrade automatically INDY-1919
Node that does not upgrade spams the config ledger INDY-1918
Incorrect pool upgrade txn validation INDY-1953
Upgrade appears to have broken "validator-info --nagios" INDY-1920
Node can't order after view change and catch up INDY-1955
Unclear error messages when Trustee send a NYM with the same verkey INDY-1963
A role that has been removed can't be added back INDY-1971

Changes and Additions

Description Ticket Number
Limit the number of attributes in schema INDY-1914
Enable Clear Request Queue strategy INDY-1836
A Node needs to be able to order requests received during catch-up INDY-1876
Network maintenance role INDY-1916
There should always be fresh enough signature of a state INDY-933
Node stops working without any services failure INDY-1949
As a user of Valdiator Info script, I need to know whether the pool has write consensus and when the state was updated the last time INDY-1928
Trust anchor permission not needed for ledger writes INDY-1528

Known issues

Description Ticket Number
Some of the nodes lagging behind during the load test INDY-1965

1.1.34

Notices for Stewards

(!) There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

(!) Pool upgrade to sovrin 1.1.32 should be performed simultaneously for all nodes due to txn format changes.

(!) indy-node 1.6.82 isn't using the latest versions of libindy-crypto (must be 0.9.5) and python3-indy-crypto(must be 0.9.5). Set these packages versions in case of new node manual installation.

Component Version Information

Components Version Numbers
indy-plenum 1.6.57
indy-node 1.6.82
release version number 1.1.34

Major Fixes

Description Additional Information Ticket Number
Added old instance change messages discarding. INDY-1909
Increased ToleratePrimaryDisconnection and bind re-try time. INDY-1836
Added check for None of replica's primary name during logging. INDY-1926

1.1.33

Notices for Stewards

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Pool upgrade to sovrin 1.1.32 should be performed simultaneously for all nodes due to txn format changes.

Component Version Information

Components Version Numbers
indy-plenum 1.6.55
indy-node 1.6.80
release version number 1.1.33

Major Fixes

Description Additional Information Ticket Number
Fixed an issue with the pool doing a restart if there was no consensus. INDY-1896
Fixed an issue with intermittent test failure: test_primary_selection_increase_f INDY-1872
Fixed the throughput class creation bug. INDY-1888

1.1.32

Notices for Stewards

Warning: Embedded command-line tool indy is no longer available. For further pool interaction use the indy-cli package https://github.com/hyperledger/indy-sdk/tree/master/cli

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

Validator-info output has been changed. If you use validator-info as a data source make sure that you have done necessary changes for compatibility.

The INDY-1818 (Init Indy Node should output Base58-encrypted verkey already) affects nodes adding.

Pool upgrade to sovrin 1.1.32 should be performed simultaneously for all nodes due to txn format changes.

There are still some issues with token functionality.

Component Version Information

Components Version Numbers
indy-plenum 1.6.54
indy-node 1.6.79
release version number 1.1.32

Major Fixes

Description Additional Information Ticket Number
Fixed an issue with intermittent test failure: test_primary_selection_increase_f. INDY-1872
Fixed an issue with the pool stopped writing after production load with fees. INDY-1867
Node wasn't validating CLAIM_DEF's filed ref. INDY-1862
Fixed an issue where the Sovrin package couldn't be upgraded. INDY-1850
There was an error with Expected object or value error during batch handling. INDY-1849
Fixed an issue with a node that wasn't on the network was showing as 'unreachable'. INDY-1842
There wasn't enough information about upgrade in journalctl. INDY-1834
Fixed 'aws_manage' playbook that was failing when inventory directory wasn't specified. INDY-1827
A Validator on Sovrin MainNet was failing to upgrade, then was failing to revert. INDY-1824
The pool stopped writing during load testing against domain and pool ledgers together. INDY-1823
The node service was stopping during node key validation. INDY-1820
Made it so that Init Indy Node outputs the Base58-encrypted verkey already. INDY-1818
Investigated slowness on TestNet due to demotion. INDY-1816
Fixed and issue where the new instance was removed after creating it. INDY-1815
Fixed issues relating to upgrading from 1.6.645+ version resulting in errors about packages versions in journalctl. INDY-1781
Fixed an issue with RequestQueue in Replica wasn't clearing after View Change. INDY-1765

Changes - Additions - Known Issues

Description Workaround Ticket
Created a Diagram for Components. INDY-1870
Created a Catch-up Sequence Diagram. INDY-1869
Made a Write and Read request flow. INDY-1868
Created a Plenum Consensus Protocol Diagram. INDY-1851
Made it so that change pool state root hash for BLS-signature is in Commit messages. INDY-1846
Made a 3rd party open source manifest. INDY-1839
Enabled PreViewChange Strategy INDY-1835
Added Names to AWS ec2 instances and security groups. INDY-1828
Securely automated SSH authenticity checking. INDY-1826
Limited RocksDB memory consumption. INDY-1822
Ran a very long load test on a small local pool. INDY-1821
Created AWS tags for pool automation AWS resources. INDY-1813
Adjusted last_ordered_3pc and performed GC when detecting lag in checkpoints on backup. INDY-1795
Improved usability of current pool automation PoC. INDY-1792
Created the ability to refer different groups in the same namespace using one inventory. INDY-1788
Removed security groups at tear-down phase for both tests and playbooks. INDY-1784
Cleared the Requests queue periodically. INDY-1780
Tested ZMQ Memory Consumption with restarting of listener on every X connections. INDY-1776
Got information about how many client connections is usually in progress. INDY-1775
Did a long test with a load pool can handle. INDY-1774
Found out why validation of PrePrepares with Fees takes so long. INDY-1773
Checked why backup instances stop ordering so often. INDY-1772
Made it so that you can perform tests on docker. INDY-1771
Tested ZMQ Memory Consumption with restricted number of client connections. INDY-1770
Ran load tests with file storage. INDY-1769
Changed dependency building for upgrade procedure. INDY-1762
Used persisted last_pp_seq_no for recovery of backup primaries. INDY-1759
Extended the Load Script with GET_TXN. INDY-1756
Avoided redundant static validation during signature verification. INDY-1753
Found out why max node prod time increases during long load test. INDY-1747

Sovrin-related Issues

Description Additional Information Ticket Number
Upgrade to latest Sovrin doesn't work if it depends on a newer IndyNode. INDY-1764
Verify Upgrade of the MainNet to the latest IndyNode. INDY-1763
Failed to upgrade from Sovrin 1.1.24 to Sovrin 1.1.26. INDY-1748

1.1.30

Component Version Information

Components Version Numbers
indy-plenum 1.6.53
indy-anoncreds 1.0.11
indy-node 1.6.78
release version number 1.1.30

Major Fixes

Description Additional Information Ticket Number
Fixed and issue where the ledger statuses and maximal consistency proofs is not canceled. INDY-1740
Fixed an issue where Bug in calling notifier methods in . INDY-1741
Fixed an issue where 35 view changes were happened during 10 minutes after nodes failure because of an invalid request. INDY-1696
Fixed an issue where the requests queue is not cleared in case of reject-nym transactions. INDY-1700
Fixed an issue where throughput critically decreases without causing view_change INDY-1672
Fixed an issue where Node can't catch up large ledger. INDY-1595
Fixed an issue where we were unable to demote node in STN. INDY-1621
Fixed an issue where view changes happen when all responses should be rejected during load testing scenario. INDY-1653
Fixed an issue where node doesn't write txns after disconnection from the rest nodes. INDY-1580
Fixed an issue where throughput is degrading if backup primary is stopped. INDY-1618

Changes - Additions - Known Issues

Description Workaround Ticket
Switch off a replica that stopped because disconnected from a backup primary. INDY-1681
Extended load scripts emulating non-smooth load according to the changes in the core script. INDY-1667
Proof of stability under load. INDY-1607
Investigate Out of memory issues with the current load testing. INDY-1688
Do not re-verify signature for Propagates with already verified requests. INDY-1649
POA: Require multiple signatures for important transactions. INDY-1704
Support all FEEs txns in the load script. INDY-1665
Test domain transactions with FEEs. INDY-1661
3PC Batch should preserve the order of requests when applying PrePrepare on non-primary. INDY-1642
Ability to switch off (remove) replicas with no changes of F value. INDY-1680
A node should be able to participate in BLS multi-signature only if it has a valid proof of possession. INDY-1589
Make validator info as a historical data. INDY-1637
Known Issue: There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.
Known Issue: INDY-1447 - Upgrade failed on pool from 1.3.62 to 1.4.66 Note that INDY-1447 was fixed in indy-node 1.5.68, but it still presents in indy-node 1.3.62 and 1.4.66 code. So, some of the nodes may not to be upgraded during simultaneous pool-upgrade. If this problem will appear, stewards should perform manual upgrade of indy-node in accordance with this instruction.

Upgrade Scripts:

Pool upgrade from indy-node 1.3.62 to indy-node 1.6.78 should be performed simultaneously for all nodes due to txn format changes.

Additional Information:

To reduce the risk of reproducing INDY-1447, it is recommended to use the old CLI for pool upgrade from indy-node 1.3.62.

All indy-cli pools should be recreated with actual genesis files. For more details about txn format changes see [INDY-1421]((https://jira.hyperledger.org/browse/INDY-1421).

1.1.17

Important: Several iterations were done very rapidly between the last release and this one. All of the changes, upgrades, etc... are included in this new release. Simply upgrading will include them all from 1.1.13 until 1.1.17. To see further, specific numerous changes, please reference the appropriate tickets in the Hyperledger Jira ticketing system.

Component Version Information

Components Version Numbers
indy-plenum 1.6.51
indy-anoncreds 1.0.11
indy-node 1.6.73
release version number 1.1.17

Major Fixes

Description Additional Information Ticket Number
Fixed and issue where the pool stopped writing after F change. INDY-1583
Fixed an issue where read_ledger was passing incorrectly formatted stdout and breaks convention INDY-1645
Fixed an issue where the node couldn't catch up a large ledger. INDY-1595
Fixed an issue where the Validator Info may hang for a couple of minutes. INDY-1603
Fixed an issue where the read_ledger tool is not able to read the sovrin plugin ledger. INDY-1548

Changes - Additions - Known Issues

Description Workaround Ticket
Made it so that the 3PC Batch should preserve the order of requests when applying PrePrepare on non-primary. INDY-1642
Made it so that Monitor takes into account requests not passing the dynamic validation when triggering view change. INDY-1643
Improved throughput calculation to reduce a chance of false positive View Changes. INDY-1565
Made it so that the performance of monitor is improved. INDY-1660
Made it so that Stewards, can have a script that can generates Proof of possession for their BLS key. That value can now be used in a NODE txn. INDY-1588
Added Support Proof of Possession for BLS keys. INDY-1389
Made it so that the average is not used when calculating total throughput/latency of backups. INDY-1582
Made it so that any client requests are discarded during view change. INDY-1564
Created a simple tool to show graphical representation of some common metrics. INDY-1568
Changed default configs for better performance and stability. INDY-1549
Known Issue: Upgrade failed on pool from 1.3.62 to 1.4.66. Note that INDY-1447 was fixed in indy-node 1.5.68, but it still presents in indy-node 1.3.62 and 1.4.66 code. So, some of the nodes may not to be upgraded during simultaneous pool-upgrade. If this problem will appear, stewards should perform manual upgrade of indy-node in accordance with this instruction (!) To reduce the risk of reproducing INDY-1447, it is recommended to use old CLI for pool upgrade. INDY-1447

Upgrade Scripts:

Pool upgrade from indy-node 1.3.62 to indy-node 1.6.73 should be performed simultaneously for all nodes due to txn format changes.

There must be sovrin package upgrade to 1.1.17 version after indy-node package upgrade. You need to specify package=sovrin in pool-upgrade command to do this.

Additional Information:

All indy-cli pools should be recreated with actual genesis files.

For more details about txn format changes see INDY-1421 .

There are possible OOM issues during 3+ hours of target load or large catch-ups at 8 GB RAM nodes pool so 32 GB is recommended.

1.1.13

Component Version Information

Components Version Numbers
indy-plenum 1.6.49
indy-anoncreds 1.0.11
indy-node 1.6.70
release version number 1.1.13

Major Fixes

Description Additional Information Ticket Number
Fixed and issue where several nodes (less than f) were getting ahead the rest ones under load. INDY-1473
Fixed an issue where the pool has stopped to write txns. INDY-1539
Fixed an issue where re-send messages to disconnected remotes. INDY-1497
Fixed an issue where the pool stopped writing under 20txns/sec load. INDY-1478
Fixed an issue where 1.3.62 -> 1.5.67 forced upgrade without one node in schedule failed. INDY-1519
Fixed an issue where tmp.log must have unique name. INDY-1502
Fixed an issue where a node needed to hook up to a lower viewChange. INDY-1199
Fixed an issue where the one of the nodes lagged behind others after forced view changes. INDY-1470
Made it so that View Change should not be triggered by re-sending Primary disconnected if Primary is not disconnected anymore. INDY-1544

Changes - Additions - Known Issues

Description Workaround Ticket
Made it so that as a Trustee running POOL_UPGRADE txn, you can specify any package depending on indy-node, so that the package with the dependencies get upgraded. INDY-1491
Made it so that Monitor is reset after the view change. INDY-1555
Made it so that GC by Checkpoints are not triggered during View Change. INDY-1545
Made it so that the validator info must show committed and uncommitted roots for all states. INDY-1542
Explored timing and execution time. INDY-1475
Memory leaks profiling. INDY-1493
Bound connection socket to NODE_IP INDY-1531
Enable TRACK_CONNECTED_CLIENTS_NUM option INDY-1496
Updated revocation registry delta value during REG_ENTRY_REVOC writing. INDY-1378
Support latest SDK in Indy Plenum and Node. INDY-1480
Latency measurements in monitor are windowed. INDY-1468
Trust anchor permissions are not needed for ledger writes. INDY-1528
Known Issue: Docker pool can't be built because of new python3-indy-crypto in sdk repo. The problem described in INDY-1517 will be fixed in the next release of indy-node. Workaround for this problem is to add python3-indy-crypto=0.4.1 to the list of packages to be installed. INDY-1517
Known Issue: Upgrade failed on pool from 1.3.62 to 1.4.66. Note that INDY-1447 was fixed in indy-node 1.5.68, but it still presents in indy-node 1.3.62 and 1.4.66 code. So, some of the nodes may not to be upgraded during simultaneous pool-upgrade. If this problem will appear, stewards should perform manual upgrade of indy-node in accordance with this instruction:(!) To reduce the risk of reproducing INDY-1447, it is recommended to use old CLI for pool upgrade. INDY-1447

Upgrade Scripts:

Pool upgrade from indy-node 1.3.62 to indy-node 1.6.70 should be performed simultaneously for all nodes due to txn format changes.

There must be sovrin package upgrade to 1.1.13 version after indy-node package upgrade. You need to specify package=sovrin in pool-upgrade command to do this.

Additional Information:

All indy-cli pools should be recreated with actual genesis files.

For more details about txn format changes see INDY-1421 .

1.1.12

Component Version Information

Components Version Numbers
indy-plenum 1.5.48
indy-anoncreds 1.0.11
indy-node 1.5.68
release version number 1.1.12

Major Fixes

Description Additional Information Ticket Number
Fixed and issue where logs were appearing in the old CLI. INDY-1471
Fixed an issue where there were numerous blacklists under high loads. INDY-1461
Fixed an issue where the pool stopped writing after 1114k txns (different view_no). INDY-1460
Fixed an issue where the "AttributeError: NoneType object has no attribute 'request' during load"; was appearing. INDY-1464
Fixed an issue where the validator-info was reading an empty file. INDY-1406
Fixed an issue where validator-info -v --json wasn't producing valid JSON. INDY-1443
Fixed an issue where the first Pre-Prepare message had incorrect state trie root right after view_change (on master replica). INDY-1459
Fixed an issue where the pool could not order transactions because Node set incorrect watermarks after its restart. INDY-1455
Fixed an issue where the pool stopped working due to several incomplete view changes. INDY-1454
Fixed an issue where the node crashes on _remove_stashed_checkpoints. INDY-1427
Fixed an issue where memory was running out during non-completed viewChange process (under load). INDY-1360
Fixed an issue where part of nodes continued ordering txns after incorrect state trie under load. INDY-1422
Fixed an issue where the upgrade failed on pool from 1.3.62 to 1.4.66. INDY-1447
Fixed an issue where a forced upgrade from 1.3.62 -> 1.5.67 without one node in schedule failed. INDY-1519

Changes - Additions - Known Issues

Description Workaround Ticket
Implemented periodic restart of client stack to allow new clients to connect. INDY-1431
Got rid of peersWithoutRemotes. INDY-1467
High Watermark on backup may be reset to 300. INDY-1462
We now allow optional field in node-to-node and client-to-node. INDY-1494
Catchup during view change may last forever under the load. INDY-1463
Propagate Primary mode should not be set for already started view change. INDY-1458
Catchup needs to be finished during high load. INDY-1450
Included reviewed logging strings in Indy. INDY-1416
Added benchmark performance impact of recorder tool. INDY-1483
Decreased the amount of logging with INFO level. INDY-1311
Made it so that throughput measurements in monitor should are windowed. INDY-1435
Limited the number of requested PROPAGATES in MessageRequests. INDY-1386
Made it so that any client requests during view change are not processed. INDY-1453
Made it so that a node must send LEDGER_STATUS with correct last ordered 3PC after catch-up. INDY-1452
Fixed calculation of prepared certificates during View Change. INDY-1385
Made it so that catchup should not be interrupted by external events. INDY-1404
Known Issue: Upgrade failed on pool from 1.3.62 to 1.4.66. Note that INDY-1447 was fixed in indy-node 1.5.68, but it still presents in indy-node 1.3.62 and 1.4.66 code. So, some of the nodes may not to be upgraded during simultaneous pool-upgrade. If this problem will appear, stewards should perform manual upgrade of indy-node in accordance with this instruction: (!) To reduce the risk of reproducing INDY-1447, it is recommended to use old CLI for pool upgrade. INDY-1447

Upgrade Scripts:

Pool upgrade from indy-node 1.3.62 should be performed simultaneously for all nodes due to txn format changes.

Additional Information:

All indy-cli pools should be recreated with actual genesis files. For more details about txn format changes see INDY-1421.

1.1.11

Component Version Information

Components Version Numbers
indy-plenum 1.4.45
indy-anoncreds 1.0.11
indy-node 1.4.66
release version number 1.1.11

Major Fixes

Description Additional Information Ticket Number
Fixed and issues where one of the nodes stopped writing after 44287 txns with errors in status. INDY-1410
Fixed an issue where the pool stopped accepting transactions on 5731 txns (1 sec delays, no logging). INDY-1365
Fixed an issue where the pool stopped writing after ~300,000 txns from 5 clients INDY-1315
Fixed an issue where STN was not accepting transactions with only one node down. INDY-1351
Fixed an issue where the pool stops taking txns at ~178k txns written in ledger. INDY-1260
Fixed an issue where ReqIdrToTxn does not store information about the ledger. INDY-1327
Made simple Timeout fixes of the current View Change protocol. INDY-1341
Fixed an issue where the migration fails in case of upgrade to version with new transactions format. INDY-1379
Fixed an issue where --network parameter of read_ledger doesn't work. INDY-1318
Fixed an issue where the /var/log/indy/validator-info.log was inappropriately owned by root. INDY-1310
Created a fix around the issues found in the current logic of catch-up. INDY-1298
Fixed GetValidatorInfo so it has correct validation for signature and permissions. INDY-1363
Fixed an issue where there was an unhandled exception during node working. INDY-1316
Fixed an issue where validator-info and read_ledger were giving inconsistent responses in node on provisional. INDY-1219
Fixed an issue where the pool stops taking txns at 3000 writing connections. INDY-1259

Changes - Additions - Known Issues

Description Workaround Ticket
Reviewed and replaced assert with exceptions in indy-plenum where needed. INDY-810
Tuned RocksDB options for the best performance. INDY-1245
Created a migration guide from Indy-node 1.3 to 1.4. INDY-1392
Сhanged a key in the requests map and field reqIdr in Pre Prepare and Ordered. INDY-1370
Investigated issues found during load testing of 25-nodes pool with increased timeouts for catchups and viewchange. INDY-1400
We now support binding on separate NICs for Client-to-Node and Node-to-Node communication. INDY-1332
Added short checkpoints stabilization without matching digests. INDY-1329
Added indy-crypto package to the hold list. INDY-1323
Removed ledger status based catch-up trigger together with the wrong catch-up workflow. INDY-1297
Read-ledger without storage copy in case of RocksDB (RocksDB read-only mode support). INDY-1243
Applied state machine to Catchup code. INDY-971
Refactored the common Request structure. INDY-1124
Refactored the common transactions structure. INDY-1123
We now support the new libindy with changed txn format. INDY-1319
Explored config parameters to find the best performance/stability settings. INDY-1334
Extended the Validator Info tool to provide more information about the current state of the pool. INDY-1175
A Steward needs to be able to get validator-info from all nodes. INDY-1184
Modified existing load scripts for a better load testing. INDY-1279
Performed a migration from LevelDB to RocksDB INDY-1244
A Trustee needs to be able to restart the pool in critical situations. INDY-1173
Move the log compression into separate process. INDY-1275
Known Issue: There's an incorrect read_ledger info with seq_no parameter. INDY-1415
Known Issue: Pool upgrade should be performed simultaneously for all nodes due to txn format changes. All indy-cli pools should be recreated with actual genesis files.
List of breaking changes for migration from indy-node 1.3 to 1.4: 1.3-1.4 Migration Guide

Upgrade Scripts:

Pool upgrade should be performed simultaneously for all nodes due to txn format changes.

All indy-cli pools should be recreated with actual genesis files.

CLI Upgrading:

Old CLI (indy):

upgrade from 1.3 to 1.4 version delete ~.ind-cli/networks/<network_name>/data folder replace both old genesis files by new ones (from 1.4 node or from sovrin repo)

New CLI (indy-cli):

upgrade from 1.4 to 1.5 version recreate indy-cli pool using 1.4 pool genesis file (from 1.4 node or from sovrin repo)

Additional Information:

List of breaking changes for migration from indy-node 1.3 to 1.4:

https://github.com/hyperledger/indy-node/blob/master/docs/1.3\_to\_1.4\_migration\_guide.md

IndyNode 1.4 and LibIndy 1.5 compatibility:

General

By default LibIndy 1.5 will be compatible with IndyNode 1.3 (current stable), and not 1.4 (the new one).

LibIndy 1.5 can become compatible with IndyNode 1.4 if indy_set_protocol_version(2) is called during app initialization.

Guideline for teams and apps

Applications can freely update to LibIndy 1.5 and still use stable Node 1.3

If an app wants to work with the latest master or Stable Node 1.4, then they need to support breaking changes (there are not so many, mostly a new reply for write txns as txn format is changed, see 1.3_to_1.4_migration_guide.md)

call indy_set_protocol_version(2) during app initialization

Use https://github.com/hyperledger/indy-sdk/blob/b4a2bb82087e2eafe5e55bddb20a3069e5fb7d0b/cli/README.md#old-python-based-cli-migration to export dids from your old CLI wallet to the new one (new indy-cli).

1.1.10

Component Version Information

Components Version Numbers
indy-plenum 1.2.42
indy-anoncreds 1.0.11
indy-node 1.3.62
release version number 1.1.10

Major Fixes

Description Additional Information Ticket Number
Fixed an issue where the STN was losing consensus. INDY-1256
Fixed an issue where we were unable to use the read_ledger tool with the parameter "to". INDY-1284
Fixed the upgrade from 1.2.223 (1.3.55 stable analogue) to 1.3.410 (rocksdb) wasn't working. INDY-1330

Changes - Additions - Known Issues

Description Workaround Ticket
Support was added for supervisord. https://github.com/hyperledger/indy-node/pull/588
Indy-node dependencies are fixed.

Upgrade Scripts:

None for this release.

Additional Information:

None at this time.

1.1.9

Component Version Information

Components Version Numbers
indy-node 1.3.57
release version number 1.1.9

Major Fixes

Description Additional Information Ticket Number
The Node was restarting because of an "Out of memory" error. INDY-1238
The pool was not working after not simultaneous manual pool upgrades. INDY-1197
When adding a new schema, field 'attr_names' of schema json can be an empty list. INDY-1169
This prevents an Identity Owner from creating a schema or claimDef. INDY-1111
There was the same primary for both instances 0 and 1. INDY-1112
The node logs were being duplicated in syslog. INDY-1102
It was possible to create several nodes with the same alias. INDY-1148
There was ambiguous behavior after node demotion. INDY-1179
One of the nodes were not responding to libindy after several running load tests. INDY-1180
When returning N-F nodes to the pool, "View change" was not occurring if the Primary node was stopped. INDY-1151
There was a failed restart after getting the "unhandled exception (KeyError)". INDY-1152
Fixed a bug where you were unable to install indy-node if sdk repo is in sources.list INDY-1269

Changes - Additions - Known Issues

Description Workaround Ticket
Made it so that a developer can distinguish logs of each replica. INDY-1186
Made it so a developer, can track the path of each request. INDY-1187
Made it so that you can use RocksDB as a key-value storage. INDY-1205
Refactored the common Request structure. INDY-1124
Made it so that it supports anoncreds revocation in Indy. INDY-680
Made it so that it supports REVOC_REG_DEF transaction. INDY-1134
Made it so that it supports GET_REVOC_REG_DEF request. INDY-1135
Made it so that it supports REVOC_REG_ENTRY transaction. INDY-1136
Made it so that it supports GET_REVOC_REG request. INDY-1137
Made it so that it supports getting state root by timestamp. INDY-1138
Got rid of the RAET code. INDY-1057
Incubation: Move CI part of pipelines to Hyperledger infrastructure. INDY-837
Made it so that a user can revoke a connection by rotating the new key to nothing. INDY-582
Known Issue: Define the policy how to restore node from the state when it's stashing all the reqs and there is a risk of running out of memory. INDY-1250
Known Issue: Re-promoted node cannot hook up to a lower viewChange. INDY-1199
Known Issue: One of the nodes does not respond to libindy after several running load test. INDY-1180
Known Issue: One node fails behind others during the load_test with a high load. INDY-1188
Known Issue: Pool can be broken by primary node reboot in case of network issues between nodes. Note: RocksDB was added as dependency (INDY-1205). It is used for revocation, but the rest part of node functionality is still using LevelDB. INDY-1256

Upgrade Scripts

None for this release.

Additional Information:

None at this time.

1.1.8

Important: Upgrade to this version should be performed simultaneously for all nodes (with force=True).

Component Version Information

Components Version Numbers
indy-plenum 1.2.34
indy-anoncreds 1.0.11
indy-node 1.3.55
release version number 1.1.8

Major Fixes

Description Additional Information Ticket Number
Transactions were missing from the config ledger after the upgrade. INDY-799
The node was broken after a load_test.py run. INDY-960
The pool stopped taking transactions after sending 1,000 simultaneous transactions. INDY-911
The pool stopped working: Node services stop with 1,000 simultaneous clients doing GET_NYM reads INDY-986
The node is broken after adding it to the pool. INDY-948
The generate_indy_pool_transactions command can be run only by an indy user. INDY-1048
Made it so that updates to existing Schemas are not allowed. INDY-1035
The pool was unable to write txns after two nodes adding. INDY-1018
Fixed a bug where it was possible to override CLAIM_DEF for existing schema-did pair. INDY-1083
Fixed a bug where here was a huge amount of calls and a lot of execution time in kv_store.py. INDY-1077
One of added nodes wasn't catching up. INDY-1029
The pool stopped working and lost consensus while new node was performing a catch-up. INDY-1025
Performing a View Change on large pools of 19 or more nodes can cause pool to stop functioning. INDY-1054
Performing a View Change issue stopped the pool from accepting new transactions. INDY-1034
We were unable to send transactions in STN. INDY-1076 INDY-1079
Replica.lastPrePrepareSeqNo may not be reset on view change. INDY-1061
We were unable to send an upgrade transaction without including demoted nodes. INDY-897
The Nym request to STN was resulting in inconsistent responses. INDY-1069
The validator node was being re-promoted during view change. INDY-959
There was a false cancel message during an upgrade. INDY-1078
Transactions were being added to nodes in STN during system reboot.. INDY-1045
There were problems with nodes demotion during load test. INDY-1033
The node monitoring tool (email plugin) wasn't working. INDY-995
ATTRIB transaction with ENC and HASH wasn't working. INDY-1074
When returning N-F nodes to the pool, View Change does not occur if Primary node is stopped. INDY-1151
We were unable to recover write consensus at n-f after f+1 descent. INDY-1166
Newly upgraded STN fails to accept transactions (pool has been broken after upgrade because of one not upgraded node). INDY-1183
We were unable to submit upgrade transactions to STN. INDY-1190

Changes - Additions - Known Issues

Description Workaround Ticket
Added indy-sdk test dependency to plenum and use indy-sdk for plenum tests. INDY-900 INDY-901
Published docker images to dockerhub. INDY-962
Simplified the view change code. INDY-480
Refactored config.py to reflect file folder re-factoring for Incubation. INDY-878
Added Abstract Observers Support. INDY-628
Moved scripts from sovrin-environment to one of Indy repos. INDY-1055
Got rid of Sovrin dependency in the environment scripts. INDY-1064
Updated information in "Getting Started with Indy". INDY-1062
Updated information in "Setting Up a Test Indy Network in VMs". INDY-1062
Add iptables rules to limit the number of clients connections. INDY-1087
Knowledge transfer on Indy build processes. INDY-1088
Incubation: Move CI part of pipelines to Hyperledger infrastructure. INDY-837
Made it so that a user can revoke a connection by rotating the new key to nothing. INDY-582
Client needs to be able to make sure that we have the latest State Proof. INDY-928
Created it so that anyone could have access to an up-to-date Technical overview of plenum and indy. INDY-1022
Known Issue: Pool has lost consensus after primary demotion (with 4 nodes setup only). INDY-1163
Known Issue: Ambiguous behavior after node demotion. INDY-1179
Known Issue: One of the nodes does not respond to libindy after several running load test. INDY-1180
Known Issue: Pool does not work after not simultaneous manual pool upgrades. INDY-1197
Known Issue: Pool stops working if the primary node was not included to schedule in the upgrade transaction. INDY-1198

Additional Information:

Node promoting is not recommended for 1.3.52 version according to known issues because backup protocol instances may work incorrectly until next view change.

As mentioned above, upgrade to this version should be performed simultaneously for all nodes (with force=True).

1.1.7

Component Version Information

Components Version Numbers
indy-plenum 1.2.29
indy-anoncreds 1.0.11
indy-node 1.2.50
release version number 1.1.7

Major Fixes

Description Additional Information Ticket Number
A node was maintaining a pace with the network exactly 12 transactions behind. INDY-759
New nodes added to an existing pool were unable to sync ledgers with the pool. INDY-895
Scheduled upgrades were happening at the current time on some of the nodes. INDY-231
Some nodes were not restarting after a canceled pool upgrade. INDY-157
Sovrin logs were insufficient for failed upgrade. INDY-801
A node was getting the wrong upgrade_log entries after restarting and was running the wrong upgrade. INDY-917
An earlier pool_upgrade was not happening when there was an upgrade to schedule to happen in the future. INDY-701
A validator was running instance change continually on the live pool. INDY-932
New nodes added to an existing pool were unable to participate in consensus after the upgrade. INDY-909
The node logs were repeating the message, "NodeRequestSuspiciousSpike suspicious spike has been noticed." INDY-541
Unable to catch up the agent if a validator was down. INDY-941
The pool was unable to write nyms after BLS keys enabling. INDY-958
The last pool node is failed to upgrade; during a pool upgrade. INDY-953
State Proof creating is fixed. INDY-954
State Proof verifying is fixed. INDY-949

Changes - Additions - Known Issues

Description Workaround Ticket
Signed State implementation INDY-670
State Proofs implementation INDY-790
Removed all non-Indy branding from the indy-plenum repo. INDY-829
Removed all non-Indy branding from the indy-anoncreds repo. INDY-855
Removed all non-Indy branding from the indy-node repo. INDY-830
Backward compatibility of nodes with state proofs support with old clients. INDY-877
Supported rebranding in sovrin package. INDY-880
Supported rebranding in Docker and Vagrant environments of sovrin-environments. INDY-891
Support of multiple pool networks by Indy Node. INDY-831
Support of multiple pool networks by Indy Client (CLI). INDY-832
Created proper file folder paths for system service. INDY-833
Client needs to be able to send read requests to one Node only. INDY-927
Client needs to be able to make sure that we have the latest State Proof. INDY-928
Known Issue: Node is broken after load_test.py run INDY-960

Additional Information:

Mapping of all file/folder changes are located here.

Upgrade Steps

  1. Send Pool Upgrade command so all nodes upgrade.

  2. Sometime later each Steward will need to do the following steps to add their BLS Keys:

Steps to Add BLS Keys

From the Validator Node:

  1. Generate a new 32-byte seed for the bls key (we recommend pwgen):

$ sudo apt install pwgen

$ pwgen -s -y -B 32 1

If the output has a single-quote symbol ('), rerun until it doesn't.

NOTE: This is not your Steward or Node seed.

  1. Record the seed somewhere secure.

  2. Switch to the indy user.

$ sudo su - indy

  1. Configure the BLS key.

$ init_bls_keys --name <NODE_ALIAS> --seed '<SEED>'

The --seed is the seed you generated above, and will be used to create the BLS key.

Example with Seed:

$ init_bls_keys --name Node1 --seed 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

Capture the stdout at the end of the output, which looks like the following, and record it.

BLS Public key is 3AfkzUZVn2WT9mxW2zQXMgX39FXSY5qzohnMVpdvNS5KSath1YG5Ux4u9ubTFTaP6W55XX9Yx7xPWeYos489oyY53WzwNBG7X4o32ESnZ9xacLmNsQLBjqc6oqpWGTbEXv4edFTrZ88n93sEh4fjFhQMumaXxDfWJgd9aj7KCSpf38F

  1. Exit the indy user.

$ exit

From the CLI Node:

  1. Manually upgrade the CLI.

$ sudo apt update

$ sudo apt upgrade

  1. Launch the CLI.

$ indy

The first time running the upgraded CLI you will be prompted to migrate your previous settings. Answer "Yes."

  1. Connect to the pool.

indy> connect live

  1. Set your Steward as the signer in the CLI.

indy@live> use DID <Steward DID>

Example:

indy@live> use DID Th7MpTaRZVRYnPiabds81Y

Note: If your DID is not found in the wallet, you will need to use your steward seed:

indy@live> new key with seed <steward_seed>

  1. Now you will send a node transaction like what you did when you added the node to the pool. You will add the BLS key as a new parameter to the transaction to update the pool ledger with this additional public key. For 'dest', use the same base58 value for this that was used when you initially onboarded your VM onto the provisional pool.

indy@live> send NODE dest=<node_dest> data={'alias':'<node name>','blskey': '<key_generated_by_init_bls_keys>'}

Example:

indy@live> send NODE dest=Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv data={'alias':'Node1','blskey': '3AfkzUZVn2WT9mxW2zQXMgX39FXSY5qzohnMVpdvNS5KSath1YG5Ux4u9ubTFTaP6W55XX9Yx7xPWeYos489oyY53WzwNBG7X4o32ESnZ9xacLmNsQLBjqc6oqpWGTbEXv4edFTrZ88n93sEh4fjFhQMumaXxDfWJgd9aj7KCSpf38F'}

Note: The 'node_dest' value can be found on the node with sudo read_ledger --type pool.

Questions and Answers

BLS Keys for State Proofs

What does BLS stand for?

Boneh-Lynn-Shacham - The BLS signature scheme is used to verify that a signer is authentic.

How does the CLI use State Proof for confirmation?

When the CLI requests information about a transaction it checks the BLS signatures to verify the transaction was written by nodes that are part of the validator pool. The CLI sends a request to one node (arbitrary one). If the Reply doesn't have a State Proof, or the reply is incorrect/invalid, then CLI falls back to sending requests to all Nodes and waiting for f+1 equal Replies.

What if not all nodes in the pool have BLS signing keys for a transaction?

Transactions only get signed if all nodes reaching consensus can sign it (>= n-f Nodes with correct BLS signatures).

Can the bls_seed be any 32 character seed like the Steward seed?

Yes.

When adding a new node to an existing pool where do I find my BLS key?

When initializing your node using init_indy_node the output will display the keys for the node including the BLS key. It can be found in /var/lib/indy/<network_name>/keys/<node_name>/bls_keys/bls_pk file (e.g.: /var/lib/indy/sandbox/keys/Node1/bls_keys/bls_pk)

When you send the transaction to add the new node to the pool it will also contain the BLS key in the transaction shown in this example.

Example of send node command with BLS for 5th node in test pool:

send NODE dest=4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc data= {'client_port': 9702, 'client_ip': '10.0.0.105', 'alias': 'Node5', 'node_ip': '10.0.0.105', 'node_port': 9701, 'services': ['VALIDATOR'], 'blskey':'2RdajPq6rCidK5gQbMzSJo1NfBMYiS3e44GxjTqZUk3RhBdtF28qEABHRo4MgHS2hwekoLWRTza9XiGEMRCompeujWpX85MPt87WdbTMysXZfb7J1ZXUEMrtE5aZahfx6p2YdhZdrArFvTmFWdojaD2V5SuvuaQL4G92anZ1yteay3R'}

Can I use a seed when generating my BLS keys?

For a new node when using init_indy_node if you specify a seed for this script that same seed is used to generate your BLS keys.

For existing nodes being upgraded to 1.2.50, which includes state proofs, you would use the script init_bls_keys where you can specify a 32-character seed on the command line.

init_bls_keys --name <NODE_ALIAS> --seed '<SEED>'

After running init_bls_keys, Stewards of existing nodes will be required use their CLI node to update their validator's information on the ledger to include the bls keys:

send NODE dest=<node_dest> data={'alias':'<node name>', 'blskey': '<key_generated_by_init_bls_keys>'}

Multi-network and indy_config.py

Where do I find the configuration file settings?

With file and folder changes the new location for indy_config.py is in the directory location /etc/indy/. The configuration file has a new setting called "NETWORK_NAME" which is used to identify which network and associated genesis transaction files to use, such as sandbox or live. If adding a new node to a live pool, change this setting before initializing the node. The genesis files are now located in their own directory based off the network name "/var/lib/indy/NETWORK_NAME". The defaults are live, local, and sandbox. Setting the "NETWORK_NAME" in the indy_config.py file will determine which network is used. The default setting in the indy_config.py file is ""NETWORK_NAME=sandbox".

1.1.6 Hot Fix

Component Version Information

Components Version Numbers
indy-plenum 1.1.27
indy-anoncreds 1.0.10
indy-node 1.1.43
release version number 1.1.6

Major Fixes

Description Additional Information Ticket Number
Added a migration script which eliminates redundant fields with null values from legacy transactions in the domain ledger. INDY-895 INDY-869
Added a constraint on version field of POOL_UPGRADE transaction that denies values lower than the current installed version. INDY-895 INDY-869
Added prevention of upgrade to a lower version to Upgrader class. INDY-895 INDY-869
Fixed a bug in Upgrader class in search for a POOL_UPGRADE cancel transaction for the last POOL_UPGRADE start transaction. INDY-895 INDY-869
Added a test verifying prevention of upgrade to a lower version. INDY-895 INDY-869
Corrected existing tests according to introduced prevention of upgrade to a lower version. INDY-895 INDY-869

1.1.6

Component Version Information

Components Version Numbers
indy-plenum 1.1.27
indy-anoncreds 1.0.10
indy-node 1.1.37
release version number 1.1.6

Major Fixes

Description Additional Information Ticket Number
Stewards can now demote and promote their own nodes. INDY-410
Fixed problem with timezones for timestamp in a transaction. INDY-466
Limited incoming message size from 128k to 128MB (Temporary solution). INDY-25
Fixed send CLAIM_DEF command. INDY-378
Masked private information in the CLI logs/output. INDY-725
Fixes crashes on ubuntu 17.04. INDY-8
Python interpreter is executed in optimized mode. INDY-211
Memory leak fixes. INDY-223
Some minor stability fixes.
Fixed a problem with migration during manual upgrades. INDY-808
Fixed a problem with the message length limitation. This was a permanent solution of INDY-25. INDY-765
Fixed a problem when the pool was writing transactions when more than F nodes were stopped. INDY -786
Fixed a problem when the pool was broken after processing lots of transactions at once. INDY-760
Fixed a problem when the pool doesn't come back to consensus in cases when less than n-f nodes are alive. INDY-804
Partially fixed a problem when the pool responded with outdated data. INDY-761

Changes - Additions - Known Issues

Description Workaround Ticket
New ledger serialization is supported and Leveldb is used as a storage for all ledgers : msgpack is used for the ledger serialization (both transaction log and merkle tree).
The new serialization change created changes to the directory structure for the nodes. The directory name changes are located on a node under .sovrin/data/nodes/<node name>/<directories>. The change removes the ledger files as plain text files and creates them as binary files. A new tool was created to view the ledger entries called read_ledger. This tool also provides you with a count of the transactions. To learn more about this tool and to see a list of available commands, run this as the sovrin user: read_ledger --h
Genesis transaction files are renamed adding a _genesis to the end of each file name.
Added the commands to the POOL_UPGRADE to support downgrade and re-installation. However both have issues and should not be used at this time. INDY-735 INDY-755
Fixes to upgrade procedure, in particular an issue which caused an infinite loop. INDY-316
A new CLI command was added to ease the process of rotating a verification key (verkey). The command is change current key or change current key with seed xxxxx.
Improvements to log messages.
Publishing only to repo.sovrin.org
In your sources.list you only need the entry "deb https://repo.evernym.com/deb xenial stable".
Implemented a command line tool to provide validator status. INDY-715
"Debug" mode for tests was moved to parameter. INDY-716
Log levels were changed on some debug level messages to an info level. INDY-800
If the pool loses enough nodes and cannot reach consensus when enough nodes become available, the pool will still not reach consensus. If you restart all the nodes in the pool, it will start reaching consensus again. INDY-849

1.0

Major Features

Description Additional Information
Identities can be created, and their exercise and evolution can be controlled by ed25519 key pairs (where the public verkey is stored and updated on the ledger).
Identities are fully self-sovereign by default. Guardianship, as described in The Sovrin Provisional Trust Framework, is also possible. The Sovrin Provisional Trust Framework
Identities can have an agent, and the agent endpoint can be declared on the ledger.
Identities are associated with the roles defined in The Sovrin Provisional Trust Framework. Specifically see: trustee, steward, trust anchor, TGB member, and normal identity. The Sovrin Provisional Trust Framework
Identities with special roles have the appropriate permissions. http://bit.ly/2eRfeIV
The ledger supports a core set of transaction types suitable for early workflows http://bit.ly/2v1OAmO
Byzantine consensus is robust and transparent, providing diffuse trust.
Ledger state is enstructured as a Patricia Trie. This is the precondition for implementing Observer nodes and for having high-trust answers with any number of invalid high-trust answers with any number of malicious intermediaries.
The network generates log files that stewards can use as an audit trail.
The network can be upgraded without manual tasks for stewards, using a POOL_UPGRADE transaction.
Batching is functional, allowing bulk submission and reasonable throughput. (Note, however, that very large batches of more than a few thousand transactions are discouraged because they consume network resources unfairly and may cause brownouts for other users.)
Simple safeguards are in place to react to hacking and DDoS attempts.
Terminology in the CLI has been updated to align with usage in the Provisional Trust Framework and the DID specification. In particular, this release is DID-centric, whereas earlier previews were cryptonym-centric.
Registering simple schemas for claims, and then deriving claims and proofs from them, is possible. This supports many of the use cases envisioned for simple verifiable claims, but not the advanced privacy-preserving features offered by Camenisch-Lysyanskya-style schemes.

Deferred Features

Description Additional Information
DDO support: Currently, the DID support in Sovrin uses ATTRIB transactions. This limits the complexity and granularity of identity control. Full DDO support will be necessary to support models such as individual signers being empowered to represent a corporation, for example.
Rich schema: W3C verifiable claim schema thinking is still evolving, and we need time to experiment with the best ways to represent certain constructs. Very complex credential types may not yet be representable.
Revocation: Although the cryptographic foundation for revocation is fully implemented in the anoncreds component that Sovrin depends on, the ledger currently lacks a transaction type to update accumulators. This means initial claims won't support revocation.
Observers and state proofs: Although the state trie is present in the ledger, code to return a state proof with each answer from the ledger has not yet been released. This means clients need to hear f+1 consistent answers from validator nodes before they can trust any response from the ledger. The Indy SDK supports this type of response checking, and will cut over to using the more efficient state proof mechanism as soon as Sovrin does. Once we have that mechanism in place, observer nodes become trustworthy because they cannot simulate state proofs. This drastically increases the scale and performance of the system. Indy SDK

Known Issues

Description Workaround Ticket
Stewards demoting a node: Stewards should be able to demote and promote their own node. Demoting a node removes it from the pool so it does not participate in the pool reaching consensus. INDY-410
Ubuntu 17.04 support: Currently the CLI and Node are not supported on Ubuntu 17.04. INDY-8
Timezones not matching: If a system timezone of the primary does not match that of the other nodes the pool cannot function. 1. Shut down the sovrin-node service. 2. Use "sudo dpkg-reconfigure tzdata" to change your timezone to UTC. (Select "None of the Above" and then "UTC".) INDY-466
Installing client as root: Installing the client as root does not allow users to access the transaction files and they are not able to connect to the pool. INDY-24

New Terminology Changes to the CLI

Please be aware that the following terminology has changed:

  • . keyring is now "wallet"
  • . link is now "connection"
  • . invitation is now "request"
  • . identifier is now "DID"

Emergencies

For emergencies where the Network is under a DOS attack, there is a new command for Trustees only that will put the Pool into a read-only mode.

send POOL\_CONFIG writes=False

or optional

send POOL\_CONFIG writes=False force=False

The following are bool parameters: writes and force. Writes is required. Force is not required. The default for force is False.

Other Important Information

Keyrings to Wallets Directory

With the terminology change from "keyring" to "wallet," the wallets that have already been created will be the client in the .sovrin/keyrings/live directory. When you upgrade, you'll have to start the CLI and connect to live just once for the wallet directory to be created. The new directory is .sovrin/wallets/live. The upgrade does not move your existing wallets from /keyrings/live to /wallets/live since the wallet directory does not exist until you connect for the first time. You will need to move the existing wallets from /keyrings/live to /wallets/live and they will be ready for use.

Time Stamps

With the latest builds (which will be on your new stable) you will start to see a time stamp in the ledger for the transactions. It will look like the following:

V4SGRU86Z58d6TV7PBUe6f|1500090331922505|5iBLSL3iujkNsX4RiGN7RjDk5UVtE8GHsn8XHsRsmhUUN9Jz8GEmYtpPrso8e96jo3YafhYSwPkd9QXYA3wYyst4|1500090332|1|Pqs6Tyj1huDsjKYsiF8w4f|~WwZD24Fk4ka7ZY47MPopoo||||||2||

The time stamp is the entry that shows 1500090332.

The 1500090332 is the utc epoch and a converter can be found at this site https://www.epochconverter.com/ so you can see the human readable version of the time.