[AUD-430] Remove identity endpoint to serve registered content nodes

[dmanjunath]

Description

This PR removes the identity route + fallback logic in the discovery index_network_peers.py task to peer with all the IPFS nodes within each content node.

Tests

Ran the system locally, and observed that the peering logic worked successfully.

Discovery celery logs

celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Received task: update_network_peers[3b8c611c-e0a6-412f-b615-85b5a8a5c0c4]  ", "timestamp": "2021-03-17 19:56:57,269"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Task update_discovery_provider[95e02893-4035-4166-8ac5-c6f61b4a30de] succeeded in 0.041606896003941074s: None", "timestamp": "2021-03-17 19:56:57,318", "data": {"id": "95e02893-4035-4166-8ac5-c6f61b4a30de", "name": "update_discovery_provider", "return_value": "None", "runtime": 0.041606896003941074}}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | DEBUG | number of total_cn_type_providers 3", "timestamp": "2021-03-17 19:56:57,482"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Redis Cache - hit sp:id:1", "timestamp": "2021-03-17 19:56:57,488"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Found cached value for spID=1 - ['0x17362305aeC6F706Fe033908AEfc94f023e16E11', 'http://cn1_creator-node_1:4000', 163, '0x17362305aeC6F706Fe033908AEfc94f023e16E11']", "timestamp": "2021-03-17 19:56:57,488"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Redis Cache - hit sp:id:2", "timestamp": "2021-03-17 19:56:57,489"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Found cached value for spID=2 - ['0x0baae229c597CB0E0218813ae253781034aa7245', 'http://cn2_creator-node_1:4001', 165, '0x0baae229c597CB0E0218813ae253781034aa7245']", "timestamp": "2021-03-17 19:56:57,489"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Redis Cache - hit sp:id:3", "timestamp": "2021-03-17 19:56:57,489"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Found cached value for spID=3 - ['0x5CB40AcFfBb5E631A86AC4823704a19D6eb4c966', 'http://cn3_creator-node_1:4002', 167, '0x5CB40AcFfBb5E631A86AC4823704a19D6eb4c966']", "timestamp": "2021-03-17 19:56:57,489"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | DEBUG | eth_sp_endpoint http://cn1_creator-node_1:4000", "timestamp": "2021-03-17 19:56:57,494"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | DEBUG | eth_sp_endpoint http://cn3_creator-node_1:4002", "timestamp": "2021-03-17 19:56:57,494"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | DEBUG | eth_sp_endpoint http://cn2_creator-node_1:4001", "timestamp": "2021-03-17 19:56:57,494"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Peers from eth-contracts: {'http://cn2_creator-node_1:4001', 'http://cn1_creator-node_1:4000', 'http://cn3_creator-node_1:4002'}", "timestamp": "2021-03-17 19:56:57,495"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index.py | update_user_replica_set_manager_address_if_necessary, found 0x0000000000000000000000000000000000000000", "timestamp": "2021-03-17 19:56:57,530"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Peers from db : set()", "timestamp": "2021-03-17 19:56:57,535"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | All known peers {'http://cn2_creator-node_1:4001', 'http://cn1_creator-node_1:4000', 'http://cn3_creator-node_1:4002'}", "timestamp": "2021-03-17 19:56:57,537"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Peering with http://cn2_creator-node_1:4001", "timestamp": "2021-03-17 19:56:57,539"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index.py | get_latest_block | current=31 target=31", "timestamp": "2021-03-17 19:56:57,551"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Peering with http://cn3_creator-node_1:4002", "timestamp": "2021-03-17 19:56:57,563"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Peering with http://cn1_creator-node_1:4000", "timestamp": "2021-03-17 19:56:57,543"}
celery-worker_1          | {"levelno": 30, "level": "WARNING", "msg": "returning /ip4/172.18.0.4/tcp/4001/p2p/12D3KooWByR6NJxMzf8WgVZ4FYhwkP394WYfn5Uw6XzBsfXZ33Rt", "timestamp": "2021-03-17 19:56:57,592"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index.py | update_task | Intersect_blockhash : 0xb945690e8c8c5a91c3b72284e4b9e26413335f416ce715e75af9504e8b3e683f", "timestamp": "2021-03-17 19:56:57,601"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index.py | update_task | f3315ca3-1a43-4efe-929f-36a61c1db561 | Processing complete within session", "timestamp": "2021-03-17 19:56:57,602"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "<ipfshttpclient.client.base.ResponseBase: {'Strings': ['connect 12D3KooWByR6NJxMzf8WgVZ4FYhwkP394WYfn5Uw6XzBsfXZ33Rt success']}>", "timestamp": "2021-03-17 19:56:57,605"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Successfully peered with http://cn2_creator-node_1:4001", "timestamp": "2021-03-17 19:56:57,606"}
celery-worker_1          | {"levelno": 30, "level": "WARNING", "msg": "returning /ip4/172.18.0.4/tcp/4001/p2p/12D3KooWByR6NJxMzf8WgVZ4FYhwkP394WYfn5Uw6XzBsfXZ33Rt", "timestamp": "2021-03-17 19:56:57,603"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Task update_discovery_provider[f3315ca3-1a43-4efe-929f-36a61c1db561] succeeded in 0.3461497270036489s: None", "timestamp": "2021-03-17 19:56:57,607", "data": {"id": "f3315ca3-1a43-4efe-929f-36a61c1db561", "name": "update_discovery_provider", "return_value": "None", "runtime": 0.3461497270036489}}
celery-worker_1          | {"levelno": 30, "level": "WARNING", "msg": "returning /ip4/172.18.0.4/tcp/4001/p2p/12D3KooWByR6NJxMzf8WgVZ4FYhwkP394WYfn5Uw6XzBsfXZ33Rt", "timestamp": "2021-03-17 19:56:57,617"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "<ipfshttpclient.client.base.ResponseBase: {'Strings': ['connect 12D3KooWByR6NJxMzf8WgVZ4FYhwkP394WYfn5Uw6XzBsfXZ33Rt success']}>", "timestamp": "2021-03-17 19:56:57,622"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Successfully peered with http://cn1_creator-node_1:4000", "timestamp": "2021-03-17 19:56:57,623"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "<ipfshttpclient.client.base.ResponseBase: {'Strings': ['connect 12D3KooWByR6NJxMzf8WgVZ4FYhwkP394WYfn5Uw6XzBsfXZ33Rt success']}>", "timestamp": "2021-03-17 19:56:57,636"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "index_network_peers.py | Successfully peered with http://cn3_creator-node_1:4002", "timestamp": "2021-03-17 19:56:57,636"}
celery-worker_1          | {"levelno": 20, "level": "INFO", "msg": "Task update_network_peers[3b8c611c-e0a6-412f-b615-85b5a8a5c0c4] succeeded in 0.3162562199868262s: None", "timestamp": "2021-03-17 19:56:57,639", "data": {"id": "3b8c611c-e0a6-412f-b615-85b5a8a5c0c4", "name": "update_network_peers", "return_value": "None", "runtime": 0.3162562199868262}}

[raymondjacobson]

Why do we need this endpoint? Discovery should be able to hit chain for this info, and could also do the same caching logic

[dmanjunath]

Resolution is to delete this on identity and discovery

[raymondjacobson]

looks good!
note to self -- if we release identity first, discprov will start spamming logs about not being able to fetch, but nothing should break

[hareeshnagaraj]

Looks great, thanks for handling this!