Shard distribution is still using responseTime

[littleskunk]

Package Versions

Replace the values below using the output from npm list storj-bridge.

"version":"7.2.2"
"x-protocol-version":"1.2.0"
"x-core-version":"8.3.1"

Replace the values below using the output from node --version.

v6.11.5

Test Setup

As renter I try to upload a shard. The bridge will send out a group of ALLOC requests and give me one farmer for my shard upload.

I will not upload and ask the bridge again for the same shard hash. The bridge will use the queued ALLOC responses.

Repeat that to get a list of farmer that are qualified for the shard upload.

Expected Behavior

Please describe the program's expected behavior. Include an example of your
usage code in the back ticks below if applicable.

I would expect to see random farmer on the list. The ALLOC request is working correct. A few farmer with nodeIDs close to the generated random

I would expect to see random farmer on the list.
The ALLOC request is working correct. I can see a group of ~20 low responseTime farmer plus a few high responseTime farmer.

Actual Behavior

Please describe the program's actual behavior. Please include any stack traces
or log output in the back ticks below.

The bridge is still using the fastest response for the shard upload. The farmer list is ordered by responseTime.

Shard hash | farmer nodeID | farmer responseTime (from bride contact list)

{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d6139f6a90611b57e788003689d2cd97ff9a39ea | 574,566808227
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d644c827a006ede029cf1de0fb0e4d6aa10bae95 | 686,308401186
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d63b18a3515935129ab4e03301541fe518ab7666 | 793,959601653
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d642c74fca02b49463dd709ce7ad75783028ab55 | 796,186922938
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d66af2ea8c2a8d986a85d1c539ba490979f0654b | 877,257877886
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d6037aff91c777ac22396888349ded79f66db3dc | 905,484050479
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d6520087b1b357667184df08b67aadfcf1283c3c | 922,510174959
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d618299da1d082c1894ab3a526b4d334b1825a33 | 978,633517695
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d62d634802a48a8a6be57ffe7e7715df4982672f | 1061,79420809
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d6720672b536c9a593f8c56c3c1d0841e7e06b59 | 1072,28169209
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d5f1035dd3c171a668d8c5f642885cbd27244216 | 1273,05088715
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d65eb1901cf062b5843cb2926598962155a3dc1c | 2073,53911353
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d6980a2d19e03eee7530899d02b30c42049450f3 | 3885,99929707
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | f55a854f73c642b9670ffa0d65c8da5404aefd77 | 5814,44442847
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | f56930219825e9469b4866e6155fc6a72ef7ee67 | 6809,07802175
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | f5540d433be977bf4071e0f5004dc8d42308247a | 10557,2785414
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | f55a5783fcbfb1bdb30bd4270a2b58f2b3ca9e8a | 28281,7766281
[...]

{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 6767ed8a4dc3910d28bdfe942f7406928ff87a23 | 723,342857806
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 6782bc5089c8a04729fabe525fd5321315ba33ce | 913,041345344
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67460a044878ab900e8e971e541dc40d95bdf157 | 1006,68289961
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67439a6a586daf07e4fc2c2a9cd987c28824c88b | 1050,33080192
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67986487c1c88e3629e0d6af8bf9883019657a7c | 1078,15046381
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 6763b40eb87d34815182ba9ad56d62eba413fdd0 | 1172,22662475
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 671dcac32111a5dea3b76951dc11e434306d8900 | 1139,83266631
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67aa2d690a88f8ed97b95f7913eb2f54cdcdf0eb | 1238,3554508
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67ad0caab538025b582aff5f81dc163af609929c | 1280,52852672
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67402ef2ff0571ceb7bd849300c75747d1137a95 | 1299,75805259
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 676863843fdbd42169af1db13c2d58c56932bb67 | 1529,11213986
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 679498b872445feddd74519e4270f76e96446a43 | 1631,05202229
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 674573ab5aabc2959e735fb0ffe685b2c8a72bf5 | 1663,82308676
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67b9a530cb7ea4944b4975feb7a0aba0d03f1217 | 1774,11363815
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 673c09d868d4c725b8495b2b88dee0f3fcc63b4a | 1746,96366977
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | 67c2dbd7ea608c3d79907c5d307bc2955a25ecd0 | 3301,23400633
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d69c603d493f849d0b09b6587a2ba5557d46183f | 11076,6169027
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | d699137df1b92062dbc5e1238419e8e53b020b48 | 31491,6992256
[...]
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a935afe67642825b63e55a644a988c50538368c8 | 828,996005638
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a911112417ca0cf0086316912fdcf679be23e1df | 663,558334783
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8e3f61545f78a2f7ab9a90fa6fd1dcd3adc9be6 | 814,35294945
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a935afe67642825b63e55a644a988c50538368c8 | 828,996005638
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a956739b148cf9ab17e3efc6a5e6872294f9977b | 900,493907191
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8f148218874adf46dea9497bd2dad199a1c2529 | 959,748137754
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8f8e7eeed30aa510c3caa692d73e7bb663ee1d6 | 1048,18681799
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8fc6b5e9dbe57a62c80d8244f125f3aaf19ea7e | 1091,36395808
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a916c40887f1edd0a049b52f55d08a27b966519d | 1144,19707989
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8b7fc607ead54fc28f70614c678bbfb756c6967 | 1157,06543471
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8c9076c98307ad77244fc35c3e0f7bb09ce3807 | 1192,47658834
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a952f087aa74640bee4ea6a93f107449cfe2783b | 1222,35957853
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a937017585d70a2c22d5ee60d1126859adb392df | 1282,97783214
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8ae4e1945c32948951bf420e67afcb95cc1caf1 | 1319,31303792
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a914c51b309a5ddc942da189f2e0bf1227f017f4 | 1277,47940146
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a919e4397af85b93163537398f870c9dbcc14809 | 1389,925937
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8eb877cb181dcb9281991ea101c89dda39f5ba4 | 1392,10514402
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a8b607ecfa69dc019dcd5cd2eb6e6606f94dd0c5 | 3008,13559824
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a93744a9b7e35209119f7771ff8815d4f4717dfd | 3508,63836465
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | a910610c0f79e9792cb8133d17be5cd9470fc950 | 3527,28544834
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | cd4d48fb75c42b47e4234ddcb705d5c59e098891 | 6660,41399287
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | cd501f25793df6f60edb8ef5cd7c4f6caa6deba9 | 8354,30326035
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | cd51548baf5ff7ebaae8ae428eb1a9f3fa5077c4 | 8910,17340487
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | cd63540d45848a5ae509e8c9b82436cc038efda3 | 9445,21485393
{ "hash": "e4c6fb6e369c22925d2904dd5c0a316c817f3af9" | cd7ef8b9de39b388182f810d7467165522941a45 | 23911,6609087

Steps to Reproduce

Please include the steps the reproduce the issue, numbered below. Include as
much detail as possible.

1. Request a farmer for the same shard hash a few times in a row without uploading the shard

Complete list

Daten.csv.txt

[littleskunk]

As far as I can see this methode should be working correct and select a random farmer for the shard upload: https://github.com/Storj/complex/blob/master/lib/renter.js#L592

Stupid question. The bridge is not calling it right? It looks like the bridge is bypassing the renter function and is calling the network function with the same name: https://github.com/Storj/bridge/blob/3134e47d0fd1ffed3591498c0af41a73faa2656d/lib/server/routes/frames.js#L195

Could it be that simple of am I wrong?

[braydonf]

publishContract on network is a complex client that talks to landlord which adds work to rabbitmq and a renter will then call _publishContract

[littleskunk]

Well than I have no Idea why it is not working. The list on the renter side is ordered by responseTime and that should not happen.

[braydonf]

Okay, the "repeated" with the same shard hash is what makes this different.

[braydonf]

In that case it pulls from an existing queue of offers, which is sorted by response time: https://github.com/Storj/bridge/blob/master/lib/server/routes/frames.js#L325