Skip to content
This repository has been archived by the owner on Oct 30, 2018. It is now read-only.

queueing publishContract and ping on worker queue close to nodeID #91

Merged
merged 2 commits into from
Jan 24, 2018

Conversation

littleskunk
Copy link
Contributor

@littleskunk littleskunk commented Jan 17, 2018

I would expect alloc and ping message comming from renterIDs close to my nodeID. At the moment all renterIDs try to comunicate with me. In return I as farmer will have to spam the bridge with PUBLISH messages because sooner or later my contact list is full with to many renterIDs.

This pull request will fix it.

Mirror has this problem as well but I don't want to change to much at the same time. Mirror will use the shard hash to queue the job. That was fine the early days because the shard hash was close to all mirror farmer. We removed that relationship and would have to use the first destination now. However I would need a test environment to try this. I don't want to risk it on the main net. Thats why I havent included it in this pull request.

I noticed that contractRenewal has duplicate code blocks. It has a serializer and desiralizer but is bypassing them by sending a RAW message itself. If we are going to keep complex for a few more days I could try to clean up the unused code. Would still look ugly but not as confusing as the current situation.

@littleskunk littleskunk changed the title [WIP] improve unit tests queueing publishContract and ping on worker queue close to nodeID Jan 17, 2018
@littleskunk
Copy link
Contributor Author

And I added nodejs 8 to the unit test matrix. It will still run nodejs 6. That way we can update the bridge related repos one by one. The unit tests will run on both versions.

@braydonf
Copy link
Contributor

I tried testing this and get this error message when trying to upload:

{"message": "fn[push_frame] - JSON Response: { "error": "First argument must be a string, Buffer, ArrayBuffer, Array, or array-like object." }", "level": 4, "timestamp": 1516322790875}`

@braydonf
Copy link
Contributor

braydonf commented Jan 19, 2018

Data structure for publishContract is like this:

[
  [ 
    { 
      _id: '4bb49fb779e9233f9ed14f6ef34e29048911f018',
      spaceAvailable: true,
      lastSeen: '2018-01-19T00:56:34.578Z',
      port: 9003,
      address: '127.0.0.1',
      protocol: '1.2.0',
      responseTime: 10000,
      userAgent: '8.5.0' 
    }, { 
      _id: '3274ad1931da1d04d720601bd672c78a0f9bda9d',
      spaceAvailable: true,
      lastSeen: '2018-01-19T00:56:34.580Z',
      port: 9013,
      address: '127.0.0.1',
      protocol: '1.2.0',
      responseTime: 10000,
      userAgent: '8.5.0' 
    }
  ]
]

So one renter will send to many farmers, having a more similar nodeID shouldn't have a benefit.

@littleskunk
Copy link
Contributor Author

Oh. I was so blind. That explains a lot.

publishContract should now work and thanks to your comment I was able to understand how getMirror will look like. It will now select the queue based on the first destination point.

Please test it again and let me know if something is still broken.

I am aware that publishContract and getMirror are comming with a farmer list. Your example is showing that the farmer on the list are close to each other. The worker queues are overlapping. Not perfect but it should work.

@braydonf
Copy link
Contributor

braydonf commented Jan 24, 2018

Okay, pulled the latest changes at b6645b7 and the error ""First argument..." is gone now, and uploads are working.

@braydonf braydonf merged commit 5873328 into storj-archived:master Jan 24, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants