This repository has been archived by the owner. It is now read-only.

Update hostdb scan quanity/sleep time #2081

Closed
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@S-anasol
Contributor

S-anasol commented Jul 5, 2017

Current default settings cant handle hostdb grow.
So hostdb have a lot outdated data.

Adjusted settings for siahub wallet.
Enough to keep fresh db for now.

After update we have big fall since many hosts not scanned fast/properly due big hostdb.
https://siahub.info/network

Update scan quanity/sleep time
Current default settings cant handle hostdb grow.
So hostdb have a lot outdated data.

Adjusted settings for siahub wallet.
Enough to keep fresh db for now.

After update big fall since many hosts not scanned fast/properly due big hostdb.
https://siahub.info/network

@S-anasol S-anasol changed the title from Update scan quanity/sleep time to Update hostdb scan quanity/sleep time Jul 5, 2017

@S-anasol

This comment has been minimized.

Show comment
Hide comment
@S-anasol

S-anasol Jul 7, 2017

Contributor

Actually problem is that scan doesnt cover DB size and every scan get only top N hosts.

// Set up a scan for the hostCheckupQuanity most valuable hosts in the

So in this moment we have some good hosts that actually online and work, but not scanned due quanity limit.

Contributor

S-anasol commented Jul 7, 2017

Actually problem is that scan doesnt cover DB size and every scan get only top N hosts.

// Set up a scan for the hostCheckupQuanity most valuable hosts in the

So in this moment we have some good hosts that actually online and work, but not scanned due quanity limit.

@@ -39,15 +39,15 @@ var (
// hostCheckupQuantity specifies the number of hosts that get scanned every
// time there is a regular scanning operation.
hostCheckupQuantity = build.Select(build.Var{
Standard: int(200),
Standard: int(500),

This comment has been minimized.

@S-anasol

S-anasol Jul 12, 2017

Contributor

actually outdated limit, it will not cover all online hosts already

@S-anasol

S-anasol Jul 12, 2017

Contributor

actually outdated limit, it will not cover all online hosts already

@Cinerar

This comment has been minimized.

Show comment
Hide comment
@Cinerar

Cinerar Jul 18, 2017

I believe this is not the best way to go. (as a temporary fix i think this is ok)
As long as i understand this aproach can leave best host (the one that changed settings recently) with outdated info and almoust never scan it again.

Limiting scanned hosts centralize the network, as long as only best hosts will be regulary rescanned.
I believe that Score of the host can be tied to period of rescan. This way even not so good hosts will be rescanned sooner or later and there will be not so much network overload.

Cinerar commented Jul 18, 2017

I believe this is not the best way to go. (as a temporary fix i think this is ok)
As long as i understand this aproach can leave best host (the one that changed settings recently) with outdated info and almoust never scan it again.

Limiting scanned hosts centralize the network, as long as only best hosts will be regulary rescanned.
I believe that Score of the host can be tied to period of rescan. This way even not so good hosts will be rescanned sooner or later and there will be not so much network overload.

@DavidVorick

This comment has been minimized.

Show comment
Hide comment
@DavidVorick

DavidVorick Aug 2, 2017

Member

I'm going to go ahead and close this as a pull request, I think a full discussion is needed when searching for better options. At 1000 hosts or so, I think we're above what we should be scanning every block. I like the 200 hosts limit per 8 hours I think, we probably just need a more intelligent way to scan them.

The most important thing is that we're aware of all the hosts that might be interesting to us.

Member

DavidVorick commented Aug 2, 2017

I'm going to go ahead and close this as a pull request, I think a full discussion is needed when searching for better options. At 1000 hosts or so, I think we're above what we should be scanning every block. I like the 200 hosts limit per 8 hours I think, we probably just need a more intelligent way to scan them.

The most important thing is that we're aware of all the hosts that might be interesting to us.

@DavidVorick DavidVorick closed this Aug 2, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.