Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Go through hosts in parallel instead of 1 host at a time #5

Open
dannyhajj opened this issue Feb 1, 2019 · 0 comments
Open

Go through hosts in parallel instead of 1 host at a time #5

dannyhajj opened this issue Feb 1, 2019 · 0 comments
Assignees
@sys0dm1n
Labels
enhancement question

Comments

@dannyhajj
Copy link
Collaborator

I think it might be a good idea to warm the cache by URL instead of by host. Maybe add that as on option after implementing #4.

Currently, that's how it works:

  1. Loop hosts
  2. For each host, load sitemap
  3. Load all URLs in that sitemap for one host.
  4. Repeat for next host

As a command line argument, allow the following instead?

  1. Fetch sitemap on the first host
  2. Loop URLs
  3. For each URL, load it on each host
  4. Repeat for next URL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sys0dm1n sys0dm1n

Labels
enhancement question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dannyhajj @sys0dm1n