You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
any change to get a feature where we can download a site from a range of dates? for example 2015-Today to try and get every copy of a URL, but only save the most successful download?
the use case is im trying to get a website, but some pages are "blocked by cloudflare" on certain versions of archive.org
thanks!
The text was updated successfully, but these errors were encountered:
devinschumacher
changed the title
able to download a website historically while only saving the 1st successful page?
question: able to download a website historically while only saving the 1st successful page?
Nov 26, 2023
I don't think waybackpack currently supports this, but would be open to a PR that adds it. One tricky bit might be defining a criteria for "successful", particularly if the HTTP status code does not make it clear.
I don't think waybackpack currently supports this, but would be open to a PR that adds it. One tricky bit might be defining a criteria for "successful", particularly if the HTTP status code does not make it clear.
yeah i was thinking that same thing about the criteria.
it would probably be a series of words/patterns that would get added to over time until it was reasonably comprehensive? might be some stuff in the the HTML tags as well i bet the meta title and description on pages like that would always give it away
what i normally see are things like Cloudflare, Login, Too Many Requests etc.
any change to get a feature where we can download a site from a range of dates? for example 2015-Today to try and get every copy of a URL, but only save the most successful download?
the use case is im trying to get a website, but some pages are "blocked by cloudflare" on certain versions of archive.org
thanks!
The text was updated successfully, but these errors were encountered: