Tumblr rate limits by IP, consider partnering with VPNs to adjust IP periodically

[timkatje]

So, maybe this is a symptom of having really great Internet (Google Fiber 1G), so I can download a lot of content from Tumblr. But I noticed after just a single night of contributing to your project, that there was basically no traffic running through the VM. Later, when trying to just access my own personal blog, I was hit with a Access Denied error message that vaguely mentioned rate limits. If I activate my own VPN service, then I'm able to get past Tumblr's block.

However, your install instructions have us bridge our VMs (VMware) to our physical NICs, so it's more difficult for me to force your traffic to use my VPN, unless I do something at the router level.

As such, for big projects with very short timelines like this one, you may wish to include VPN options inside the VM... either just support many of the most popular ones and give us a way to configure our settings to connect using our accounts. Or, better yet, find a way to partner with a sympathetic VPN company and just do it automatically.

Unfortunately, it's probably too late in the case of Tumblr, but we might run into a similar situation in the future.

[Ganonmaster]

I don't think the use of VPNs is something to be encouraged, as you will likely end up sharing the endpoint with other users. You might actually end up getting the IP address of the VPN endpoint rate limited; potentially making the VPN endpoint unusable for other people using it for legitimate browsing.

You want more, different IP addresses. I'm running the docker image on a set of VPSes, all with different IPs, which is likely a better solution than a VPN.

[chfoo]

Linking a related discussion so it doesn't get lost: ArchiveTeam/warrior-dockerfile#27

[TomGlass]

As discussed at link above, we're heavily against VPNs