Work-around for HTTP GET limits #10

Merged
merged 1 commit into from Apr 1, 2012

Projects

None yet

3 participants

@grierj

This is a work-around for the size limit on HTTP GETs that range uses.
Very long requests fail and thus there's a certain type of query that
can't be made. This chane updates the library to automatically split
and make multiple requests when the size is larger than 7500 characters.
Generally apache is configured for 8190 characters, but with the fqdn
plus other bits of the header, this is a safer limit.

Let me know if it's too hacky... :)

@grierj grierj HTTP GET URL size limit workaround
THis is a work-around for the size limit on HTTP GETs that range uses.
Very long requests fail and thus there's a certain type of query that
can't be made.  This chane updates the library to automatically split
and make multiple requests when the size is larger than 7500 characters.
Generally apache is configured for 8190 characters, but with the fqdn
plus other bits of the header, this is a safer limit.
7b2f33a
@cuzelac
ytoolshed member

I agree that it's gross but until we use something other than GET, it's necessary.

Is there a way to future-proof this by only enabling it for GET requests?

@grierj

The fix is only in the python library at this point, which just uses urllib2 to do a GET request. We'd have to change that part anyways if a different request method ever came into the picture. The extra requests could be removed at that point, or reserved for legacy GET requests at that time.

@erikwb erikwb merged commit ef1d7e9 into ytoolshed:master Apr 1, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment