Google does browser sniffing, 403 on no User-Agent #44

andreineculau · 2015-03-31T09:16:50Z

Without a UA header:

$ curl -v "http://translate.google.com/translate_a/t?client=t&ie=UTF-8&oe=UTF-8&text=before&sl=en&tl=sv&hl=sv"                                                                                                                                                = 11:14:59 Tue 31
* Hostname was NOT found in DNS cache
*   Trying 216.58.209.110...
* Connected to translate.google.com (216.58.209.110) port 80 (#0)
> GET /translate_a/t?client=t&ie=UTF-8&oe=UTF-8&text=before&sl=en&tl=sv&hl=sv HTTP/1.1
> User-Agent: curl/7.37.1
> Host: translate.google.com
> Accept: */*
> Referer:
>
< HTTP/1.1 403 Forbidden
< X-Content-Type-Options: nosniff
< Date: Tue, 31 Mar 2015 09:15:43 GMT
* Server HTTP server (unknown) is not blacklisted
< Server: HTTP server (unknown)
< Cache-Control: private
< X-XSS-Protection: 1; mode=block
< Set-Cookie: PREF=ID=363e6de56a4b1b07:TM=1427793343:LM=1427793343:S=q7Yaoo05hm79JH_k; expires=Thu, 30-Mar-2017 09:15:43 GMT; path=/; domain=.google.com
< Alternate-Protocol: 80:quic,p=0.5
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< Via: 1.1 aer01-mda2-dmz-wsa-4-mgmt.cisco.com:80 (Cisco-WSA/8.5.0-497)
< Connection: keep-alive
<
<!DOCTYPE html><html lang=en><meta charset=utf-8><meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width"><title>Error 403 (Forbidden)!!1</title><style>*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/errors/logo_sm_2.png) no-repeat}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/errors/logo_sm_2_hr.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:55px;width:150px}</style><a href=//www.google.com/><span id=logo aria-label=Google></span></a><p><b>403.</b> <ins>That’s an error.</ins><p>Your client does not have permission to get URL <code>/translate_a/t?client=t&amp;ie=UTF-8&amp;oe=UTF-8&amp;text=before&amp;sl=en&amp;tl=sv&amp;hl=sv</code> from this server.  (Client IP address: 173.38.209.9)<br><br>


* Connection #0 to host translate.google.com left intact
  <ins>That’s all we know.</ins>

With a UA header:

  /Users/aneculau/bin as aneculau@ANECULAU-M-H01K
  master *$%> a38c2b3 8↑ origin/master
$ curl -v -A "tranlate-shell/0.8.22.5" "http://translate.google.com/translate_a/t?client=t&ie=UTF-8&oe=UTF-8&text=before&sl=en&tl=sv&hl=sv"                                                                                                                   = 11:15:44 Tue 31
* Hostname was NOT found in DNS cache
*   Trying 216.58.209.110...
* Connected to translate.google.com (216.58.209.110) port 80 (#0)
> GET /translate_a/t?client=t&ie=UTF-8&oe=UTF-8&text=before&sl=en&tl=sv&hl=sv HTTP/1.1
> User-Agent: tranlate-shell/0.8.22.5
> Host: translate.google.com
> Accept: */*
> Referer:
>
< HTTP/1.1 200 OK
< Pragma: no-cache
< Date: Tue, 31 Mar 2015 09:15:50 GMT
< Expires: Tue, 31 Mar 2015 09:15:50 GMT
< Cache-Control: private, max-age=600
< Content-Language: sv
< X-Content-Type-Options: nosniff
< Content-Disposition: attachment; filename="f.txt"
* Server HTTP server (unknown) is not blacklisted
< Server: HTTP server (unknown)
< X-XSS-Protection: 1; mode=block
< Set-Cookie: PREF=ID=8cc78e93d473970a:TM=1427793350:LM=1427793350:S=3qV6SmN9yFLCLdVV; expires=Thu, 30-Mar-2017 09:15:50 GMT; path=/; domain=.google.com
< Alternate-Protocol: 80:quic,p=0.5
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/javascript; charset=UTF-8
< Via: 1.1 aer01-mda2-dmz-wsa-4-mgmt.cisco.com:80 (Cisco-WSA/8.5.0-497)
< Connection: keep-alive
<
* Connection #0 to host translate.google.com left intact
[[["före","before","",""]],[["adverb",["före","innan","förut","förrän","förr"],[["före",["before","prior to","preparatory to","in front","ahead"],,0.44374731],["innan",["before"],,0.362176],["förut",["before","previously","heretofore","formerly","before then","pre-term"]],["förrän",["before"]],["förr",["before","sooner","formerly","once","aforetime","yore"]]],"before",4],["konjunktion",["före","innan","förr","förut","förrän","framför"],[["före",["before"],,0.44374731],["innan",["before"],,0.362176],["förr",["before"]],["förut",["before"]],["förrän",["before","than"]],["framför",["before"]]],"before",7],["preposition",["före","innan","inför","framför","framom","för"],[["före",["before","previous to","ahead of","in front of"],,0.44374731],["innan",["before","prior to"],,0.362176],["inför",["before","in front of","at","fore"],,0.03758125],["framför",["in front of","above","before","ahead of","rather than"]],["framom",["before","ahead of"]],["för",["for","of","To","before","from","about"]]],"before",5]],"en",,[["före",[1],true,false,649,0,1,0]],[["before",1,[["före",649,true,false],["innan",350,true,false],["innan du",0,true,false],["tidigare",0,true,false],["inför",0,true,false]],[[0,6]],"before"]],,,[],3]
`
``

The text was updated successfully, but these errors were encountered:

soimort · 2015-03-31T09:28:26Z

Is this reproducible on translate-shell?

andreineculau · 2015-03-31T10:57:58Z

Yes, I get the "Oops" failure message now.

On Tue, Mar 31, 2015 at 11:28 AM, Mort Yao notifications@github.com wrote:

Is this reproducible on translate-shell?

—
Reply to this email directly or view it on GitHub
#44 (comment)
.

andreineculau.com http://www.andreineculau.com

soimort · 2015-04-02T03:49:28Z

curl is blocked for sure, but simply dropping the UA works fine for me:

$ curl -v -A "" "http://translate.google.com/translate_a/t?client=t&ie=UTF-8&oe=UTF-8&text=before&sl=en&tl=sv&hl=sv"
* About to connect() to translate.google.com port 80 (#0)
*   Trying 224.0.0.1...
* connected
* Connected to translate.google.com (224.0.0.1) port 80 (#0)
> GET /translate_a/t?client=t&ie=UTF-8&oe=UTF-8&text=before&sl=en&tl=sv&hl=sv HTTP/1.1
> Host: translate.google.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Pragma: no-cache
< Date: Thu, 02 Apr 2015 03:39:40 GMT
< Expires: Thu, 02 Apr 2015 03:39:40 GMT
< Cache-Control: private, max-age=600
< Content-Type: text/javascript; charset=UTF-8
< Content-Language: sv
< X-Content-Type-Options: nosniff
< Content-Disposition: attachment; filename="f.txt"
< Server: HTTP server (unknown)
< X-XSS-Protection: 1; mode=block
< Set-Cookie: PREF=ID=a3d6db3eba2fb01f:TM=1427945980:LM=1427945980:S=YhQJS68EZqhg-F_x; expires=Sat, 01-Apr-2017 03:39:40 GMT; path=/; domain=.google.com
< Alternate-Protocol: 80:quic,p=0.5
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< 
[[["före","before","",""]],[["adverb",["före","innan","förut","förrän","förr"],[["före",["before","prior to","preparatory to","in front","ahead"],,0.44374731],["innan",["before"],,0.362176],["förut",["before","previously","heretofore","formerly","before then","pre-term"]],["förrän",["before"]],["förr",["before","sooner","formerly","once","aforetime","yore"]]],"before",4],["konjunktion",["före","innan","förr","förut","förrän","framför"],[["före",["before"],,0.44374731],["innan",["before"],,0.362176],["förr",["before"]],["förut",["before"]],["förrän",["before","than"]],["framför",["before"]]],"before",7],["preposition",["före","innan","inför","framför","framom","för"],[["före",["before","previous to","ahead of","in front of"],,0.44374731],["innan",["before","prior to"],,0.362176],["inför",["before","in front of","at","fore"],,0.03758125],["framför",["in front of","above","before","ahead of","rather than"]],["framom",["before","ahead of"]],["för",["for","of","To","before","from","a* Connection #0 to host translate.google.com left intact
bout"]]],"before",5]],"en",,[["före",[1],true,false,649,0,1,0]],[["before",1,[["före",649,true,false],["innan",350,true,false],["innan du",0,true,false],["tidigare",0,true,false],["inför",0,true,false]],[[0,6]],"before"]],,,[],3]* Closing connection #0

It would be helpful if other users who encountered the same issue report details here.

Fil · 2015-04-11T22:15:47Z

is this why trans stopped working for me (on two very different machines)?

soimort · 2015-04-12T00:59:58Z

@Fil Perhaps not. Have you updated to the latest version?

Fil · 2015-04-12T09:35:16Z

you're right — my brew had 0.8.22.5 and it didn't know how to upgrade; removing and reinstalling from your formula gets me the (functional) 0.8.23

soimort · 2015-05-22T02:09:25Z

New option -user-agent provided for users to customize their own UA. Closing this.

andreineculau mentioned this issue Apr 14, 2015

empty output #50

Closed

soimort added the CNR cannot reproduce label Apr 28, 2015

soimort added this to the 0.9 milestone Apr 30, 2015

soimort mentioned this issue Apr 30, 2015

[CHANGELOG] 0.9.0 #54

Merged

23 tasks

soimort closed this as completed in #54 May 22, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Google does browser sniffing, 403 on no User-Agent #44

Google does browser sniffing, 403 on no User-Agent #44

andreineculau commented Mar 31, 2015

soimort commented Mar 31, 2015

andreineculau commented Mar 31, 2015

soimort commented Apr 2, 2015

Fil commented Apr 11, 2015

soimort commented Apr 12, 2015

Fil commented Apr 12, 2015

soimort commented May 22, 2015

Google does browser sniffing, 403 on no User-Agent #44

Google does browser sniffing, 403 on no User-Agent #44

Comments

andreineculau commented Mar 31, 2015

soimort commented Mar 31, 2015

andreineculau commented Mar 31, 2015

soimort commented Apr 2, 2015

Fil commented Apr 11, 2015

soimort commented Apr 12, 2015

Fil commented Apr 12, 2015

soimort commented May 22, 2015