New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Language Detection Not Working (HTTP Error 503: Service Unavailable) #137

Closed
manurajhada opened this Issue Sep 1, 2016 · 20 comments

Comments

Projects
None yet
@manurajhada

manurajhada commented Sep 1, 2016

from textblob import TextBlob
txt = u"Test Language Detection"
b = TextBlob(txt)
b.detect_language()

It is giving "HTTPError: HTTP Error 503: Service Unavailable"

Python Version: 2.7.6
TextBlob Version: 0.11.1
OS: Ubuntu 14.04 LTS & CentOS 6.8

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Sep 1, 2016

from textblob import TextBlob
text = '''Please Miss Fried, He don't mean to be mean , He just needs to leave, He hopes he does not '''
blob = TextBlob(text)
blob.translate(to="es")

It shows:
HTTPError: Service Unavailable

ghost commented Sep 1, 2016

from textblob import TextBlob
text = '''Please Miss Fried, He don't mean to be mean , He just needs to leave, He hopes he does not '''
blob = TextBlob(text)
blob.translate(to="es")

It shows:
HTTPError: Service Unavailable

@Sickness-chan

This comment has been minimized.

Show comment
Hide comment
@Sickness-chan

Sickness-chan Sep 1, 2016

Same issue on Python 3 with a simple translate, it always throws error 503, since approximately 2 days. It has worked just fine before.

import discord
from discord.ext import commands
from .utils.chat_formatting import *
from .utils import checks
from textblob import TextBlob
import re

class Translate:
    """Translates text to desired language."""

    def __init__(self, bot):
        self.bot = bot

    @commands.command()
    async def translate(self, *, text : str):
        """Translates text to desired language."""
        pattern = re.compile("\-([a-z]{1,3})")
        m = pattern.match(text)

        if m:
            blob = TextBlob(text.split(' ', 1)[1])
            await self.bot.say("```" + str(blob.translate(to=str(m.group()[1:]))) + "```")
        else:
            blob = TextBlob(text)
            await self.bot.say("```" + str(blob.translate(to="en")) + "```")

Sickness-chan commented Sep 1, 2016

Same issue on Python 3 with a simple translate, it always throws error 503, since approximately 2 days. It has worked just fine before.

import discord
from discord.ext import commands
from .utils.chat_formatting import *
from .utils import checks
from textblob import TextBlob
import re

class Translate:
    """Translates text to desired language."""

    def __init__(self, bot):
        self.bot = bot

    @commands.command()
    async def translate(self, *, text : str):
        """Translates text to desired language."""
        pattern = re.compile("\-([a-z]{1,3})")
        m = pattern.match(text)

        if m:
            blob = TextBlob(text.split(' ', 1)[1])
            await self.bot.say("```" + str(blob.translate(to=str(m.group()[1:]))) + "```")
        else:
            blob = TextBlob(text)
            await self.bot.say("```" + str(blob.translate(to="en")) + "```")
@manurajhada

This comment has been minimized.

Show comment
Hide comment
@manurajhada

manurajhada Sep 2, 2016

@Sickness-chan "since approximately 2 days. It has worked just fine before." Same for me.

manurajhada commented Sep 2, 2016

@Sickness-chan "since approximately 2 days. It has worked just fine before." Same for me.

@mcamachoruiz

This comment has been minimized.

Show comment
Hide comment
@mcamachoruiz

mcamachoruiz Sep 2, 2016

Same here. It seems that google translation uses a token (called tk) generated in javascript to avoid bypasses in its calls. I saw that there is a function in translate.py (_calculate_tk) that calculates it. I guess that that function should be updated, but I still didn't figure out how. I hope that this points someone in the right direction.

mcamachoruiz commented Sep 2, 2016

Same here. It seems that google translation uses a token (called tk) generated in javascript to avoid bypasses in its calls. I saw that there is a function in translate.py (_calculate_tk) that calculates it. I guess that that function should be updated, but I still didn't figure out how. I hope that this points someone in the right direction.

@manurajhada

This comment has been minimized.

Show comment
Hide comment
@manurajhada

manurajhada Sep 5, 2016

Any update on this?

manurajhada commented Sep 5, 2016

Any update on this?

@gcucurull

This comment has been minimized.

Show comment
Hide comment
@gcucurull

gcucurull Sep 5, 2016

I'm experiencing this same problem since 4 or 5 days ago.

gcucurull commented Sep 5, 2016

I'm experiencing this same problem since 4 or 5 days ago.

@manurajhada

This comment has been minimized.

Show comment
Hide comment
@manurajhada

manurajhada Sep 5, 2016

Somehow fixed it. Not sure its a solution or workaround :D I just started fighting with it and made few changes in translate.py (/python/dist-packages/textblob/)

Change List (PFA translate.py.txt):

  • Updated tk calculation method. Now calculating tk using gTTS-token
  • TextBlob use to make http request using urllib (PY-3) & urllib2 (PY-2); somehow google marking this request as robot (although code is trying to fake it as browser using user-agent header). So I changed it to "requests" module of python.
  • Modified client param in the code which is creating data for http request, changed client's value as webapp, i.e. "client": "webapp".

Tested with Python 2.7.6, Ubuntu 14.04, TextBlob 0.11.1

Here is the Updated Code File:
translate.py.txt

Test Code:

from textblob import TextBlob
txt = u"मेरा नाम एलेक्स है"
#txt = "my name is alex"
b = TextBlob(txt)
print b.detect_language()
print b.translate(to='en')

Note: Changes made in existing code file is just to share a workable solution to the reported issue. It is shared by and as an individual; not from the owner or community.

manurajhada commented Sep 5, 2016

Somehow fixed it. Not sure its a solution or workaround :D I just started fighting with it and made few changes in translate.py (/python/dist-packages/textblob/)

Change List (PFA translate.py.txt):

  • Updated tk calculation method. Now calculating tk using gTTS-token
  • TextBlob use to make http request using urllib (PY-3) & urllib2 (PY-2); somehow google marking this request as robot (although code is trying to fake it as browser using user-agent header). So I changed it to "requests" module of python.
  • Modified client param in the code which is creating data for http request, changed client's value as webapp, i.e. "client": "webapp".

Tested with Python 2.7.6, Ubuntu 14.04, TextBlob 0.11.1

Here is the Updated Code File:
translate.py.txt

Test Code:

from textblob import TextBlob
txt = u"मेरा नाम एलेक्स है"
#txt = "my name is alex"
b = TextBlob(txt)
print b.detect_language()
print b.translate(to='en')

Note: Changes made in existing code file is just to share a workable solution to the reported issue. It is shared by and as an individual; not from the owner or community.

@sloria

This comment has been minimized.

Show comment
Hide comment
@sloria

sloria Sep 5, 2016

Owner

@manurajhada Would you be so kind as to send a PR with your fix?

Owner

sloria commented Sep 5, 2016

@manurajhada Would you be so kind as to send a PR with your fix?

@manurajhada

This comment has been minimized.

Show comment
Hide comment
@manurajhada

manurajhada Sep 6, 2016

Its working for small text only, for large paragraph google response is "Your client has issued a malformed or illegal request."

manurajhada commented Sep 6, 2016

Its working for small text only, for large paragraph google response is "Your client has issued a malformed or illegal request."

@AdrianLC

This comment has been minimized.

Show comment
Hide comment
@AdrianLC

AdrianLC Sep 6, 2016

Contributor

Porting the code to requests cannot be very significant to the fix as requests uses urllib itself.

They must have changed something to the tk parameter that we might never have gotten right, in the first place. We got it from another project but it's reverse engineered from minified javascript.

The repo for gTTS-token hasn't been updated since June so it won't be enough, assuming they indeed changed the code. Let's hope that someone can understand the new algorithm.

Contributor

AdrianLC commented Sep 6, 2016

Porting the code to requests cannot be very significant to the fix as requests uses urllib itself.

They must have changed something to the tk parameter that we might never have gotten right, in the first place. We got it from another project but it's reverse engineered from minified javascript.

The repo for gTTS-token hasn't been updated since June so it won't be enough, assuming they indeed changed the code. Let's hope that someone can understand the new algorithm.

@valmikadik

This comment has been minimized.

Show comment
Hide comment
@valmikadik

valmikadik Sep 6, 2016

Any update on this issue ? Getting the same while using textblob.
It was working fine until 2-3 days back. Any idea how to fix this ???

valmikadik commented Sep 6, 2016

Any update on this issue ? Getting the same while using textblob.
It was working fine until 2-3 days back. Any idea how to fix this ???

@valmikadik

This comment has been minimized.

Show comment
Hide comment
@valmikadik

valmikadik Sep 8, 2016

Seems like all the modules which internally uses the Google Translation APIs are having the same problem. I have tried goslate and same error.

valmikadik commented Sep 8, 2016

Seems like all the modules which internally uses the Google Translation APIs are having the same problem. I have tried goslate and same error.

@andrediaslopes

This comment has been minimized.

Show comment
Hide comment
@andrediaslopes

andrediaslopes Sep 8, 2016

Yep, same problem here.. I've also tried other modules and they are all replying error 503 :-\

andrediaslopes commented Sep 8, 2016

Yep, same problem here.. I've also tried other modules and they are all replying error 503 :-\

@jschnurr

This comment has been minimized.

Show comment
Hide comment
@jschnurr

jschnurr Sep 10, 2016

Collaborator

I've submitted PR #139, which works for me most of the time, but I still get some 503's. Let me know if it works for you.

Collaborator

jschnurr commented Sep 10, 2016

I've submitted PR #139, which works for me most of the time, but I still get some 503's. Let me know if it works for you.

@ysolis

This comment has been minimized.

Show comment
Hide comment
@ysolis

ysolis Sep 12, 2016

Affects me too, I learn traducing first to english (from spanish) now the learn databases are corrupted because the latest thing learned come from text in spanish, not traduced. I deploy my app and install TextBlob egg from pypi

ysolis commented Sep 12, 2016

Affects me too, I learn traducing first to english (from spanish) now the learn databases are corrupted because the latest thing learned come from text in spanish, not traduced. I deploy my app and install TextBlob egg from pypi

@WisdomWolf

This comment has been minimized.

Show comment
Hide comment
@WisdomWolf

WisdomWolf Oct 4, 2016

It seems that this is still broken. I was getting this error in HangoutsBot, so I setup a virtualenv to test TextBlob specifically and I'm seeing the same 503 error. :(

WisdomWolf commented Oct 4, 2016

It seems that this is still broken. I was getting this error in HangoutsBot, so I setup a virtualenv to test TextBlob specifically and I'm seeing the same 503 error. :(

@scne

This comment has been minimized.

Show comment
Hide comment
@scne

scne commented Oct 8, 2016

up!

@diegoesteves

This comment has been minimized.

Show comment
Hide comment
@diegoesteves

diegoesteves Oct 17, 2016

any API that uses Google as service will get 503! it's the same for others such as Goslate. It turns out that now you have to pay (https://cloud.google.com/translate/v2/pricing). Once they get your IP, it's done 👎

diegoesteves commented Oct 17, 2016

any API that uses Google as service will get 503! it's the same for others such as Goslate. It turns out that now you have to pay (https://cloud.google.com/translate/v2/pricing). Once they get your IP, it's done 👎

@jabbalaci

This comment has been minimized.

Show comment
Hide comment
@jabbalaci

jabbalaci Dec 31, 2016

Same issue here. I switched to https://github.com/Mimino666/langdetect , which seems to work.

jabbalaci commented Dec 31, 2016

Same issue here. I switched to https://github.com/Mimino666/langdetect , which seems to work.

@sloria

This comment has been minimized.

Show comment
Hide comment
@sloria

sloria Feb 27, 2017

Owner

This has been fixed in #148 . I will cut a release shortly.

Owner

sloria commented Feb 27, 2017

This has been fixed in #148 . I will cut a release shortly.

@sloria sloria closed this Feb 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment