Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace polyglot with langdetect #259

Open
abitrolly opened this issue Nov 16, 2020 · 5 comments
Open

Replace polyglot with langdetect #259

abitrolly opened this issue Nov 16, 2020 · 5 comments

Comments

@abitrolly
Copy link
Collaborator

https://github.com/aboSamoor/polyglot depends on pycld2, which depends on PyICU, which needs C compiler and tons of stuff to build, because it doesn't ship wheels.

The only polyglot feature what is used it language detection - https://polyglot.readthedocs.io/en/latest/Detection.html

This can be done with less sophisticated dependencies, such as https://github.com/Mimino666/langdetect

@chubin
Copy link
Owner

chubin commented Nov 22, 2020

It is a good idea. We should test it and see if it detects language reliably enough. I believe, I considered it first, before switching to polyglot, and it detected languages way worse than the latter one

@abitrolly
Copy link
Collaborator Author

I am not even sure why language is needed as all https://cheat.sh output seems to be only English.

@chubin
Copy link
Owner

chubin commented Nov 22, 2020

It is for input, not for output.

$ curl cht.sh/python/прочитать+json
#  python - Parsing values from a JSON file?
#
#  I think what Ignacio is saying is that your JSON file is incorrect.
#  You have [] when you should have {}. [] are for lists, {} are for
#  dictionaries.
#
#  Here's how your JSON file should look, your JSON file wouldn't even
#  load for me:

{
    "maps": [
        {
            "id": "blabla",
            "iscategorical": "0"
        },
        {
            "id": "blabla",
            "iscategorical": "0"
        }
    ],
    "masks": {
        "id": "valore"
    },
    "om_points": "value",
    "parameters": {
        "id": "valore"
    }
}
...

This feature is broken at the moment though, but we will fix it

@abitrolly
Copy link
Collaborator Author

@chubin is that supposed to search only for Russian results?

@chubin
Copy link
Owner

chubin commented Nov 23, 2020

No, for any language; the feature is currently broken though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants