Split LangCheck into language-specific packages such as langcheck[ja], langcheck[de], etc

[kennysong]

Since some language-specific packages (e.g. fugashi for Japanese) may require extra work to install, we can simplify the installation instructions for users by allowing them to choose only the languages they want to install.

[Vela-zz]

hello, kenny, I may have time add a small update for this

but I have few questions

1. what should default install command pip install langcheck do, would it only install support for default language, like [en] e.g. or let it install supoprt for all language? install former way seem easier to realized.
2. if split like this

dev = [
    "yapf==0.40.1",
    "flake8",
    "isort",
    "pyright",
    "pandas-stubs",
    "pytest",
    "sphinx",
    "sphinx-book-theme",
    "myst-parser",
    "ipykernel"
]
zh = [
    'hanlp >= 2.1.0b33',
]
ja = [
    'fugashi',  # For tokenizer of metrics.ja.toxicity()
    'janome >= 0.3.1',
]

the github action file may also need a small update like, pip install -e .[dev,zh,ja], e.g.

what's your opinion ?

[kennysong]

Hi @Vela-zz, that would be fantastic!

• what should default install command pip install langcheck do, would it only install support for default language, like [en] e.g. or let it install supoprt for all language? install former way seem easier to realized.

I think we can make pip install langcheck install a default language (English) to improve the default installation process.

the github action file may also need a small update like, pip install -e .[dev,zh,ja], e.g.

Agreed, we'll need to update all GH Actions, including the new Pip Install Matrix (we might want to add a third matrix dimension for "language").

dev = [
    "yapf==0.40.1",
    "flake8",
    "isort",
    "pyright",
    "pandas-stubs",
    "pytest",
    "sphinx",
    "sphinx-book-theme",
    "myst-parser",
    "ipykernel"
]
zh = [
    'hanlp >= 2.1.0b33',
]
ja = [
    'fugashi',  # For tokenizer of metrics.ja.toxicity()
    'janome >= 0.3.1',
]

@liwii do you know which dependencies in pyproject.toml are Japanese-only?

@ischender do you know which dependencies in pyproject.toml are German-only? (if any?)

---

I'm happy to help you with parts of this PR, e.g. I can update GitHub Actions, if you want.

[kennysong]

Also, not sure if this is possible, but it might be nice to have pip install langcheck[all] which installs all languages without dev dependencies.

[ischender]

That would be neat!
@kennysong , I am pretty sure the only German only part is that we download the translation model from Huggingface, but that happens when it's run the first time.