Add option to configure which Linguist categories to display

[Nixinova]

Currently the languages plugin mirrors GitHub and only shows programming and markup languages as categorized by Linguist. However it may be useful to also be able to display other types such as data and prose, if this is possible. Id like this for recent languages in particular as that section doesn't need to show only programming languages and can show if you've, e.g., been working on markdown docs.

The options should be something like plugin_languages_{overall,recent}_types with csv options programming, markup, data, prose corresponding to Linguist categories (defaults to the first two)

[Nixinova]

Hm, I don't think this is possible as the languages API endpoint doesn't offer any parameters to control which languages are outputted. Would it be possible with indepth, though, as it manually runs github-linguist?

[lowlighter]

Maybe, though I didn't see any stuff about this in linguist 🤔

Does it have other cli options apart from --breakdown and --json ?

[Nixinova]

Doesn't look like it

[lowlighter]

If these are already in linguist but just not exposed through cli, it may be worth to open an issue upstream

[Nixinova]

If you have a list of files changed you can loop through them and run github-linguist <file> in them individually to get their languages

[lowlighter]

I think metrics already has the information about which file is in which language (it's included in the --json file)

It's more the programming, markup, data, prose categorie that is isn't accessible, unless I missed something

[Nixinova]

I've made my own Linguist (with language data retrieved from the github-linquist repo via JS-fetch), and allows for all language types to be parsed. Maybe try this?

[lowlighter]

Seems promising 👍

When you say that data are retrieved through fetch, do you mean that all languages/grammar definitions are fetched from upstream github/linguist at runtime to keep them updated ?

Also, does it has the same constraint as ruby linguist where you need to create a git repo first or a regular directory will do ?

I think that the --analyze --full should make it possible to switch to it.

I'm working on other stuff currently, but if you want you can give it a shoot, the editions to do should be around here:

metrics/source/plugins/languages/analyzers.mjs

Line 145 in b24c26f

const files = Object.fromEntries(Object.entries(JSON.parse(await imports.run("github-linguist --json", {cwd:path}, {log:false}))).flatMap(([lang, files]) => files.map(file => [file, lang])))

[Nixinova]

When you say that data are retrieved through fetch, do you mean that all languages/grammar definitions are fetched from upstream github/linguist at runtime to keep them updated ?

HTTP-fetch from github-linguist - fetch(githubLinguistURL + 'languages.yml')

Also, does it has the same constraint as ruby linguist where you need to create a git repo first or a regular directory will do ?

Any folder works.

I think that the --analyze --full should make it possible to switch to it.

It isn't CLI-only so using linguist(folder) in JS would work

[lowlighter]

Seems good so, I think we should poc it

[lowlighter]

This was shipped in #477 thanks to @Nixinova 🎉