Languages plugin with empty result

[lowlighter]

Ref: #511

Seems in some case the languages plugin returns an empty result

Need to investigate

[Nixinova]

Copying the relevent comments:

---

@Nixinova #511 (comment)

Hm, this merge seems to have broken the languages plugin, await linguist(path) returns nothing:

I don't see where this would have broken, though, since it seems to be working perfectly fine for me, even when using a Linux subsystem:

🤔 🤔

---

@lowlighter #511 (comment)

I don't think it's globally broken, mine still seems to be up (it was refreshed 2h ago)

Will try to investigate about it later 👍

---

@Nixinova #511 (comment)

Huh, this is weird.

#502 also doesn't appear, double weird...

[Nixinova]

Seems to be an independent issue, Nixinova/Nixinova@d3719c3 was generated with e3cd805 (Aug 26) which was working prior.

[Nixinova]

Also typo in issue title: 'Langage'

[Nixinova]

Now crashes with TypeError: Cannot read property 'length' of undefined at file:///metrics/source/plugins/languages/analyzers.mjs:81:34

[lowlighter]

Seems to be an independent issue, Nixinova/Nixinova@d3719c3 was generated with e3cd805 (Aug 26) which was working prior.

The latter seems to be caused because of how paths were reported (not sure whether it's git or linguist which changed), but one was absolute and the other relative to the repository. Also on windows the paths used partially \\ instead of /

Now crashes with TypeError: Cannot read property 'length' of undefined at file:///metrics/source/plugins/languages/analyzers.mjs:81:34

This was introduced because of #517, but should be fix now 👍

[Milo123459]

Still getting this issue: https://github.com/Milo123459/Milo123459/runs/3527913159?check_suite_focus=true

I had 18 emails.. 😆

[Nixinova]

I've got a fix for that crash coming up

[Milo123459]

I've got a fix for that crash coming up

Woo! Can't wait. Currently had to switch to latest branch to suppress emails. :(

[thib3113]

The PR seems to doesn't resolve the problem :

file:///metrics/source/plugins/languages/analyzers.mjs:172
            if (!categories.includes(languageResults[lang].type))
                                                           ^

TypeError: Cannot read property 'type' of undefined
    at Interface.stdout (file:///metrics/source/plugins/languages/analyzers.mjs:172:60)
    at Interface.emit (node:events:394:28)
    at Interface._onLine (node:readline:487:10)
    at Interface._normalWrite (node:readline:661:12)
    at Socket.ondata (node:readline:269:10)
    at Socket.emit (node:events:394:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at Pipe.onStreamRead (node:internal/stream_base_commons:199:23)

[Milo123459]

Is this now fixed?

[thib3113]

@Milo123459 not sure ... on my side it doesn't crash ... But "locale" part is now totally empty ... ( https://gist.githubusercontent.com/thib3113/83a96e16f8bca103f1b0e376186c66ec/raw/github-metrics.svg )

[lowlighter]

@Milo123459 @thib3113 Currently only the crash issue has been fixed, but the initial issue is remaining

---

I debugged a bit and it seems to come from here:

metrics/source/plugins/languages/analyzers.mjs

Lines 142 to 144 in af8c476

	//Gather language data
	console.debug(`metrics/compute/${login}/plugins > languages > indepth > running linguist`)
	const {files:{results:files}, languages:{results:languageResults}} = await linguist(path)

I'm not sure but I think it's an issue with linguist.

Seems that despite the repository path is correctly given (in this case /tmp/22963968-lowlighter_itsudeno) it looks for files in the first processed repository (in this case /tmp/22963968-lowlighter_metrics/)

DEBUG | repository to analyze /tmp/22963968-lowlighter_itsudeno
Error: ENOENT: no such file or directory, lstat '/tmp/22963968-lowlighter_metrics/.dockerignore'
    at Object.lstatSync (node:fs:1514:3)
    at analyse (/metrics/node_modules/linguist-js/dist/index.js:106:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async analyze (file:///metrics/source/plugins/languages/analyzers.mjs:147:72)
    at async indepth (file:///metrics/source/plugins/languages/analyzers.mjs:32:7)
    at async Object.default [as languages] (file:///metrics/source/plugins/languages/index.mjs:72:34)
    at async file:///metrics/source/plugins/core/index.mjs:50:30
    at async Promise.all (index 0)
    at async metrics (file:///metrics/source/app/metrics/index.mjs:61:22)
    at async file:///metrics/source/app/action/index.mjs:292:24 {
  errno: -2,
  syscall: 'lstat',
  code: 'ENOENT',
  path: '/tmp/22963968-lowlighter_metrics/.dockerignore'
}

Seems that only the first repository can be correctly processed.

Languages results may still be empty if the first repository processed contains only data/prose languages (like YAML or markdown) which are ignore by default.

The error thrown by linguist-js seems to be have fixed in 2.0.2 but it's weird that it is looking for a file of a separate repository 🤔
@Nixinova do you have more insight about this?

[Nixinova]

The lstat issue is Nixinova/LinguistJS#4 but I'm not sure why it would be looking in a different directory. I'd have to have the full linguist-js output to know.

[lowlighter]

Upgrading to linguist latest version seems to have fixed it 👍