Implement libmagic in JS

[mifi]

The holy grail would be to build a JS parser for the magic database from libmagic and bundle it in the module, so we don't have to reimplement all the format parsing manually in code.
libmagic is the database used by the "file" command in unix. However this is not trivial.

See:

• https://github.com/threatstack/libmagic/tree/master/magic/Magdir
• https://linux.die.net/man/5/magic
• https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man5/magic.5.html
• MKV detected as WebM #67

[sindresorhus]

This is not something I intend to work on, but I asked for this issue to be opened in case someone wants a good challenge 🦄🤘

[hollowdoor]

Out of curiosity would you want the database downloaded in a build step? Then parse it into js arrays, or let be to be parsed as a stream? Or downloaded post install then parsed in the user program?

Not saying I'm going to do it. I'm just curious. :)

[mifi]

I think it should be compiled to js for maximum speed. I tried to just make a simple runtime parser that loads the magdir files and parses them and then runs those rules on input files, but it was quite slow. And some of the rules in libmagic are quite obscure and tricky to implement correctly. And the detection of many popular file types uses these complicated rules... so i gave up 😅

[hollowdoor]

Would the ideal js output be a structure that was meant to be consumed, or a full process with logic?

And some of the rules in libmagic are quite obscure and tricky to implement correctly

Looking at the magic database I would agree.

[aaronArinder]

There's another project, mmmagic, that uses node addons to keep from re-implementing everything in magic.mgc. Pretty sweet stuff! The only problem, though, is that it says it uses file 5.32, but the md5sums from my in-system magic.mgc (compiled from file 5.32) and the project's magic.mgc are different. Still, gives some indication of how to do it.

[mifi]

Cool! Great find @aaronArinder !
Another problem is that if using that lib, it will only work in Node.js and not in the browser or react native like file-type does.

[mceachen]

FYI, file doesn't support many of the file types that this library does (including raw image types)

[sindresorhus]

I'm gonna close this as I don't see enough value in adopting libmagic. This package already covers most popular formats now, without all the legacy of libmagic, and it also works in the browser.