Repository navigation
v0.5.0: a port of libmagic to pure Go
magic is now a port of libmagic to pure Go. It needs no cgo, no C compiler and no libmagic, and a program that imports it builds for every target of Go.
The port gives the output of libmagic 5.48, with the commits of file after the release, to bbffb78d. The rules of file are compiled into the package.
A new API
The API is new, so this is a new minor version (D39). Detect, DetectFile, DetectReader and DetectReaderAt take a context.Context and return a Result. A Result holds every form of the answer at once: Description, MIME, Charset, Extensions, Apple and Matches. Options such as WithAll, WithDecompress and WithLimit take the place of the flags of libmagic. The package has no handle to close.
The README has a table that maps each call of v0.4.1 to the port. A file that cannot be opened is now an error, where libmagic gave the text "cannot open".
Compressed data
WithDecompress decodes a format only when the program imports its decoder, as a program imports the driver of database/sql:
import (
_ "github.com/xo/magic/compress/bzip2"
_ "github.com/xo/magic/compress/xz"
_ "github.com/xo/magic/compress/zlib"
_ "github.com/xo/magic/compress/zstd"
)compress/xz and compress/zstd are modules of their own, tagged compress/xz/v0.5.0 and compress/zstd/v0.5.0. magic never runs a program.
Size and speed
A program that prints the MIME type of a file grows by about 980 KB, stripped, on linux/amd64. Most of it is the compiled rules.
The engine is about as fast as libmagic on most files, and slower on text. On linux/amd64, one walk for the MIME type took, with v0.4.1 and with the port:
| Input | v0.4.1 (cgo) | v0.5.0 |
|---|---|---|
| Word document | 5.1 µs | 4.3 µs |
| GIF image | 20 µs | 26 µs |
| XLSX file | 35 µs | 37 µs |
| random bytes | 252 µs | 334 µs |
| UTF-8 text | 334 µs | 1.09 ms |
Most of the time on text is in Go's regexp. docs/BACKLOG.md lists the work that can make it faster.
How it is tested
The test module builds libmagic from the source of file at bbffb78d and compares the port with it: 2,471 files and damaged copies of them, 14 combinations of flags, as bytes and as files. Every answer is the same, except 50 where a decoder of Go reads damaged compressed data in another way than the library of C. All 113 tests of file pass.
License
The port translates file, which is under its own BSD license, in LICENSE.file. A program that ships magic must reproduce the notices that the README lists (D29).
Removed
The cgo package, MagicDB, the static archives and the modules of the targets are gone. The tags of those modules, such as libmagic/linux_amd64/v5.48.2, stay published.