Go library to list files quickly (using goroutines)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
c
LICENSE
README.md
dirlist.c
go.mod
listfiles.go
listfiles_test.go

README.md

listfiles

List files recursively, requires cgo.

There are three methods here, it seems one of them is much better on Windows and one is much better on Linux (no tests on OSX). Neither of the better methods if the std filepath.Walk (StdLib).

Benchmarks

On Windows (8-core) with ntfs-formatted SSD and tens of thousands files:

  • ListFilesUsingC: 132 seconds
  • ListStdLib: 24 seconds
  • ListInParallel: 1.8 seconds

On Linux (8-core) with ext4-formatted SSD and a million files:

  • ListFilesUsingC: 5.1 seconds
  • ListStdLib: 5.5 seconds
  • ListInParallel: 7.2 seconds

On Linux (8-core) with xfs-formatted 7200rpm disk and hundred thousand files:

  • ListFilesUsingC: 1.8 seconds
  • ListStdLib: 3.0 seconds
  • ListInParallel: 3.9 seconds