Goofys is a high-performance, POSIX-ish Amazon S3 file system written in Go
Goofys allows you to mount an S3 bucket as a filey system.
It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.
On Linux, install via pre-built binaries. You may also need to install fuse-utils first.
On Mac OS X, install via Homebrew:
$ brew cask install osxfuse $ brew install goofys
- Or build from source:
$ export GOPATH=$HOME/work $ go get github.com/kahing/goofys $ go install github.com/kahing/goofys
$ cat ~/.aws/credentials [default] aws_access_key_id = AKID1234567890 aws_secret_access_key = MY-SECRET-KEY $ $GOPATH/bin/goofys <bucket> <mountpoint> $ $GOPATH/bin/goofys <bucket:prefix> <mountpoint> # if you only want to mount objects under a prefix
Users can also configure credentials via the
AWS_SECRET_ACCESS_KEY environment variables.
To mount an S3 bucket on startup, make sure the credential is
root, and can add this to
goofys#bucket /mnt/mountpoint fuse _netdev,allow_other,--file-mode=0666 0 0
Got more questions? Check out questions other people asked
--stat-cache-ttl 1s --type-cache-ttl 1s for goofys
-ostat_cache_expire=1 for s3fs to simulate cold runs. Detail for the
benchmark can be found in
bench.sh. Raw data
is available as well. Test was run on an EC2 m4.16xlarge in us-west-2a
connecting to a bucket in us-west-2. Units are seconds.
(†) riofs does not wait for HTTP response before returning from
release(), so the create files benchmarks do not measure the right thing for it
Benchmark with caching enabled
--cache has little impact on write speed (since
implements a write-through cache) but read has a large variance. Time
to first byte is competitive with
s3fs which suggests layering fuse
filesystems can be a viable approach.
$ cat > ~/.passwd-riofs export AWS_ACCESS_KEY_ID=AKID1234567890 export AWS_SECRET_ACCESS_KEY=MY-SECRET-KEY $ sudo docker run -e BUCKET=$TESTBUCKET -e CACHE=false --rm --privileged --net=host -v ~/.passwd-riofs:/root/.passwd-riofs -v /tmp/cache:/tmp/cache kahing/goofys-bench # result will be written to $TESTBUCKET
CACHE is set to
true, the read benchmarks ('Read 1GB' and 'Time to 1st byte') will be cached read.
Copyright (C) 2015 - 2017 Ka-Hing Cheung
Licensed under the Apache License, Version 2.0
goofys has been tested under Linux and OS X.
List of non-POSIX behaviors/limitations:
- only sequential writes supported
- does not store file mode/owner/group
- does not support symlink or hardlink
atimeis always the same as
- cannot rename non-empty directories
unlinkreturns success even if file is not present
fsyncis ignored, files are only flushed on
In addition to the items above, the following supportable but not yet implemented:
- creating files larger than 1TB
Compatibility with non-AWS S3
goofys has been tested with the following non-AWS providers:
- DreamObjects (Ceph)
- EMC Atmos
- Google Cloud Storage
- OpenStack Swift
- Minio (limited)
- Data is stored on Amazon S3
- Amazon SDK for Go
- Other related fuse filesystems
- S3Proxy is used for
- fuse binding, also used by