FS3 Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4).
alias set, remove and list aliases in configuration file
ls list buckets and objects
mb make a bucket
rb remove a bucket
cp copy objects
mirror synchronize object(s) to a remote site
cat display object contents
head display first 'n' lines of an object
pipe stream STDIN to an object
share generate URL for temporary access to an object
find search for objects
sql run sql queries on objects
stat show object metadata
mv move objects
tree list buckets and objects in a tree format
du summarize disk usage recursively
retention set retention for object(s)
legalhold set legal hold for object(s)
diff list differences in object name, size, and date between two buckets
rm remove objects
encrypt manage bucket encryption config
event manage object notifications
watch listen for object notification events
undo undo PUT/DELETE operations
policy manage anonymous access to buckets and objects
tag manage tags for bucket(s) and object(s)
ilm manage bucket lifecycle
version manage bucket versioning
replicate configure server side bucket replication
admin manage MinIO servers
update update mc to latest release
list list swan info
car generate car file for filecoin offline deal
send send filecoin deal
Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow How to install Golang. Minimum version required is go1.13
# get submodules
git submodule update --init --recursive
make ffi
GO111MODULE=on go get github.com/filswan/fs3-mc
go build -o ./mcIf you are planning to use mc only on POSIX compatible filesystems, you may skip this step and proceed to everyday use.
To add one or more Amazon S3 compatible hosts, please follow the instructions below. mc stores all its configuration information in ~/.mc/config.json file.
./mc alias set <ALIAS> <YOUR-S3-ENDPOINT> <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api <API-SIGNATURE> --path <BUCKET-LOOKUP-TYPE>
is simply a short name to your cloud storage service. S3 end-point, access and secret keys are supplied by your cloud storage provider. API signature is an optional argument. By default, it is set to "S3v4".
Path is an optional argument. It is used to indicate whether dns or path style url requests are supported by the server. It accepts "on", "off" as valid values to enable/disable path style requests.. By default, it is set to "auto" and SDK automatically determines the type of url lookup to use.
MinIO server displays URL, access and secret keys.
./mc alias set minio http://192.168.1.51 BKIKJAA5BMMU2RHO6IBB V7f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12
Get your AccessKeyID and SecretAccessKey by following AWS Credentials Guide.
./mc alias set s3 https://s3.amazonaws.com BKIKJAA5BMMU2RHO6IBB V7f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12
Note: As an IAM user on Amazon S3 you need to make sure the user has full access to the buckets or set the following restricted policy for your IAM user
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowBucketStat",
"Effect": "Allow",
"Action": [
"s3:HeadBucket"
],
"Resource": "*"
},
{
"Sid": "AllowThisBucketOnly",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<your-restricted-bucket>/*",
"arn:aws:s3:::<your-restricted-bucket>"
]
}
]
}Get your AccessKeyID and SecretAccessKey by following Google Credentials Guide
./mc alias set gcs https://storage.googleapis.com BKIKJAA5BMMU2RHO6IBB V8f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12
mc is pre-configured with https://play.min.io, aliased as "play". It is a hosted MinIO server for testing and development purpose. To test Amazon S3, simply replace "play" with "s3" or the alias you used at the time of setup.
Example:
List all buckets from https://play.min.io
mc ls play
[2016-03-22 19:47:48 PDT] 0B my-bucketname/
[2016-03-22 22:01:07 PDT] 0B mytestbucket/
[2016-03-22 20:04:39 PDT] 0B mybucketname/
[2016-01-28 17:23:11 PST] 0B newbucket/
[2016-03-20 09:08:36 PDT] 0B s3git-test/
Make a bucket
mb command creates a new bucket.
Example:
./mc mb play/mybucket
Bucket created successfully `play/mybucket`.
Copy Objects
cp command copies data from one or more sources to a target.
Note
Example:
./mc cp myobject.txt play/mybucket
myobject.txt: 14 B / 14 B ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 100.00 % 41 B/s 0
- A running lotus node at local
- A filecoin wallet with sufficient balance to send deal, set as environment variable $FIL_WALLET
- FS3 Server credentials set as environment variables $ENDPOINT, $ACCESS_KEY, $SECRET_KEY
Environment variables ENDPOINT, ACCESS_KEY, SECRET_KEY need to be set for FS3 Server authorization and usage.
# export FS3 Serve Endpoint
export ENDPOINT=<FS3_SERER_ENDPOINT>
# export FS3 Server Access Key
export ACCESS_KEY=<FS3_SERVER_ACCESS_KEY>
# export FS3 Server SECRET Key
export SECRET_KEY=<FS3_SERVER_SECRET_KEY>Example:
# export FS3 Serve Endpoint
export ENDPOINT=http://FS3_SERVER_URL:PORT
# export FS3 Server Access Key
export ACCESS_KEY=minioadmin
# export FS3 Server SECRET Key
export SECRET_KEY=minioadminNote: The FS3 server must be running when doing the uploading process. Make sure it is runing before uploading files.
You can simply upload a local file to the designated bucket of FS3 Server using mc with cp
./mc cp PATH/TO/LOCAL/FILE minio/BUCKETExample:
# Upload test.txt to bucket 'test_bucket'
./mc cp /tmp/test/test.txt minio/test_bucketyou may send an online deal to a miner
A wallet address is a must for sending deals to miner. You can change it via setting environment variable FIL_WALLET. Or you can set it up as an augment passing to mc when sending deals.
# export wallet address
export FIL_WALLET=<MY_FIL_WALLET>Import the file stored in FS3 to Filecoin, then you can share it to your miner
./mc import --bucket [bucket] --object [object]For example:
./mc import --bucket testBucket --object test.zipNote:
The defaultVolumeAddress where fs3 store the uploaded data is ~/minio-data. It can be changed in file fs3-mc/cmd/Config-v10.go
A message that contains Bucket,Object and Datacid will be returned if successful.
sendonline command can send an online deal to a designated miner, a fully synchronized lotus node at local is required
--from: (optional) specify filecoin wallet to use, default: $FIL_WALLET
--verified-deal: specify whether deal is verified, default: "false" ('true' if verified)
--fast-retrieval: specify data retrieval type, defalult: 'true' ('false' if not using fast retrieval approach)
--data-cid: specify the valid data-cid for sending deal
--miner-id: specify which miner to send deal to
--price: specify the deal price for each GiB of file, default: 0
--duration: specify length in day to store the file, default: 1036800
Example:
./mc sendonline --from nusx7m3exqsfkxezncpefsf6fmian --verified-deal false --fast-retrieval true --data-cid m7xmefllqsixl5 --miner-id t00001 --price 0.00005 --duration 1036800A message that contains all the deal information and Dealcid will be returned if successful.
You may add shell aliases to override your common Unix tools.
alias ls='mc ls'
alias cp='mc cp'
alias cat='mc cat'
alias mkdir='mc mb'
alias pipe='mc pipe'
alias find='mc find'
In case you are using bash, zsh or fish. Shell completion is embedded by default in mc, to install auto-completion use mc --autocompletion. Restart the shell, mc will auto-complete commands as shown below.
mc <TAB>
admin config diff find ls mirror policy session sql update watch
cat cp event head mb pipe rm share stat version