Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Home

silviusrus edited this page · 66 revisions

Welcome to QFS Wiki

We have released QFS to open source, at http://quantcast.github.com/qfs.

New: QFS paper presented at VLDB.

Post comments or questions to qfs-devel@googlegroups.com or search the archives at QFS Developer Mailing List.

Introduction To QFS
Repository Organization
Administrator's Guide
Deployment Guide
Configuration Reference
Developer Documentation
Migration Guide
Performance Comparison to HDFS

This page describes the steps necessary to test-drive QFS on your computer. For multi-node setups, please refer to the Deployment Guide.

Getting Started

Binary Distribution

Several binary distributions have been provided to make it easy to try out QFS on a single node. See the Download Page for a list of tarballs organized by operating system.

Single Node Test-Drive

The first step of the single-node setup is to download the tarball that matches the Linux/OS X distribution you are running.

Setup

Setting up a single node configuration to familiarize yourself with QFS is very easy.

1. Extract the distribution tarball.
$ tar -xzf $QFSTAR.tgz && cd $QFSTAR

2. Set up a single node QFS instance. This will create a workspace in ~/qfsbase, start two chunk servers and one metaserver.
$ ./examples/sampleservers/sample_setup.py -a install

Binaries presence checking - OK.
Setup directories - OK.
Setup config files - OK.
Started servers - OK.

3. Confirm everything is running.

$ ps x | grep [q]fsbase
14424 pts/6    Sl     0:00 ./examples/sampleservers/../../bin/metaserver -c /home/zim/qfsbase/meta/conf/MetaServer.prp /home/zim/qfsbase/meta/MetaServer.log
14429 pts/6    Sl     0:00 ./examples/sampleservers/../../bin/chunkserver /home/zim/qfsbase/chunk2/conf/ChunkServer.prp /home/zim/qfsbase/chunk2/ChunkServer.log
14431 pts/6    Sl     0:00 ./examples/sampleservers/../../bin/chunkserver /home/zim/qfsbase/chunk1/conf/ChunkServer.prp /home/zim/qfsbase/chunk1/ChunkServer.log
14442 pts/6    S      0:00 python ./examples/sampleservers/../../webui/qfsstatus.py /home/zim/qfsbase/web/conf/WebUI.cfg

The file system web UI will be running on port 22000:

http://localhost:22000

Note: In a production environment only one chunk server is deployed per QFS instance per host. For simplicity this example deploys two chunk servers that belong to the same QFS instance to a single host. Hence the capacity statistics are counted twice and reported as such via the web UI at http://localhost:22000.

Trying It Out

Ensure that the metaserver and chunk servers are running.

Add tools binary path to PATH:
$ PATH=${PWD}/bin/tools:${PATH}

Make a temp directory on the file system.
$ qfsshell -s localhost -p 20000 -q -- mkdir /qfs/tmp

Create a file containing "Hello World", Reed-Solomon encoded, replication 1.
$ echo 'Hello World' | cptoqfs -s localhost -p 20000 -S -r 1 -k /qfs/tmp/helloworld -d -

Cat the file content.
$ qfscat -s localhost -p 20000 /qfs/tmp/helloworld

Stat the file to see encoding (RS or not), replication level, and mtime.
$ qfsshell -s localhost -p 20000 -q -- stat /qfs/tmp/helloworld

Copy the file locally to the current directory.
$ cpfromqfs -s localhost -p 20000 -k /qfs/tmp/helloworld -d ./helloworld

Remove the file from QFS.
$ qfsshell -s localhost -p 20000 -q -- rm /qfs/tmp/helloworld

Stopping Servers

$ ./examples/sampleservers/sample_setup.py -a stop

Uninstalling

$ ./examples/sampleservers/sample_setup.py -a uninstall

Retrieving the Source Code

git clone https://github.com/quantcast/qfs.git

Quantcast

Something went wrong with that request. Please try again.