Skip to content

Commit

Permalink
doc fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
sylvinus committed Nov 7, 2010
1 parent 4912c69 commit c0594b5
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 26 deletions.
34 changes: 15 additions & 19 deletions README.rst
Expand Up @@ -10,6 +10,7 @@ Goals
fsdfs was originally developed at Jamendo.com to manage the distribution of millions of MP3/OGG files.

fsdfs aims to :

* make a single virtual filesystem accessible from several nodes
* be fully distributed (no single point of failure)
* automatically manage sharding/replication : adding a new node takes 10 seconds
Expand All @@ -31,13 +32,15 @@ Design for version 1.0
----------------------

Each file has two important variables :
* Its target replication count N. (N=3 by default, can be customized per-file by a user-defined python function)
* Its current replication count K

* Its target replication count N. (N=3 by default, can be customized per-file by a user-defined python function)
* Its current replication count K

The master has a list of all the nodes with their status, which includes :
* cpu/network load
* list of all the files replicated on the node
* free disk quota

* cpu/network load
* list of all the files replicated on the node
* free disk quota

The free disk quota will always tend to zero because each nodes tries to replicate as many files as it can.
Hence, max(K-N) is a more interesting number for a node, because it measures the extent of "over-replication".
Expand All @@ -51,9 +54,9 @@ The algorithm used is very simple :

1. select the file having min(K-N)
2. choose a node :
* not already having a copy of this file
* not too loaded
* with either free disk or a high max(K-N)
* not already having a copy of this file
* not too loaded
* with either free disk or a high max(K-N)
3. Have the file copied to this node (=> K++)
4. GOTO 1

Expand All @@ -66,8 +69,11 @@ min(K-N) will naturally decrease when adding new files. min(K-N) becoming <1 or
How to test
-----------

just run::

python tests/basic.py
python tests/quota.py
python tests/persist.py


.. include:: TODO.rst
Expand All @@ -76,17 +82,7 @@ How to test
Installation
------------

The easiest way to get fsdfs is if you have setuptools_ installed::

easy_install fsdfs

Without setuptools, it's still pretty easy. Download the fsdfs.tgz file from
`fsdfs's Cheeseshop page`_, untar it and run::

python setup.py install

.. _fsdfs's Cheeseshop page: http://pypi.python.org/pypi/fsdfs/
.. _setuptools: http://peak.telecommunity.com/DevCenter/EasyInstall
There are no packages yet, just clone our github repo!


Help and development
Expand Down
10 changes: 5 additions & 5 deletions TODO.rst
Expand Up @@ -4,14 +4,14 @@ Todo
- Admin command (watch replication)
- Test crashes/disconnects/...
- Decide on how to manage file updates. Either:
- Nuke then insert
- Append md5 to all filenames
- Manage a file version system
- Nuke then insert
- Append md5 to all filenames
- Manage a file version system
- SQLite or other persistent filedb backend
- Complete Sphinx documentation
- Outreach to community
- setup twitter account / polish website
- do public talks / pycon / ...
- setup twitter account / polish website
- do public talks / pycon / ...
- performance tests
- fsdfsd
- FUSE layer?
Expand Down
4 changes: 2 additions & 2 deletions docs/api.rst
@@ -1,12 +1,12 @@
API Reference
=============

.. automodule:: fsdfs.filesystem
.. autoclass:: fsdfs.filesystem.Filesystem
:members:
:undoc-members:
:inherited-members:

.. automodule:: fsdfs.replicator
.. autoclass:: fsdfs.replicator.Replicator
:members:
:undoc-members:
:inherited-members:
Expand Down

0 comments on commit c0594b5

Please sign in to comment.