Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Big brother file system (distributed file system)
Ruby C
Branch: master
Pull request Compare This branch is 917 commits behind bbfsdev:master.

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
bin
examples
lib
test
Gemfile
README
Rakefile
content_data.gemspec
content_server.gemspec
file_copy.gemspec
file_indexing.gemspec
file_monitoring.gemspec
file_monitoring_win32.gemspec_
file_utils.gemspec
log.gemspec
params.gemspec
reinitialize_all_bbfs_gems.rb
run_all_tests.rb

README

Big brother file system (distributed file system).
--------------------------------------------------

Backup system:
--------------
Content server monitors a set of directories (blobs/patterns).
The files in those directories are indexed (calculating their SHA1).
Each unique content is backed up to the remote (backup) server.
The backup server also monitors a set of directories (blobs/patterns).
The files in those directories are also indexed.
The content server copies new/changed files in the backup server.
The backup server index those new files too and send the content data (their SHA1 to the original server).

How to install and run:
-----------------------
A) Installation of BBFS development kit:
  1) Download and install git.
  2) Download and install ruby.
  3) Download and unpack and install DevKit (Windows). Note that after unpacking there are installation instructions at
     https://github.com/oneclick/rubyinstaller/wiki/Development-Kit.
  4) Mac (only) users will have to install XCode for gcc compiler.
  Run in command prompt (Windows users should run in Git Bash):
  5) git clone https://github.com/kolmanv/bbfs.git
  6) cd bbfs
  7) gem install bundler
  8) bundle install
  9) rake  # This will run all project tests and specs.
  10) You are done, you can start coding on the project.

B) (Optional) Recommended:
  1) Download and install smartgit.
  2) Download and install Ruby IDE (for example RubyMine).

C) Installing SFTP server backup server dependency (For now a must!):
  Install and run sftp server (for example set a 'test' user with 'test' password) on same computer
  which runs backup server with user working directory set to backup file monitoring dir,
  for example: C:/Users/kolman/backup_data
  Make sure the user have write permissions into the directory.
  Make sure the sftp server configured to support username/password authentication.

D) Preparing configuration files:
  Prepare two monitoring yml files. Each server, i.e., content server and backup server needs
  a monitoring configuration file.
  1) Configuration file for content server which is located in ~/.bbfs/etc/file_monitoring.yml:

    paths:
      - path: C:/Users/kolman/test_files  # <=== replace this directory with your local directory
        scan_period: 1
        stable_state: 5

  2) Configuration file for backup server which is located in ~/.bbfs/etc/backup_file_monitoring.yml:

    paths:
      - path: C:/Users/kolman/backup_data  # <=== replace this directory with your local directory
        scan_period: 1
        stable_state: 5

E) Note that for now parameter parsing is not yet available so just hard-code the parameters in the
  executables and run the local code:

  For content server edit the executable bin/content_server and add those parameters just before the
  run command (BBFS::ContentServer.run):
  ...
  # Replace values to suite your local configuration
  BBFS::Params.remote_server = '127.0.0.1'
  BBFS::Params.backup_username = 'test'
  BBFS::Params.backup_password = 'test'
  BBFS::Params.content_data_path = File.expand_path('~/.bbfs/var/server_content.data')
  ...

  For backup server edit the executable bin/backup_server and add the parameter just before the
  run command (BBFS::ContentServer.run_backup_server):
  ...
  # Replace value to suite your local configuration
  BBFS::Params.monitoring_config_path = File.expand_path('~/.bbfs/etc/backup_file_monitoring.yml')
  ...

F) Run content server:
  bbfs> ruby -Ilib bin/content_server
  Run backup server (after content server is running):
  bbfs> ruby -Ilib bin/backup_server

G) (Optional) Later when paramter parsing will be available we could do:
  Run content server:
  bbfs> ruby -Ilib bin/content_server --remote_server='127.0.0.1' --backup_username=test \
              --backup_password=test --content_data_path=~/.bbfs/var/server_content.data
  Run backup server (after content server is running):
  bbfs> ruby -Ilib bin/backup_server --monitoring_config_path=~/.bbfs/etc/backup_file_monitoring.yml

How to contribute to this project:
----------------------------------
A) Clone the project:
  > git@github.com:kolmanv/bbfs.git
  > cd bbfs
B) Master is sarced! Note this, this is important : you will never commit or merge into master branch.
  Master is master. You will use to get the changes, and only that.
C) Topic branch:
  So you want to fix a bug, add a great feature ? No problem, you create a new branch (because remember you will never
  commit or merge into your master branch!) :
  bbfs> git checkout -b feature-world-conquest
  Then you work in this branch, commit when you want, revert, etc. In short: you work. If you work on two different topics
  (add the world conquest feature and fix a bug) you create a topic branch for each. Don't forget to test your changes!
D) Fetch master: It's been a while you work on different topics. You want to test your changes with last changes :
  bbfs> git merge master
  Resolve conflicts if any, and try.
E) OK, it works. Now, you want to publish your changes to master branch. You go to github at
  https://github.com/kolmanv/bbfs, then for each new topic branches, you select it, then do a pull request. We'll review
  the code, eventually merge it in the master branch. Then you'll be able to drop your old topic branches.
Something went wrong with that request. Please try again.