Big brother file system (distributed file system)
Ruby C
Pull request Compare This branch is 917 commits behind bbfsdev:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
examples
lib
test
Gemfile
README
Rakefile
content_data.gemspec
content_server.gemspec
file_copy.gemspec
file_indexing.gemspec
file_monitoring.gemspec
file_monitoring_win32.gemspec_
file_utils.gemspec
log.gemspec
params.gemspec
reinitialize_all_bbfs_gems.rb
run_all_tests.rb

README

Big brother file system (distributed file system).
--------------------------------------------------

Backup system:
--------------
Content server monitors a set of directories (blobs/patterns).
The files in those directories are indexed (calculating their SHA1).
Each unique content is backed up to the remote (backup) server.
The backup server also monitors a set of directories (blobs/patterns).
The files in those directories are also indexed.
The content server copies new/changed files in the backup server.
The backup server index those new files too and send the content data (their SHA1 to the original server).

How to install and run:
-----------------------
A) Installation of BBFS development kit:
  1) Download and install git.
  2) Download and install ruby.
  3) Download and unpack and install DevKit (Windows). Note that after unpacking there are installation instructions at
     https://github.com/oneclick/rubyinstaller/wiki/Development-Kit.
  4) Mac (only) users will have to install XCode for gcc compiler.
  Run in command prompt (Windows users should run in Git Bash):
  5) git clone https://github.com/kolmanv/bbfs.git
  6) cd bbfs
  7) gem install bundler
  8) bundle install
  9) rake  # This will run all project tests and specs.
  10) You are done, you can start coding on the project.

B) (Optional) Recommended:
  1) Download and install smartgit.
  2) Download and install Ruby IDE (for example RubyMine).

C) Installing SFTP server backup server dependency (For now a must!):
  Install and run sftp server (for example set a 'test' user with 'test' password) on same computer
  which runs backup server with user working directory set to backup file monitoring dir,
  for example: C:/Users/kolman/backup_data
  Make sure the user have write permissions into the directory.
  Make sure the sftp server configured to support username/password authentication.

D) Preparing configuration files:
  Prepare two monitoring yml files. Each server, i.e., content server and backup server needs
  a monitoring configuration file.
  1) Configuration file for content server which is located in ~/.bbfs/etc/file_monitoring.yml:

    paths:
      - path: C:/Users/kolman/test_files  # <=== replace this directory with your local directory
        scan_period: 1
        stable_state: 5

  2) Configuration file for backup server which is located in ~/.bbfs/etc/backup_file_monitoring.yml:

    paths:
      - path: C:/Users/kolman/backup_data  # <=== replace this directory with your local directory
        scan_period: 1
        stable_state: 5

E) Note that for now parameter parsing is not yet available so just hard-code the parameters in the
  executables and run the local code:

  For content server edit the executable bin/content_server and add those parameters just before the
  run command (BBFS::ContentServer.run):
  ...
  # Replace values to suite your local configuration
  BBFS::Params.remote_server = '127.0.0.1'
  BBFS::Params.backup_username = 'test'
  BBFS::Params.backup_password = 'test'
  BBFS::Params.content_data_path = File.expand_path('~/.bbfs/var/server_content.data')
  ...

  For backup server edit the executable bin/backup_server and add the parameter just before the
  run command (BBFS::ContentServer.run_backup_server):
  ...
  # Replace value to suite your local configuration
  BBFS::Params.monitoring_config_path = File.expand_path('~/.bbfs/etc/backup_file_monitoring.yml')
  ...

F) Run content server:
  bbfs> ruby -Ilib bin/content_server
  Run backup server (after content server is running):
  bbfs> ruby -Ilib bin/backup_server

G) (Optional) Later when paramter parsing will be available we could do:
  Run content server:
  bbfs> ruby -Ilib bin/content_server --remote_server='127.0.0.1' --backup_username=test \
              --backup_password=test --content_data_path=~/.bbfs/var/server_content.data
  Run backup server (after content server is running):
  bbfs> ruby -Ilib bin/backup_server --monitoring_config_path=~/.bbfs/etc/backup_file_monitoring.yml

How to contribute to this project:
----------------------------------
A) Clone the project:
  > git@github.com:kolmanv/bbfs.git
  > cd bbfs
B) Master is sarced! Note this, this is important : you will never commit or merge into master branch.
  Master is master. You will use to get the changes, and only that.
C) Topic branch:
  So you want to fix a bug, add a great feature ? No problem, you create a new branch (because remember you will never
  commit or merge into your master branch!) :
  bbfs> git checkout -b feature-world-conquest
  Then you work in this branch, commit when you want, revert, etc. In short: you work. If you work on two different topics
  (add the world conquest feature and fix a bug) you create a topic branch for each. Don't forget to test your changes!
D) Fetch master: It's been a while you work on different topics. You want to test your changes with last changes :
  bbfs> git merge master
  Resolve conflicts if any, and try.
E) OK, it works. Now, you want to publish your changes to master branch. You go to github at
  https://github.com/kolmanv/bbfs, then for each new topic branches, you select it, then do a pull request. We'll review
  the code, eventually merge it in the master branch. Then you'll be able to drop your old topic branches.