An Open-Source Ruby on Rails/AngularJS application for creating point-and-click data integrations between Open Data Kit, SurveyCTO, and Force.com
N.B.: Check out language-common, fn-lang, and our language packages on Github for where we've focused most of our recent work.
Find and connect technologies across the international development sector. openFn contains an "AppStore" for social impact and a point-and-click data integration tool to automate the flow of data between mobile survey tools, medical devices, clinic systems, sensors, ERPs, data-visualization applications, and other critical technologies being used by impact-first organizations.
rbenv install 2.1.5
rbenv local 2.1.5
gem install bundler
Copy the configuration defaults
cp config/database.yml.example config/database.yml
cp config/application.yml.template config/application.yml
Setting up trust authentication for postgres
Ensure you have the following line somewhere in your
# TYPE DATABASE USER ADDRESS METHOD local all all trust
pg_hba.conf file is located in
/etc/postgresql/9.3/main/pg_hba.conf on Ubuntu
Clone the staging database over
heroku pg:pull HEROKU_POSTGRESQL_PURPLE_URL openfn_dev --app staging-openfn
The application uses figaro to manage configuration.
Using the 12-factor application pattern, anything in the
will be seeded to the environment as Rails initializes.
There is a template located in
To get started, copy it:
$ cp config/application.yml.template config/application.yml
Using ENV variables certain aspects of the application can be configured or disabled.
GA_TRACKING_CODEEnabled Google Analytics by setting this to your GA tracking code.
SYNC_WITH_SALESFORCEWhen a user record is changed, set this to "true" to update the Salesforce backend.
The application uses sidekiq to perform background tasks.
The environment variable
REDIS_URL is required to be set for any environment
where redis is not available on
Sidekiq includes a UI for managing the queues, this is available at the
/sidekiq URL on the given server.
The management UI requires you to be logged in as admin user.
This appication has spring and guard installed. So running tests should be super easy and fast.
# Running via rspec $ spring rspec # Running guard $ guard
Building Mapping Fixtures
There is a rake task provided for this purpose:
$ rake db:fixtures:dump[x] # Where x is the ID of the mapping.
In order to load the fixtures, you will need to use namespacing.
The files are stored in
spec/fixtures/[mapping name]/[table name].yml.
So the following method is used to load namespaced fixtures:
fixtures "[mapping name]/mappings" set_fixture_class "[mapping name]/mappings" => Mapping
We use Github to track our features and bugs, so head over to the issues page.
Being a remote team, we encourage everyone to work out in the open. Part of that is being vocal, asking questions, giving feedback and generally being a good person.
When doing pull requests, there are a few critera that need to be met:
Up to date with develop On the theme of being nice, make sure your branch can be cleanly merged with develop.
We don't have super strong oppinions (yet) on rebasing vs. merging, but the least you can do is resolve your merge conflicts before we get your changes upstream.
#protip keep your branches lean, if you feel stuff getting crazy theres a good chance we underestimated the issue. Maybe it's 2 issues?
Tag your commits Since we work using Github Issues, it's super cool to be able to track a feature/bug in one place. So please use the #[issue number] convention on your commits.
This way everyone knows whats going on, and it helps keep you focussed on a single issue.
No mixed pull requests Seriously, shouldn't need to tell you this. Keep your branches on point and focussed on what the issue describes.
No one likes cherry picking. We will totally ask you to split your commits out if you mix code from other issues.
Clean commits Squashing commits is a great way to compact and condense your work, and allows a scattered commit trail to end up being clear and concise.
We all have to live with the code history, so try avoid 'oops, I forgot one thing' commits.
Any questions or feedback, pleasure raise an issue. We're all about improving things.
Ubuntu 14.10 Clean Slate Setup
sudo apt-get update sudo apt-get install libreadline-dev sudo apt-get install libpq-dev sudo apt-get rbenv rbenv install 2.1.5 rbenv global 2.1.5 gem install bundler gem install pg rbenv rehash bundle install
sudo apt-get update sudo apt-get -y install python-software-properties
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
lsb_release -c //note down the codename, and use it below:
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ codename-pgdg main" >> /etc/apt/sources.list.d/postgresql.list'
sudo apt-get update sudo apt-get install postgresql-9.3
sudo su postgres creatuser -d -s -r taylor
cd into the /etc/postgresql/9.3/main/pg_hba.conf directory and edit pg_hba.conf
Setting up trust authentication for postgres
Ensure you have the following line somewhere at you pg_hba.conf file.
TYPE DATABASE USER ADDRESS METHOD
local all all trust
psql -l //to test
sudo apt-get update sudo apt-get install build-essential sudo apt-get install tcl8.5
wget http://download.redis.io/releases/redis-2.8.9.tar.gz tar xzf redis-2.8.9.tar.gz cd redis-2.8.9
Run the recommended make test:
Finish up by running make install, which installs the program system-wide.
sudo make install
Once the program has been installed, Redis comes with a built in script that sets up Redis to run as a background daemon.
To access the script move into the utils directory:
From there, run the Ubuntu/Debian install script:
As the script runs, you can choose the default options by pressing enter. Once the script completes, the redis-server will be running in the background.
You can start and stop redis with these commands (the number depends on the port you set during the installation. 6379 is the default port setting):
sudo service redis_6379 start sudo service redis_6379 stop
You can then access the redis database by typing the following command:
You now have Redis installed and running. The prompt will look like this:
To set Redis to automatically start at boot, run:
sudo update-rc.d redis_6379 defaults
The next step is to take the newly generated SSH key and add it to your Github account. You want to copy and paste the output of the following command and paste it here.
Once you've done this, you can check and see if it worked:
ssh -T firstname.lastname@example.org
You should get a message like this:
Hi excid3! You've successfully authenticated, but GitHub does not provide shel
//Set up GIT credentials
cd git clone git://github.com/sstephenson/rbenv.git .rbenv echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc echo 'eval "$(rbenv init -)"' >> ~/.bashrc exec $SHELL
git clone git://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc exec $SHELL
//Install Heroku Toolbelt
wget -qO- https://toolbelt.heroku.com/install-ubuntu.sh | sh
Clone the staging database over heroku pg:pull HEROKU_POSTGRESQL_PURPLE_URL openfn_dev --app staging-openfn