A data collection and monitoring tool that works anywhere.
Switch branches/tags
Clone or download
Latest commit 594428b Sep 14, 2018
Permalink
Failed to load latest commit information.
.github Added checklist Nov 14, 2016
Config [#2395] Add java8 to the appengine-web.xml template file Jul 23, 2018
Dashboard Merge pull request #2841 from akvo/issue/2823-remove-instance-count Sep 12, 2018
GAE Merge pull request #2851 from akvo/issue/2639-form-export-full Sep 14, 2018
ci [#2756] Copy stripped off jar into staging folder Jul 3, 2018
scripts [#2846] Use `GAE/target` folder to build remote api scripts Sep 8, 2018
tests [#2395] Use 1.8 source version, enable java8 runtime Jun 25, 2018
.dockerignore [#2711] ignore tmp folder Jun 19, 2018
.env [#2715] Setting appengine.fullscan.seconds to 30 secs Jun 19, 2018
.gitattributes [#2807] Tell git to always use LF for shell scripts Aug 3, 2018
.gitignore [#2673] Deploy using AppCfg using uat1 service account Jun 5, 2018
.travis.yml Merge branch 'develop' into issue-2558/exporterapplet-jar Jun 21, 2018
AUTHORS.txt [#1914] Update authors list Jan 19, 2017
Dockerfile [#2704] Use fuzzy version for apk packages Jul 2, 2018
LICENSE.md [#1845]Only need the GNU license here. Jan 12, 2017
README.md [#2748] Document workflow when working with Java Report classes Jul 2, 2018
RELEASE_NOTES.md [#2854] Add release name Sep 12, 2018
dev-deploy.sh [#2673] Use AppCfg for dev deployment Jun 5, 2018
docker-compose.together.yml [#2715] Setting GAE_FULL_SCAN_SECS in "together" dev env Jul 2, 2018
docker-compose.yml [#2715] Setting appengine.fullscan.seconds to 30 secs Jun 19, 2018
maven-ci-settings.xml [#2558] Deploying to clojars Mar 27, 2018
switch_tenant.sh [#2628] Use the gcloud maven plugin and deploy cron, index and queues May 16, 2018
switch_to_local_tenant.sh [#2553] Using scripts to change tenants instead of maven Apr 16, 2018

README.md

Build Status

Akvo Flow is a tool for collecting, evaluating and displaying of geographically referenced data. It is composed of an android mobile app and an online web-based platform. This repository contains code for the web-based platform that comprises a backend engine and a dashboard user interface. Alongside the dashboard and mobile apps, is a data import and export component.

You can read more about the motivation and history of Akvo Flow as well as its place in the platform of tools created by Akvo.

Development

Start

To run Flow:

docker-compose up --build -d && docker-compose logs -f

Flow should be running here and you can login with user "akvo.flow.user.test@gmail.com"

You also have the appengine admin console here

The Docker-Compose environment will have:

  1. A fake s3 server.
  2. A fake flow services, that always returns 200.
  3. The dev environment.

It downloads a prepopulated database from https://s3-eu-west-1.amazonaws.com/akvoflow/test-data/local_db.bin if there is none.

It will use the dev appengine-web.xml if there is none.

See the devserver.sh for more details.

UI development

Once Flow is started, any changes in the Dashboard folder will trigger a build of the UI code, except for the ClojureScript bit.

If you are going to work on the ClojureScript side, you can run a watch process with:

docker-compose exec -u akvo akvo-flow /bin/bash -c "cd Dashboard/app/cljs && lein watch"

Or run the commands from a terminal inside the container:

docker-compose exec -u akvo akvo-flow /bin/bash
cd Dashboard/app/cljs
lein watch

Backend development

The appengine dev server is started in debug mode, listening in port 5005.

It is expected that your IDE understand the Maven pom and that it compiles the Java classes to the right place.

After you IDE compiles the classes, the dev server should refresh the webcontext. Due to some Mac performance issues with Docker, the refresh interval is 20 secs instead of the default 5 secs. You can change the scan interval by setting the property GAE_FULL_SCAN_SECS to the desired value. You can also trigger a reload hitting the reload url.

If you need to restart the server:

docker-compose exec -u akvo -d akvo-flow /bin/bash -c "cd GAE && mvn appengine:stop appengine:run >> ./target/build.log"

Remember that you also can run those commands from a terminal inside the container.

Stop

docker-compose stop

Tear down and reset

docker-compose down
rm -rf GAE/target

Test instances and Manual deployments

If you want to use a configuration different from the dev one, checkout the akvo-flow-server-config directory into .. and run:

switch_tenant.sh akvoflowsandbox

To switch back to the dev setup:

swith_to_local_tenant.sh

To deploy the current state of the docker container to whatever tenant you last switched to, run:

docker-compose exec -u akvo akvo-flow ./dev-deploy.sh

Running Flow Services and Flow together locally

If you want to run both Flow and Flow Services locally and talking to each other, you will need add some config to your /etc/hosts:

127.0.0.1 services.akvoflow.local akvoflow.local

Then run:

docker-compose -f docker-compose.together.yml up --build -d

You will need to access Flow using the url http://akvoflow.local:8888/

Then read the Flow Services documentation for the Flow Services specific instructions.

The DNS alias is required because the UI is sending to Flow Services the baseUrl of the Flow service, which Flow Services needs to resolve to the Flow container. The way Docker works, this baseUrl cannot be "localhost", as "localhost" for the Flow Service container is itself. Adding a DNS entry allows for one level of indirection where "akvoflow.local" will be resolved to "127.0.0.1" for the Browser, while it resolves to the flow container for the flow-services container.

Changing report Java classes

Once you have both flow and flow services running, if you are making changes to the Java report classes used in flow-services, you will need to run:

docker-compose exec -u akvo akvo-flow /bin/bash -c "cd GAE && mvn install"

To package and install the Flow jar in your local maven repository. Then you can use this Flow jar as a dependency in your local Flow Services dev environment. See Flow Services README for how to set it up.


 

YourKit supports open source projects with its full-featured Java Profiler. YourKit, LLC is the creator of YourKit Java Profiler and YourKit .NET Profiler, innovative and intelligent tools for profiling Java and .NET applications.

YourKit has offered an open source license to the Akvo FLOW team for the profiling of backend to improve its performance.

BrowserStack is supporting Akvo, allowing us to use their service and infrastructure to test the code in this repository. Thank you for supporting the open source community!