Enabling the population and distribution via API of UK Customs tariffs and duties
Ruby Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
app
bin
config
data
db
lib
log
public
spec
vendor/plugins
.cfignore
.codeclimate.yml
.env.test
.foreman
.gitignore
.rspec
.rubocop.yml
.ruby-version
.sample.env
.snyk
CHANGELOG.md
CONTRIBUTING.md
Gemfile
Gemfile.lock
LICENCE.txt
Procfile
README.md
Rakefile
circle.yml
config.ru
prod2dev.sed

README.md

CircleCI Code Climate

Trade Tariff Backend

Now maintained at https://github.com/bitzesty/trade-tariff-backend

The API back-end for:

Other related projects:

Development

Dependencies

  • Ruby
  • Postgresql
  • ElasticSearch
  • Redis
  • Memcached (production)

Setup

  1. Setup your environment.

    bin/setup

  2. Update .env file with valid data.

  3. Start Foreman.

    foreman start

  4. Verify that the app is up and running.

    open http://localhost:3018/healthcheck

Load database

Check out wiki article on the subject, or get a recent database snapshot.

Performing daily updates

These are run hourly by a background worker UpdatesSynchronizerWorker.

Sync process

  • checking failures (check tariff_synchronizer.rb) - if any of updates failed in the past, sync process will not proceed
  • downloading missing files up to Date.today (check base_update.rb and download methods in taric_update.rb and chief_update.rb)
  • applying downloaded files (applying measures, etc. TARIC first, then CHIEF)

Updates are performed in portions and protected by redis lock (see TariffSynchronizer#apply).

BaseUpdate#apply is responsible for most of the logging/checking job and running import! methods located in Taric/ChiefUpdate classes. Then it runs TaricImporter and ChiefImporter to parse and store xml/csv files.

Whole process is quite similar for both TARIC and CHIEF, but CHIEF updates undergo a tranformation transformation process to convert them into a TARIC format. Check ChiefTransformer class for more info (and ChiefUpdate#import!).

In case of any errors, changes (per single update) are roll-backed and record itself is marked as failed. The sync would need to be rerun after a rollback.

Deployment

We deploy to cloud foundry, so you need to have the CLI installed, and the following cf plugin installed:

Download the plugin for your os: https://github.com/contraband/autopilot/releases

chmod +x autopilot-(YOUR_OS)
cf install-plugin autopilot-(YOUR_OS)

Set the following ENV variables:

  • CF_USER
  • CF_PASSWORD
  • CF_ORG
  • CF_SPACE
  • CF_APP
  • CF_APP_WORKER
  • HEALTHCHECK_URL
  • SLACK_CHANNEL
  • SLACK_WEBHOOK

Then run

./bin/deploy

NB: In the newer Diego architecture from CloudFoundry, no-route skips creating and binding a route for the app, but does not specify which type of health check to perform. If your app does not listen on a port, for example the sidekiq worker, then it does not satisfy the port-based health check and Cloud Foundry marks it as crashed. To prevent this, disable the port-based health check with cf set-health-check APP_NAME none.

Notes

  • When writing validators in app/validators please run the rake task audit:verify which runs the validator against existing data.

Contributing

Please check out the Contributing guide

Licence

Trade Tariff is licenced under the MIT licence