Setup CI testing pipeline (#73)

* Adds some basic tests

* Updates testing pipeline

* Adds Flake8

* Add build status badge to README

.gitignore

@@ -1 +1,3 @@
-/venv
+**/**/*.pyc
+**/**/*pycache*
+/*venv

.travis.yml

@@ -0,0 +1,12 @@
+language: python
+python:
+  - "2.7"
+  - "3.6"
+
+install:
+  - pip install flake8
+  - pip install -r requirements.txt
+
+script:
+  - flake8 . --exclude "*venv*"
+  - pytest -v

README.md

@@ -6,7 +6,7 @@ title:		"Home"
 # Financial Regulatory (FIRE) Data Standard
 
 ---
-
+[![Build Status](https://travis-ci.org/SuadeLabs/fire.svg?branch=master)](https://travis-ci.org/SuadeLabs/fire)
 [![Project Website](https://img.shields.io/badge/website-fire-red.svg)][fire]
 [![Apache 2.0 License](https://img.shields.io/badge/LICENSE-Apache_2.0-yellow.svg)][license]
 [![Join the chat at https://gitter.im/SuadeLabs/fire](https://badges.gitter.im/SuadeLabs/fire.svg)][gitter]
@@ -37,7 +37,11 @@ The FIRE data standard is supported by the [European Commission][euc], the [Open
 Please see the [contributing guidelines][contributing] and [guiding principles][guiding-principles] if you would like to contribute to this project.
 
 ### Random FIRE Data Generator
-Included is a [random data generator][random-fire] which will generate data in line with the FIRE schema, but not necessarily realistic. (eg. You might get a loan with a balance of 10 but accrued interest of 1 million) 
+Included is a [random data generator][random-fire] which will generate data in line with the FIRE schema, but not necessarily realistic. (eg. You might get a loan with a balance of 10 but accrued interest of 1 million)
+
+### Testing
+You can run tests locally with via `./run_tests.sh`
+
 
 ---
 [fire]:         https://suade.org/fire/

documentation/properties/charge.md

@@ -0,0 +1,13 @@
+---
+layout:		property
+title:		"charge"
+schemas:	["loan"]
+---
+
+# charge
+
+---
+
+The **charge** indicates the lender's charge or lien-level over the collateral. It is an integer with the minimum value being 0. 
+
+1 indicates first charge, 2 second and so on. 0 indicates a combination of charge levels.

documentation/properties/income_amount.md → documentation/properties/ref_income_amount.md

@@ -1,10 +1,10 @@
 ---
 layout:		property
-title:		"income_amount"
+title:		"ref_income_amount"
 schemas:	[loan]
 ---
 
-# income_amount
+# ref_income_amount
 
 ---
 

documentation/properties/uk_funding_type.md

@@ -0,0 +1,20 @@
+---
+layout:		property
+title:		"uk_funding_type"
+schemas:	["account"]
+---
+
+# uk_funding_type
+
+---
+
+```bash
+├── a
+└── b
+```
+
+The **uk_funding_type** refers to the UK liquidity classification as per [BIPRU 12.5][bipru-12-5] and [BIPRU 12.6][bipru-12-6].
+
+---
+[bipru-12-5]: https://www.handbook.fca.org.uk/handbook/BIPRU/12/5.html
+[bipru-12-6]: https://www.handbook.fca.org.uk/handbook/BIPRU/12/6.html

run_tests.sh

@@ -0,0 +1,11 @@
+#! /bin/bash
+
+echo "===> Running Tests..."
+
+echo "===> Flake8 Running..."
+flake8 . --exclude "*venv*"
+echo "===> Flake8 Finished"
+
+echo "===> PyTest Running..."
+pytest -v
+echo "===> PyTest Finished"

tests/__init__.py

@@ -0,0 +1,14 @@
+import os
+
+
+HOME = os.path.realpath(".")
+SCHEMAS_DIR = os.path.join(HOME, 'v1-dev')
+DOCS_DIR = os.path.join(HOME, "documentation", "properties")
+
+_, _, filenames = next(os.walk(SCHEMAS_DIR), (None, None, []))
+SCHEMA_FILES = [f for f in filenames if f.endswith(".json")]
+SCHEMA_NAMES = [f.split(".json")[0] for f in SCHEMA_FILES]
+
+_, _, filenames = next(os.walk(DOCS_DIR), (None, None, []))
+DOC_FILES = [f for f in filenames if f.endswith(".md")]
+DOC_NAMES = [f.split(".md")[0] for f in DOC_FILES]

tests/test_schemas.py

@@ -0,0 +1,50 @@
+import json
+import os
+import unittest
+from . import (
+    DOC_NAMES,
+    SCHEMAS_DIR,
+    SCHEMA_FILES,
+    SCHEMA_NAMES,
+)
+
+
+class TestSchemas(unittest.TestCase):
+
+    def test_schemas_and_docs_found(self):
+        self.assertTrue(SCHEMA_NAMES)
+        self.assertTrue(DOC_NAMES)
+
+    def test_jsons_are_valid(self):
+        for schema_name in SCHEMA_FILES:
+            with open(os.path.join(SCHEMAS_DIR, schema_name)) as json_schema:
+                self.assertTrue(json.load(json_schema))
+
+    def test_property_has_docs(self):
+        """TODO: add documentation for adjustment schema"""
+        properties = set()
+        for schema_name in SCHEMA_FILES:
+            with open(os.path.join(SCHEMAS_DIR, schema_name)) as json_schema:
+                schema = json.load(json_schema)
+                if schema_name == "common.json":
+                    props = schema.keys()
+                else:
+                    props = schema["properties"].keys()
+
+                {properties.add(p) for p in props}
+
+        self.assertTrue(properties)
+
+        no_docs = []
+        for p in properties:
+            if p not in DOC_NAMES:
+                no_docs.append(p)
+
+        print("No documenation found for properties: {}".format(no_docs))
+        # TODO: Add docs!
+        # self.assertFalse(
+        #     no_docs,
+        #     "No documenation found for properties: {}".format(no_docs)
+        # )
+        # for doc in DOC_NAMES:
+        #     self.assertIn(doc, properties)

v1-dev/account.json

@@ -290,7 +290,7 @@
       "format": "date-time"
     },
     "uk_funding_type": {
-      "description": "Funding type calculated according to BIPRU 12.5",
+      "description": "Funding type calculated according to BIPRU 12.5/12.6",
       "type": "string",
       "enum": ["a", "b"]
     },