Fix merge after #505 and #506

.travis.yml

@@ -2,6 +2,10 @@ sudo: false
 language: python
 python:
   - "3.5"
+addons:
+  apt:
+    packages:
+    - lftp
 # command to install dependencies, e.g. pip install -r requirements.txt --use-mirrors
 install: pip install -r requirements.txt
 # command to run tests, e.g. python setup.py test

build_docs.sh

@@ -1,8 +1,6 @@
 #!/bin/bash
 set -e
-mkdir -p standard/docs/field_definitions
 cd standard
-python schema/utils/make_field_definitions.py
 CODELIST_LANG=en python schema/utils/translate_codelists.py schema
 CODELIST_LANG=en python schema/utils/translate_codelists.py docs/en/extensions
 
@@ -28,7 +26,6 @@ python standard/schema/utils/translate_schema.py es fr
 
 cd standard
 for lang in es fr; do
-    SCHEMA_LANG=$lang python schema/utils/make_field_definitions.py
     CODELIST_LANG=$lang python schema/utils/translate_codelists.py schema
     CODELIST_LANG=$lang python schema/utils/translate_codelists.py docs/en/extensions
     # Create a symlink for the current language, so we can reference the

standard/.babel_schema

@@ -1,5 +1,3 @@
-[python: schema/utils/make_field_definitions.py]
-
 [jsonschema_text: schema/*.json]
 
 

standard/docs/en/schema/record_package.md

@@ -2,9 +2,9 @@
 
 The record package schema describes the container structure for publishing records. The contents of a record are based on the release schema. The package contains important meta-data.
 
-You can view an interactive version of the record schema below (requires javascript) or [download this version of the schema here](../../../record-package-schema.json).
+You can view an interactive version of the record schema below (requires javascript) or [download this version of the schema here](../../../../record-package-schema.json).
 
-A separate, auto-generated, [versioned release validation schema](../../../versioned-release-validation-schema.json) is provided for validating releases within fully versioned records.
+A separate, auto-generated, [versioned release validation schema](../../../../versioned-release-validation-schema.json) is provided for validating releases within fully versioned records.
 
 Click on schema elements to expand the tree, or use the '+' icon to expand all elements. Use { } to view the underlying schema for any section.
 

standard/docs/en/schema/release.md

@@ -1,6 +1,6 @@
 # Release Schema
 
-The release schema provides the authoritative definition of fields and their structure. You can view an interactive version of the release schema below (requires javascript) or [download this version of the schema here](../../../release-schema.json).
+The release schema provides the authoritative definition of fields and their structure. You can view an interactive version of the release schema below (requires javascript) or [download this version of the schema here](../../../../release-schema.json).
 
 The release schema is used to validate the contents of the ```releases``` array of [release packages](release_package.md), and the ```compiledReleases``` array of releases in a [record package](record_package.md).
 

standard/docs/en/schema/release_package.md

@@ -2,7 +2,7 @@
 
 The release package schema describes the container document for publishing releases. The package contains important meta-data.
 
-You can view an interactive version of the release schema below (requires javascript) or [download this version of the schema here](../../../release-package-schema.json).
+You can view an interactive version of the release schema below (requires javascript) or [download this version of the schema here](../../../../release-package-schema.json).
 
 Click on schema elements to expand the tree, or use the '+' icon to expand all elements. Use { } to view the underlying schema for any section.
 

standard/schema/tests/test_schema_integrity.py

@@ -1,3 +1,7 @@
+"""
+Ensures that `make_validation_schema.py` has been run.
+"""
+
 import json
 
 from .test_validate_schema import (
@@ -9,7 +13,7 @@
 
 
 def test_versioned_release_validation_schema_is_in_sync():
-    # Builds the validation schema and confirms its equal to the stored schema
+    # Builds the validation schema and confirms it's equal to the stored schema
     with open(VERSIONED_RELEASE_VALIDATION_SCHEMA_PATH, "r") as f:
         actual = json.loads(f.read())
 

standard/schema/tests/test_validate_packages.py

@@ -1,3 +1,7 @@
+"""
+Ensures that sample data validates correctly.
+"""
+
 import requests
 import requests_mock
 import pytest

standard/schema/tests/test_validate_schema.py

@@ -1,4 +1,10 @@
 #!/usr/bin/env python
+"""
+Validates release package, record package, release, and versioned release
+schemas against JSON Schema Draft 4, and ensures that the JSON contains no
+duplicate keys.
+"""
+
 import json
 from path import path
 from jsonschema import validate

standard/schema/utils/README.md

@@ -14,26 +14,4 @@ This can be created by running:
 
 There are tests set-up to ensure this has happened. 
 
-### make_field_definitions.py
 
-In order to display flattened version of the schema (primarily for the tables on the reference page of the documentation) we generate a set of field definition CSV files. 
-
-make_field_definition.py manually specifies each of the files to be generated.
-
-````
-python make_field_definition.py
-````
-
-should be run whenever the schema is updated so that the documentation reflects changes in field names, structures and descriptions.
-
-### get_codelists.py
-
-At present, codelist contents is managed through a Google Spreadsheet.
-
-The get_codelists.py script fetches each codelist, and stores it in the /schema/codelists directory as a CSV file. It also fetches the IATI Organisation Identifier codelist.
-
-````
-python get_codelists.py
-````
-
-should be run whenever codelists are changed.

standard/schema/utils/codelists_extract.py

@@ -1,3 +1,7 @@
+"""
+Babel extractor used in setup.py
+"""
+
 import csv
 import io
 

standard/schema/utils/jsonschema_extract.py

@@ -1,3 +1,7 @@
+"""
+Babel extractor used in setup.py
+"""
+
 import json
 
 

standard/schema/utils/make_validation_schema.py

@@ -1,4 +1,9 @@
 #!/usr/bin/env python
+
+"""
+See README for this script's documentation.
+"""
+
 import json
 from collections import OrderedDict
 import copy

standard/schema/utils/reformat_schema.py

@@ -1,3 +1,7 @@
+"""
+Makes whitespace consistent in release-schema.json.
+"""
+
 import json
 from collections import OrderedDict
 from os.path import abspath, dirname, join