Skip to content

Commit

Permalink
Field Lists: Apply suggestions from code review
Browse files Browse the repository at this point in the history
Co-authored-by: James McKinney <26463+jpmckinney@users.noreply.github.com>
  • Loading branch information
kindly and jpmckinney committed Dec 9, 2020
1 parent 3cac61f commit 02936c5
Show file tree
Hide file tree
Showing 25 changed files with 41 additions and 41 deletions.
6 changes: 3 additions & 3 deletions docs/cli/use.rst
Expand Up @@ -83,21 +83,21 @@ Use this option if:
Create array of all paths for each row in each summary table
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``--field_lists`` option adds a new field to each summary table called ``field_list`` which contains an array of all JSON paths (excluding index postiions) contained in the object that the row represents. So for example in the ``awards_summary`` table it will contain the paths of every ``award`` object.
The ``--field_lists`` option adds a ``field_list`` column to each summary table, which contains a JSON array of all JSON paths (excluding array indices) in the object that the row describes. For example, a ``field_list`` value in the ``awards_summary`` table will contain the JSON paths in an award object.

.. code-block:: bash
./manage.py add 123 "The note" --field-lists
This can be useful when asking questions about the existance of multiple child fields in a simple way. For example, if you want to know how many awards have at least one ``document`` whith an ``id`` and at least one ``item`` with an ``id``, you could run the following.
This can be used to check for the presence of multiple fields. For example, to count the number of awards that have at least one document with an ``id`` and at least one item with an ``id``, run:

```

.. code-block:: sql
SELECT count(*) FROM view_data_collection_1.awards_summary WHERE field_list @> '{documents/id, items/id}';
The ``@>`` symbol stands for 'contains' and the right hand argument is a string representation of an array.
The ``@>`` operator tests whether the left ARRAY value contains the right ARRAY values.

.. _remove:

Expand Down
2 changes: 1 addition & 1 deletion docs/definitions/award_documents_summary.csv
Expand Up @@ -10,4 +10,4 @@ data_id,integer,"id for the ""data"" table in Kingfisher that holds the original
document,jsonb,JSONB of the document
documenttype,text,`documentType` field from the document object
format,text,`format` field from the document object
field_list,ARRAY,Array of paths for the document object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the document object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/award_items_summary.csv
Expand Up @@ -15,4 +15,4 @@ unit_currency,text,`currency` from the unit/value object
item_classification,text,Concatenation of classification/scheme and classification/id
item_additionalidentifiers_ids,jsonb,JSONB list of the concatenation of additionalClassification/scheme and additionalClassification/id
additional_classification_count,integer,Count of additional classifications
field_list,ARRAY,Array of paths for the item object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the item object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/award_suppliers_summary.csv
Expand Up @@ -16,4 +16,4 @@ supplier_additionalidentifiers_count,integer,Count of additional identifiers
link_to_parties,integer,"Does this buyer link to a party in the parties array using the `id` field from buyer object linking to the `id` field in a party object? If this is true then 1, otherwise 0"
link_with_role,integer,If there is a link does the parties object have `suppliers` in its roles list? If it does then 1 otherwise 0
party_index,bigint,Position of the party in the parties array
field_list,ARRAY,Array of paths for the supplier object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the supplier object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/awards_summary.csv
Expand Up @@ -22,4 +22,4 @@ documents_count,bigint,Number of documents in documents array
documenttype_counts,jsonb,JSONB object with the keys as unique documentTypes and the values as count of the appearances of that `documentType` in the `documents` array
items_count,bigint,Count of items
award,jsonb,JSONB of award object
field_list,ARRAY,Array of paths for the award object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the award object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/buyer_summary.csv
Expand Up @@ -15,4 +15,4 @@ buyer_additionalidentifiers_count,integer,Count of additional identifiers
link_to_parties,integer,"Does this buyer link to a party in the parties array using the `id` field from buyer object linking to the `id` field in a party object? If this is true then 1, otherwise 0"
link_with_role,integer,If there is a link does the parties object have `buyer` in its roles list? If it does then 1 otherwise 0
party_index,bigint,If there is a link what is the index of the party in the `parties` array then this can be used for joining to the `parties_summary` table
field_list,ARRAY,Array of paths for the buyer object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the buyer object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/contract_documents_summary.csv
Expand Up @@ -10,4 +10,4 @@ data_id,integer,"id for the ""data"" table in Kingfisher that holds the original
document,jsonb,JSONB of the document
documenttype,text,`documentType` field from the document object
format,text,`format` field from the document object
field_list,ARRAY,Array of paths for the document object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the document object, excluding array indices.This column is only available if the --field-lists option was used."
Expand Up @@ -10,4 +10,4 @@ data_id,integer,"id for the ""data"" table in Kingfisher that holds the original
document,jsonb,JSONB of the document
documenttype,text,`documentType` field from the document object
format,text,`format` field from the document object
field_list,ARRAY,Array of paths for the document object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the document object, excluding array indices.This column is only available if the --field-lists option was used."
Expand Up @@ -11,4 +11,4 @@ milestone,jsonb,JSONB of milestone object
type,text,`type` from milestone object
code,text,`code` from milestone object
status,text,`status` from milestone object
field_list,ARRAY,Array of paths for the milestone object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the milestone object, excluding array indices.This column is only available if the --field-lists option was used."
Expand Up @@ -10,4 +10,4 @@ data_id,integer,"id for the ""data"" table in Kingfisher that holds the original
transaction_amount,numeric,`amount` field from the value object or the deprecated amount object
transaction_currency,text,`currency` field from the value object or the deprecated amount object
transaction,jsonb,JSONB of transaction object
field_list,ARRAY,Array of paths for the transaction object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the transaction object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/contract_items_summary.csv
Expand Up @@ -15,4 +15,4 @@ unit_currency,text,`currency` from the unit/value object
item_classification,text,Concatenation of classification/scheme and classification/id
item_additionalidentifiers_ids,jsonb,JSONB list of the concatenation of additionalClassification/scheme and additionalClassification/id
additional_classification_count,integer,Count of additional classifications
field_list,ARRAY,Array of paths for the item object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the item object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/contract_milestones_summary.csv
Expand Up @@ -11,4 +11,4 @@ milestone,jsonb,JSONB of milestone object
type,text,`type` from milestone object
code,text,`code` from milestone object
status,text,`status` from milestone object
field_list,ARRAY,Array of paths for the milestone object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the milestone object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/contracts_summary.csv
Expand Up @@ -29,4 +29,4 @@ implementation_documenttype_counts,jsonb,JSONB object with the keys as unique do
implementation_milestones_count,bigint,Number of documents in documents array
implementation_milestonetype_counts,jsonb,JSONB object with the keys as unique milestoneTypes and the values as count of the appearances of that `milestoneType` in the `milestone` array
contract,jsonb,JSONB of contract object
field_list,ARRAY,Array of paths for the contract object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the contract object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/parties_summary.csv
Expand Up @@ -13,4 +13,4 @@ unique_identifier_attempt,text,"The `id` from party object if it exists, otherwi
parties_additionalidentifiers_ids,jsonb,JSONB list of the concatenation of scheme and id of all additionalIdentifier objects
parties_additionalidentifiers_count,integer,Count of additional identifiers
party,jsonb,JSONB of party object
field_list,ARRAY,Array of paths for the party object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the party object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/planning_documents_summary.csv
Expand Up @@ -9,4 +9,4 @@ data_id,integer,"id for the ""data"" table in Kingfisher that holds the original
document,jsonb,JSONB of the document
documenttype,text,`documentType` field from the document object
format,text,`format` field from the document object
field_list,ARRAY,Array of paths for the document object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the document object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/planning_milestones_summary.csv
Expand Up @@ -10,4 +10,4 @@ milestone,jsonb,JSONB of milestone object
type,text,`type` from milestone object
code,text,`code` from milestone object
status,text,`status` from milestone object
field_list,ARRAY,Array of paths for the milestone object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the milestone object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/planning_summary.csv
Expand Up @@ -13,4 +13,4 @@ documenttype_counts,jsonb,JSONB object with the keys as unique documentTypes and
milestones_count,bigint,Count of milestones
milestonetype_counts,jsonb,JSONB object with the keys as unique milestoneTypes and the values as a count of the appearances of that `milestoneType` in the `milestones` array
planning,jsonb,JSONB of planning object
field_list,ARRAY,Array of paths for the planning object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the planning object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/procuringEntity_summary.csv
Expand Up @@ -14,4 +14,4 @@ procuringentity_additionalidentifiers_count,integer,Count of additional identifi
link_to_parties,integer,"Does this procuringEntity link to a party in the parties array using the `id` field from buyer object linking to the `id` field in a party object? If this is true then 1, otherwise 0"
link_with_role,integer,If there is a link does the parties object have `procuringEntity` in its roles list? If it does then 1 otherwise 0
party_index,bigint,If there is a link what is the index of the party in the `parties` array then this can be used for joining to the `parties_summary` table
field_list,ARRAY,Array of paths for the procuringentity object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the procuringentity object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/release_summary.csv
Expand Up @@ -59,4 +59,4 @@ release_check,jsonb,JSONB of Data Review Tool output which includes validation e
release_check11,jsonb,JSONB of Data Review Tool output run against 1.1 version of OCDS even if the data is from 1.0
record_check,jsonb,JSONB of Data Review Tool output which includes validation errors and additional field information
record_check11,jsonb,JSONB of Data Review Tool output run against 1.1 version of OCDS even if the data is from 1.0
field_list,ARRAY,Array of paths for the release object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the release object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/tender_documents_summary.csv
Expand Up @@ -9,4 +9,4 @@ data_id,integer,"id for the ""data"" table in Kingfisher that holds the original
document,jsonb,JSONB of the document
documenttype,text,`documentType` field from the document object
format,text,`format` field from the document object
field_list,ARRAY,Array of paths for the document object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the document object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/tender_items_summary.csv
Expand Up @@ -14,4 +14,4 @@ unit_currency,text,`currency` from the unit/value object
item_classification,text,Concatenation of classification/scheme and classification/id
item_additionalidentifiers_ids,jsonb,JSONB list of the concatenation of additionalClassification/scheme and additionalClassification/id
additional_classification_count,integer,Count of additional classifications
field_list,ARRAY,Array of paths for the item object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the item object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/tender_milestones_summary.csv
Expand Up @@ -10,4 +10,4 @@ milestone,jsonb,JSONB of milestone object
type,text,`type` from milestone object
code,text,`code` from milestone object
status,text,`status` from milestone object
field_list,ARRAY,Array of paths for the milestone object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the milestone object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/tender_summary.csv
Expand Up @@ -44,4 +44,4 @@ milestones_count,bigint,Count of milestones
milestonetype_counts,jsonb,JSONB object with the keys as unique milestoneTypes and the values as a count of the appearances of that `milestoneType` in the `milestones` array
items_count,bigint,Count of items
tender,jsonb,JSONB of tender object
field_list,ARRAY,Array of paths for the tender object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the tender object, excluding array indices.This column is only available if the --field-lists option was used."
2 changes: 1 addition & 1 deletion docs/definitions/tenderers_summary.csv
Expand Up @@ -15,4 +15,4 @@ tenderer_additionalidentifiers_count,integer,Count of additional identifiers
link_to_parties,integer,"Does this tenderer link to a party in the parties array using the `id` field from buyer object linking to the `id` field in a party object? If this is true then 1, otherwise 0"
link_with_role,integer,If there is a link does the parties object have `tenderers` in its roles list? If it does then 1 otherwise 0
party_index,bigint,If there is a link what is the index of the party in the `parties` array. This can be used for joining to the `parties_summary` table
field_list,ARRAY,Array of paths for the tenderer object. Paths excluding index numbers for arrays.This field will only appear if --field-lists option is specified
field_list,ARRAY,"Array of JSON paths in the tenderer object, excluding array indices.This column is only available if the --field-lists option was used."

0 comments on commit 02936c5

Please sign in to comment.