ocdskit.combine.merge
accepts aconvert_exceptions_to_warnings
argument.
ocdskit.combine.merge
accepts aforce_version
argument.
ocdskit.combine.merge
accepts aignore_version
argument.
ocdskit.util.Format
enumeration
ocdskit.util.detect_format
: Detect empty packages that set metadata fields but not areleases
orrecords
field.
ocdskit.util.is_linked_release
accepts amaximum_properties
argument (default 3).
schema-strict
: AddminItems
last, to match existing schema.
compile
: Prefix theocid
to a warning.
compile
: Skipnull
entries in thereleases
array of a release package.
Update documentation on PyPI.
New CLI options:
mapping-sheet
:--no-inherit-extension
- Drop support for Python 3.6 (end-of-life 2021-12-23).
ocdskit.mapping_sheet.mapping_sheet
works if a schema contains"deprecated": null
.ocdskit.mapping_sheet.mapping_sheet
works if--extension-field
is set to a value other than "extension".
ocdskit.mapping_sheet.mapping_sheet
works ifinclude_definitions=False
butbase_uri
is not provided.
- Move
ocdskit/cli/__main__.py
toocdskit/__main__.py
, to support thepython -m ocdskit
interface. - Move
ocdskit.cli.commands
toocdskit.commands
.
ocdskit.mapping_sheet.mapping_sheet
resolves$ref
properties ifinclude_definitions=False
.
ocdskit.mapping_sheet.mapping_sheet
returns columns and rows instead of writing to a file-like object.
New CLI options:
mapping-sheet
:--codelist
ocdskit.util.get_ocds_patch_tag
compile
omitspackages
from a record package for OCDS 1.2+.
compile
omitspackages
from a record package if empty.compile
raises an error if the OCDS version is not recognized, instead of failing silently.
mapping-sheet
correctly populates theextension
column for extension fields on OCDS objects withitems
properties that$ref
'erence OCDS definitions.
ocdskit.util.detect_format
accepts areader
keyword argument: for example,gzip.open
instead ofopen
.
validate
: Remove command. Instead, use lib-cove-ocds.tabulate
: Remove command. Instead, convert the JSON data to CSV format using Spoonbill or Flatten Tool, and then load the CSV files into your preferred database. See:- SQLite's .import command (see also sqlite-utils)
- PostgreSQL's COPY command
- MySQL's LOAD DATA command
- csvkit's csvsql command
convert-to-oc4ids
: Remove command. Instead, use oc4idskit's convert-from-ocds.ocdskit.oc4ids
: Remove module. Instead, use oc4idskit's transforms.
mapping-sheet
: Setdeprecated
on the extra rows for arrays.
schema-strict
: Don't add"uniqueItems": true
to coordinates fields.
- Add Python wheels distribution.
New CLI options:
mapping-sheet
:--language
upgrade
: If a party'sroles
field isn't set, no error occurs.upgrade
: If an organization reference has fewer fields than an organization, no warning is issued.
New library method:
ocdskit.util.detect_format
tabulate
: Supports linked releases and compiled releases.
upgrade
: If a party'sroles
field isn't a list of strings, no error occurs.
New CLI commands:
split-project-packages
upgrade
: If a party'sroles
field is a string, it is coerced to an array.
New CLI options:
mapping-sheet
:--no-deprecated
,--no-replace-refs
New CLI option:
schema-strict
:--check
New library method:
ocdskit.schema.add_validation_properties
convert-to-oc4ids
no longer errors if a party'sid
field isn't set.
ocdskit.util.get_ocds_minor_version
now supports records.
mapping-sheet
correctly populates theextension
column for extension fields on OCDS objects that$ref
'erence OCDS definitions.
mapping-sheet
: Theextension
column is now the name of the extension that introduced the JSON path, not the name of the extension that defined the field.
indent
respects--ascii
.tabulate
supports any OCDS data.
compile
no longer errors on Windows when using the SQLite backend.
detect-format
respects--root-path
.convert-to-oc4ids
omitssector
anddocuments
if empty.
schema-strict
accepts a filename as a positional argument, instead of a schema as standard input.schema-strict
adds constraints to all fields, not only required fields.
New CLI options:
combine-record-packages
:--version
combine-release-packages
:--version
compile
:--version
convert-to-oc4ids
:--version
package-records
:--version
package-releases
:--version
New library method:
ocdskit.util.is_compiled_release
compile
sets"version": "1.1"
even on OCDS 1.0 data (seeocdskit.combine.merge
).package-records
andpackage-releases
omit theextensions
field if empty (seeocdskit.combine.package_records
,ocdskit.combine.package_releases
).
convert-to-oc4ids
sets thepublishedDate
field, not thepublished_date
field.
New library method:
ocdskit.util.is_linked_release
combine-record-packages
andcombine-release-packages
warn if the"records"
and"releases"
fields aren't set (seeocdskit.combine.combine_record_packages
,ocdskit.combine.combine_release_packages
).
combine-record-packages
andcombine-release-packages
no longer error if the"records"
and"releases"
fields aren't set (seeocdskit.combine.combine_record_packages
,ocdskit.combine.combine_release_packages
).
convert-to-oc4ids
no longer errors.
New CLI commands:
convert-to-oc4ids
New library module:
ocdskit.oc4ids
compile
errors if anocid
field is missing from a release (seeocdskit.packager.AbstractBackend.add_release
).upgrade
upgrades records (seeocdskit.upgrade.upgrade_10_11
).
- Avoid exception when piping output to tools like
head
. package-records
,package-releases
: Use fast writer if--size
is set.echo
: Use fast writer (assuming--root-path
is set anytime input is too large).
New CLI options:
package-records
:--size
package-releases
:--size
New CLI commands:
echo
- Implement iterative JSON writer.
- Use
orjson
if available to improve performance of dumping/loading JSON, especially to/from SQL incompile
command (seeocdskit.packager
).
combine-record-packages
no longer duplicates release package URLs inpackages
field (seeocdskit.combine.combine_record_packages
).
New library module:
ocdskit.packager
CLI:
compile
accepts either release packages or individual releases (seeocdskit.combine.merge
).compile
is memory efficient if given a long list of inputs (seeocdskit.combine.merge
).
Library:
- Deprecate
ocdskit.combine.compile_release_packages
in favor ofocdskit.combine.merge
.
--linked-releases
no longer uses the same linked releases for all records (seeocdskit.packager.Packager.output_records
).
New library methods:
ocdskit.util.is_record
ocdskit.util.is_release
The internal methods ocdskit.util.json_load
and ocdskit.util.json_loads
are removed.
New CLI options:
combine-record-packages
:--fake
combine-release-packages
:--fake
compile
:--fake
package-records
:--fake
package-releases
:--fake
New CLI commands:
package-records
New library methods:
ocdskit.combine.package_records
mapping-sheet
: Improve documentation of--extension
and--extension-field
.
detect-format
correctly detects concatenated JSON, even if subsequent JSON values are non-OCDS values.
- Set missing package metadata to
""
instead ofnull
in CLI commands, to mirror library methods.
- Align the library methods
ocdskit.util.json_dump
andocdskit.util.json_dumps
.
upgrade
no longer errors if specific fields arenull
.upgrade
no longer errors on packages that haveparties
values withoutid
fields and that declare no version or a version of "1.0".
Command-line inputs can now be concatenated JSON or JSON arrays, not only line-delimited JSON.
New CLI commands:
detect-format
New CLI options:
package-releases
:--uri
--published-date
--publisher-name
--publisher-uri
--publisher-scheme
--publisher-uid
compile
:--publisher-name
--publisher-uri
--publisher-scheme
--publisher-uid
combine-record-packages
:--publisher-name
--publisher-uri
--publisher-scheme
--publisher-uid
combine-release-packages
:--publisher-name
--publisher-uri
--publisher-scheme
--publisher-uid
mapping-sheet
:--order-by
--infer-required
--extension
--extension-field
The --root-path
option is added to all OCDS commands.
New library methods:
ocdskit.combine.package_releases
ocdskit.combine.combine_record_packages
ocdskit.combine.combine_release_packages
ocdskit.combine.compile_release_packages
ocdskit.mapping_sheet.mapping_sheet
ocdskit.schema.get_schema_fields
- Backwards-incompatible:
ocdskit.upgrade.upgrade_10_10
,ocdskit.upgrade.upgrade_11_11
andocdskit.upgrade.upgrade_10_11
now return data, instead of only editing in-place. - Backwards-incompatible:
mapping-sheet
andschema-report
now read a file argument instead of standard input, to support schema that$ref
other schema. mapping-sheet
andschema-report
support schema from: Open Contracting for Infrastructure Data Standard (OC4IDS), Beneficial Ownership Data Standard (BODS), and Social Investment Data Lab Specification (SEDL).mapping-sheet
outputs:enum
values ofitems
enum
as “Enum:” instead of “Codelist:”pattern
as “Pattern:”
schema-strict
adds"uniqueItems": true
to all arrays, unless--no-unique-items
is set.- Use
https://
instead ofhttp://
forstandard.open-contracting.org
.
compile
merges extensions' schema into the release schema before merging releases.mapping-sheet
fills in the deprecated column if an object field uses$ref
.schema-strict
no longer errors if a required field uses$ref
.upgrade
no longer errors ifawards
orcontracts
isnull
.
New CLI options:
compile
:--schema
: You can create compiled releases and versioned releases using a specific release schema.--linked-releases
: You can have the record package use linked releases instead of full releases.--uri
,--published-date
: You can set theuri
andpublishedDate
of the record package.- If not set, these will be
null
instead of theuri
andpublishedDate
of the last package.
- If not set, these will be
combine-record-packages
:--uri
,--published-date
combine-release-packages
:--uri
,--published-date
New CLI commands:
upgrade
compile
raises an error if the release packages use different versions.compile
determines the version of the release schema to use if--schema
isn’t set.compile
,combine-record-packages
andcombine-release-packages
have a predictable field order.measure
is removed.
indent
prints an error if a path doesn’t exist.compile
,combine-record-packages
andcombine-release-packages
succeed if the requiredpublisher
field is missing.
New CLI options:
schema-report
:--no-codelists
,--no-definitions
,--min-occurrences
schema-report
reports definitions that can use a common$ref
in the versioned release schema.schema-report
reports open and closed codelists in CSV format.
New CLI options:
compile
:--package
,--versioned
New CLI commands:
package-releases
split-record-packages
split-release-packages
- Add helpful error messages if:
- the input is not line-delimited JSON data.
- the input to the
indent
command is not valid JSON.
- Change default behavior to print UTF-8 characters instead of escape sequences.
- Add
--ascii
option to print escape sequences instead of UTF-8 characters. - Rename base exception class from
ReportError
toOCDSKitError
.
New CLI options:
validate
:--check-urls
and--timeout
New CLI commands:
indent
schema-report
schema-strict
set-closed-codelist-enums
New CLI commands:
combine-record-packages
combine-release-packages
compile
mapping-sheet
measure
tabulate
validate