Releases: bxparks/bigquery-schema-generator
Releases · bxparks/bigquery-schema-generator
0.5.1 - add '--sanitize_names' flag
- 0.5.1 (2019-06-17)
- Add
--sanitize_names
to convert invalid characters in column names and
to shorten them if too long. (See #33; thanks @jonwarghed).
- Add
0.5 - remove fields with unsolvable errors
- 0.5 (2019-06-06)
- Add input and output parameters to run() to allow the client code using
SchemaGenerator
to redirect the input and output files. (See #30). - Remove fields with incompatible types (or other errors) from the generated
schema, instead of picking the type of the first encounter. (See #31). - Improve internal data validation handling, reserving exceptions for
programming errors only.
- Add input and output parameters to run() to allow the client code using
0.4 - Support CSV files
- 0.4 (2019-03-06)
- Support CSV input files using
--input_format
flag. Preserve
the ordering of fields in the schema file for CSV. - Implement
--infer_mode
flag for CSV files so that fields that are
present in all input records are marked asREQUIRED
in the schema
(Thanks korotkevics@, see #28).
- Support CSV input files using
0.3.2 - add flag to force quoted values to be inferred as strings
- 0.3.2 (2019-02-24)
- Add
--quoted_values_are_strings
flag to force quoted values (integers,
floats, booleans) to be interpreted as aSTRING
. (Thanks de-code@,
see #22).
- Add
0.3.1 - support integers that overflow 64 bits; support 'UTC' suffix
0.3: support quoted values of BOOLEAN, INTEGER and FLOAT types
- 0.3 (2018-12-17)
- Tighten TIMESTAMP and DATE validation (thanks jtschichold@).
- Inspect the internals of STRING values to infer BOOLEAN, INTEGER or FLOAT
types (thanks jtschichold@). - Handle conversion of these string types when mixed with their non-quoted
equivalents, matching the conversion logic followed by 'bq load'.
0.2.1 - add benchmarks using anonymized data
- 0.2.1 (2018-07-18)
- Add
anonymizer.py
script to create anonymized data files for benchmarking. - Add benchmark numbers to README.md.
- Add
DEVELOPER.md
file to record how to upload to PyPI. - Fix some minor warnings from pylint3.
- Add
version 0.2.0: support DATE and TIME types
- 0.2.0 (2018-02-10)
- Add support for
DATE
andTIME
types. - Update type conversion rules to be more compatible with bq load.
- Allow
DATE
,TIME
andTIMESTAMP
to gracefully degrade toSTRING
. - Allow type conversions of elements within arrays
(e.g. array ofINTEGER
andFLOAT
, or array of mixedDATE
,TIME, or
TIMESTAMP` elements) - Better detection of invalid values (e.g. arrays of arrays).
- Allow
- Add support for
Fix 'generate-schema' to pass along flags
I forgot to tag this when I pushed it out to PyPI. Already on PyPI.
Update README.md regarding installation under Ubuntu and MacOS.
v0.1.5 Increment version to 0.1.5.