This repository has been archived by the owner on Jul 31, 2023. It is now read-only.
Feature/convert and load #55
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also refactor create_tfrecords to convert.
cfezequiel
force-pushed
the
feature/convert-and-load
branch
from
ed27ee0
to
7f3f480
Compare
mbernico
reviewed
Oct 30, 2020
mbernico
reviewed
Oct 30, 2020
mbernico
reviewed
Oct 30, 2020
mbernico
reviewed
Oct 30, 2020
cfezequiel
force-pushed
the
feature/convert-and-load
branch
from
a33bd51
to
68d94e2
Compare
mbernico
approved these changes
Oct 30, 2020
mbernico
added a commit
that referenced
this pull request
Nov 4, 2020
* Update check_tfrecords to use new dataset load function. * Add tfrecord_dir to create_tfrecords output. * Restructure test image directory to match expected format. * Feature/dataclass (#44) * Added data classes for types. * Checking in progress. * Checking in more changes. * Converted types to classes and refactored schema into OO pattern. * Changed OrderedDict import to support py3.6. * Changed OrderedDict import to support py3.6. * Updated setup.py for version. * fixing setup.py * Patched requirements and setup. * Addressed comments in code review. * Addressed code comments round 2. * refactored IMAGE_CSV_SCHEMA. * Merged check_test.py from dev Co-authored-by: Carlos Ezequiel <cezequiel@google.com> * Feature/structured data tutorial (#45) * Converted types to classes and refactored schema into OO pattern. * Add tutorial on structured data conversion. This changes types.FloatInput to use tf.float32 for its feature_spec attribute to address potential incompatibility with using tf.float64 type in TensorFlow Transform. Co-authored-by: Mike Bernico <mikebernico@google.com> * Update structured data tutorial to use output dir. * Clarify need for proper header when using create_tfrecords. Fixes #47. * Clean up README and update image directory notebook. * Feature/test image dir (#49) * Restructure test image directory to match expected format. * Clean up README and update image directory notebook. * Fix minor issues * Add an explicit error message for missing train split * Configure automated tests for Jupyter notebooks. * Add convert_and_load function. Also refactor create_tfrecords to convert. * Refactor check and common modules to utils. * Add test targets for py files and notebooks. * Feature/convert and load (#55) * Add convert_and_load function. Also refactor create_tfrecords to convert. * Refactor check and common modules to utils. * Add test targets for py files and notebooks. * Update version in setup.py and release notes. * Fix issues with GCS path parsing. Co-authored-by: Mike Bernico <mikebernico@google.com> Co-authored-by: Sergii Khomenko <khomenko@brainscode.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implements
convert_and_loadfunction and updates other high-level APIs:create_tfrecords->convertcheck_tfrecords->inspectFixes #54
Type of change
Please delete options that are not relevant.
Checklist
Please delete options that are not relevant.
make pylintand code is rated 10/10make testandmake testnband all tests passdevbranch