This repository has been archived by the owner on Jul 31, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 36
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Update check_tfrecords to use new dataset load function. * Add tfrecord_dir to create_tfrecords output. * Restructure test image directory to match expected format. * Feature/dataclass (#44) * Added data classes for types. * Checking in progress. * Checking in more changes. * Converted types to classes and refactored schema into OO pattern. * Changed OrderedDict import to support py3.6. * Changed OrderedDict import to support py3.6. * Updated setup.py for version. * fixing setup.py * Patched requirements and setup. * Addressed comments in code review. * Addressed code comments round 2. * refactored IMAGE_CSV_SCHEMA. * Merged check_test.py from dev Co-authored-by: Carlos Ezequiel <cezequiel@google.com> * Feature/structured data tutorial (#45) * Converted types to classes and refactored schema into OO pattern. * Add tutorial on structured data conversion. This changes types.FloatInput to use tf.float32 for its feature_spec attribute to address potential incompatibility with using tf.float64 type in TensorFlow Transform. Co-authored-by: Mike Bernico <mikebernico@google.com> * Update structured data tutorial to use output dir. * Clarify need for proper header when using create_tfrecords. Fixes #47. * Clean up README and update image directory notebook. * Feature/test image dir (#49) * Restructure test image directory to match expected format. * Clean up README and update image directory notebook. * Fix minor issues * Add an explicit error message for missing train split * Configure automated tests for Jupyter notebooks. * Add convert_and_load function. Also refactor create_tfrecords to convert. * Refactor check and common modules to utils. * Add test targets for py files and notebooks. * Feature/convert and load (#55) * Add convert_and_load function. Also refactor create_tfrecords to convert. * Refactor check and common modules to utils. * Add test targets for py files and notebooks. * Update version in setup.py and release notes. * Fix issues with GCS path parsing. Co-authored-by: Mike Bernico <mikebernico@google.com> Co-authored-by: Sergii Khomenko <khomenko@brainscode.com>
- Loading branch information
1 parent
747dabe
commit f4650ca
Showing
38 changed files
with
2,333 additions
and
742 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,6 @@ | ||
build/ | ||
dist/ | ||
tfrecorder.egg-info | ||
.idea/ | ||
.ipynb_checkpoints/ | ||
.vscode/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,17 @@ | ||
all: init test pylint | ||
all: init testnb test pylint | ||
|
||
init: | ||
pip install -r requirements.txt | ||
|
||
test: | ||
test: test-nb test-py | ||
|
||
test-py: | ||
nosetests --with-coverage -v --cover-package=tfrecorder | ||
|
||
test-nb: | ||
ls -1 samples/*.ipynb | grep -v '^.*Dataflow.ipynb' | xargs py.test --nbval-lax -p no:python | ||
|
||
pylint: | ||
pylint tfrecorder | ||
pylint -j 0 tfrecorder | ||
|
||
.PHONY: all init test pylint | ||
.PHONY: all init test pylint |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.