Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parquet format for tensorflow-io #12

Closed
yongtang opened this issue Dec 9, 2018 · 0 comments · Fixed by #21
Closed

Support parquet format for tensorflow-io #12

yongtang opened this issue Dec 9, 2018 · 0 comments · Fixed by #21
Assignees

Comments

@yongtang
Copy link
Member

yongtang commented Dec 9, 2018

There are quite some interests to support parquet for tensorflow and a PR (tensorflow/tensorflow#19461) has already been approved.

With the expected deprecation of tf.contrib, it makes sense to move PR tensorflow/tensorflow#19461 to tensorflow-io.

At the moment the blocking issue with the related PR is the bazel version. The PR requires bazel 1.7.1+ to incorporate boost library while tensorflow CI still uses bazel 1.5.0. We will need to find out a way to work out the bazel version issue. See tensorflow/tensorflow#22449, tensorflow/tensorflow#22964 for details.

@yongtang yongtang self-assigned this Dec 9, 2018
yongtang pushed a commit that referenced this issue Jan 27, 2022
* feat: reading from bigtable (#2)

Implements reading from bigtable in a synchronous manner.

* feat: RowRange and RowSet API.

* feat: parallel read (#4)

In this pr we make the read methods accept a row_set reading only rows specified by the user.
We also add a parallel read, that leverages the sample_row_keys method to split work among workers.

* feat: version filters (#6)

This PR adds support for Bigtable version filters.

* feat: support for other data types (#5)

* fix: linter fixes (#8)

* feat docs (#9)

* fix: building on windows (#12)

* fix: refactor bigtable package to api folder (#14)

moved bigtable to tfensorflow_io.python.api

* fix: tests hanging (#30)

changed path to bigtable emulator and cbt in tests

moved arguments' initializations to the body of the function in bigtable_ops.py

 fixed interleaveFromRange of column filters when using only one column

* fix: temporarily disable macos tests (#32)

* disable tests on macos

Co-authored-by: Kajetan Boroszko <kajetan@unoperate.com>
Co-authored-by: Kajetan Boroszko <kajetan.boroszko@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant