Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TileDB support as external storage #227

Merged
merged 12 commits into from Feb 23, 2019

Conversation

@qinxuye
Copy link
Member

qinxuye commented Feb 22, 2019

What do these changes do?

Read array into Mars tensor from TileDB and write Mars tensor into TileDB are supported in this PR.

mt.fromtiledb is added to support read data from TileDB array, as uri can be obtained from a TileDB array, the uri parameter of fromtiledb can be a TileDB Array also, we will support it later on.

mt.totiledb and Tensor.totiledb are added to support write Mars tensor into tiledb.

Temporarily use pip to install tiledb till tiledb finishes TileDB-Inc/TileDB-Py#112.

Related issue number

Resolves #219

mars/tensor/execution/datasource.py Outdated Show resolved Hide resolved
mars/tensor/core.py Show resolved Hide resolved
mars/tensor/execution/datastore.py Show resolved Hide resolved
mars/tensor/execution/utils.py Outdated Show resolved Hide resolved
mars/tensor/expressions/datastore/__init__.py Outdated Show resolved Hide resolved
.travis.yml Show resolved Hide resolved
mars/tensor/execution/datastore.py Outdated Show resolved Hide resolved
@qinxuye qinxuye force-pushed the qinxuye:feature/tensor-tiledb-storage branch from edc4036 to d3173de Feb 22, 2019
qinxuye added 3 commits Feb 22, 2019
@wjsi
wjsi approved these changes Feb 23, 2019
@wjsi wjsi merged commit 00fc3a0 into mars-project:master Feb 23, 2019
4 checks passed
4 checks passed
Codacy/PR Quality Review Up to standards. A positive pull request.
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.2%) to 88.91%
Details
@qinxuye qinxuye deleted the qinxuye:feature/tensor-tiledb-storage branch Feb 23, 2019
qinxuye added a commit to qinxuye/mars that referenced this pull request Feb 28, 2019
* add tiledb relative expressions

* support execute fromtiledb

* support executeo totiledb

(cherry picked from commit 00fc3a0)
wjsi added a commit that referenced this pull request Feb 28, 2019
* Add TileDB support as external storage  (#227)

* fix .travis.yml
qinxuye added a commit to qinxuye/mars that referenced this pull request Mar 21, 2019
* add tiledb relative expressions

* support execute fromtiledb

* support executeo totiledb

(cherry picked from commit 00fc3a0)
qinxuye added a commit to qinxuye/mars that referenced this pull request Apr 8, 2019
* add tiledb relative expressions

* support execute fromtiledb

* support executeo totiledb

(cherry picked from commit 00fc3a0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.