Skip to content

Conversation

@gsakkis
Copy link
Contributor

@gsakkis gsakkis commented Sep 12, 2022

As mentioned in the #167 description, the TensorKind abstraction allows decoupling the TileDB array flavor (dense or sparse) from the generated tensor kind. This PR enables generating dense tensors from sparse TileDB arrays by passing tensor_kind=TensorKind.DENSE to the ArrayParams of a sparse array.

The actual implementation is contained in a single commit (56599f6) that:

  • generalizes the TensorSchemaFactories mapping to depend on the array flavor (in addition to the tensor kind).
  • adds SparseToDenseTensorSchema factory that maps sparse arrays generated by SparseTensorSchema to (dense) Numpy arrays.
  • makes MappedTensorSchema pickleable (required for num_workers>0)

@shortcut-integration
Copy link

This pull request has been linked to Shortcut Story #21303: Support reading dense tensors from sparse arrays.

@gsakkis gsakkis force-pushed the gsa/sc-21303/support-reading-dense-tensors-from-sparse-arrays branch from 631c053 to a48fa91 Compare September 14, 2022 17:06
@gsakkis gsakkis merged commit 3e8991e into master Sep 15, 2022
@gsakkis gsakkis deleted the gsa/sc-21303/support-reading-dense-tensors-from-sparse-arrays branch September 15, 2022 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants