Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Python] Add a FixedSizeTensorScalar class #37484

Closed
wjones127 opened this issue Aug 31, 2023 · 4 comments · Fixed by #37533
Closed

[Python] Add a FixedSizeTensorScalar class #37484

wjones127 opened this issue Aug 31, 2023 · 4 comments · Fixed by #37533
Assignees
@rok
Labels
Component: Python Type: enhancement
Milestone
16.0.0

Comments

@wjones127
Copy link
Member

Describe the enhancement requested

When you get a scalar off of a FixedSizeTensorArray, you get back a generic ExtensionTensor. There's not an easy way to get the tensor contained back as a numpy ndarray. It would be nice to have:

def FixedSizeTensorScalar(pa.ExtensionScalar):
    def to_numpy_ndarray(): ...

Then one could do:

tensor_array[i].to_numpy_ndarray()

As a workaround, I currently do:

tensor_array.to_numpy_ndarray()[i]

Which isn't too bad.

Component(s)

Python
@jorisvandenbossche
Copy link
Member

Sounds good!

@rok
Copy link
Member

rok commented Aug 31, 2023 •
edited
Loading

I can grab this one.

Also, another workaround would be:

tensor_array.slice(i, i+1).to_numpy_ndarray()[0]

@wjones127
Copy link
Member Author

Also, another workaround would be:
tensor_array.slice(i, i+1).to_numpy_ndarray()[0]

Yes, but that doesn't work in 12.0.0. There is a bug that was fixed in 13.0.0 where to_numpy_ndarray() didn't work properly on sliced tensors.

@rok
Copy link
Member

rok commented Aug 31, 2023 •
edited
Loading

Oh yeah, that's right, #35573.

@rok rok mentioned this issue Sep 4, 2023
Merged
@AlenkaF AlenkaF modified the milestones: 15.0.1, 16.0.0 Jan 24, 2024
pitrou added a commit that referenced this issue Feb 8, 2024
@rok @pitrou
@jorisvandenbossche @AlenkaF
GH-37484: [Python] Add a FixedSizeTensorScalar class (#37533)
026188e 
### Rationale for this change

When working with `FixedSizeTensorArray` we want to access individual tensors. This would be enabled by adding:
```python
def FixedSizeTensorScalar(pa.ExtensionScalar):
    def to_numpy_ndarray(): ...
```

See #37484.

### What changes are included in this PR?

This adds `FixedSizeTensorScalar` and tests for it.

### Are there any user-facing changes?

Yes, when calling `FixedSizeTensorArray[i]` we would get back `FixedSizeTensorScalar` instead of `ExtensionScalar`.
* Closes: #37484

Lead-authored-by: Rok Mihevc <rok@mihevc.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@pitrou pitrou modified the milestones: 15.0.1, 16.0.0 Feb 8, 2024
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@rok @pitrou
@jorisvandenbossche @AlenkaF @dgreiss
apacheGH-37484: [Python] Add a FixedSizeTensorScalar class (apache#37533
e9839e9 
)

### Rationale for this change

When working with `FixedSizeTensorArray` we want to access individual tensors. This would be enabled by adding:
```python
def FixedSizeTensorScalar(pa.ExtensionScalar):
    def to_numpy_ndarray(): ...
```

See apache#37484.

### What changes are included in this PR?

This adds `FixedSizeTensorScalar` and tests for it.

### Are there any user-facing changes?

Yes, when calling `FixedSizeTensorArray[i]` we would get back `FixedSizeTensorScalar` instead of `ExtensionScalar`.
* Closes: apache#37484

Lead-authored-by: Rok Mihevc <rok@mihevc.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@rok @pitrou
@jorisvandenbossche @AlenkaF @zanmato1984
apacheGH-37484: [Python] Add a FixedSizeTensorScalar class (apache#37533
a94253c 
)

### Rationale for this change

When working with `FixedSizeTensorArray` we want to access individual tensors. This would be enabled by adding:
```python
def FixedSizeTensorScalar(pa.ExtensionScalar):
    def to_numpy_ndarray(): ...
```

See apache#37484.

### What changes are included in this PR?

This adds `FixedSizeTensorScalar` and tests for it.

### Are there any user-facing changes?

Yes, when calling `FixedSizeTensorArray[i]` we would get back `FixedSizeTensorScalar` instead of `ExtensionScalar`.
* Closes: apache#37484

Lead-authored-by: Rok Mihevc <rok@mihevc.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
thisisnic pushed a commit to thisisnic/arrow that referenced this issue Mar 8, 2024
@rok @pitrou
@jorisvandenbossche @AlenkaF @thisisnic
apacheGH-37484: [Python] Add a FixedSizeTensorScalar class (apache#37533
db57005 
)

### Rationale for this change

When working with `FixedSizeTensorArray` we want to access individual tensors. This would be enabled by adding:
```python
def FixedSizeTensorScalar(pa.ExtensionScalar):
    def to_numpy_ndarray(): ...
```

See apache#37484.

### What changes are included in this PR?

This adds `FixedSizeTensorScalar` and tests for it.

### Are there any user-facing changes?

Yes, when calling `FixedSizeTensorArray[i]` we would get back `FixedSizeTensorScalar` instead of `ExtensionScalar`.
* Closes: apache#37484

Lead-authored-by: Rok Mihevc <rok@mihevc.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rok rok

Labels
Component: Python Type: enhancement
Projects
None yet
Milestone
16.0.0
Development

Successfully merging a pull request may close this issue.

GH-37484: [Python] Add a FixedSizeTensorScalar class rok/arrow
5 participants
@rok @jorisvandenbossche @pitrou @wjones127 @AlenkaF