Vaex string API behaves differently from pandas

[bergkvist]

Code example

Note that display() is a jupyter-function, and not available by default in Python

import pandas as pd
import vaex

df = pd.DataFrame({ 'amount': [ '1 BTC', '0.03 BTC', '0.4 ETH' ] })
vdf = vaex.from_pandas(df)

display(df.amount.str.split(' ').str.get(-1))
print('\n---\n')
display(vdf.amount.str.split(' ').str.get(-1))

Output:

0    BTC
1    BTC
2    ETH
Name: amount, dtype: object

---

Expression = str_get(str_split(amount, ' '), -1)
Length: 3 dtype: list<item: string> (expression)
------------------------------------------------
0  ['1', 'C']
1  ['3', 'C']
2  ['4', 'H']

Software information

• Vaex version (import vaex; vaex.__version__):

{'vaex': '4.4.0',
 'vaex-core': '4.4.0',
 'vaex-viz': '0.5.0',
 'vaex-hdf5': '0.9.0',
 'vaex-server': '0.6.0',
 'vaex-astro': '0.8.3',
 'vaex-jupyter': '0.6.0',
 'vaex-ml': '0.13.0'}

• Vaex was installed via: pip
• OS: Manjaro Linux

Additional information
Pandas supports 2 ways of accessing string indices, while Vaex only supports one of these:

df.amount.str.get(0)
df.amount.str[0]
vdf.amount.str.get(0)
vdf.amount.str[0] # TypeError: 'StringOperations' object is not subscriptable

[maartenbreddels]

Thanks for the report. The vaex behavior is intended, all .str functions behave on list of strings as if the function was applied on each element and I have no problem being it different from Pandas. I think what we miss is a list_get. I think similar to the .struct accessor (see https://vaex--1522.org.readthedocs.build/en/1522/api.html#struct-arrow-operations from PR #1447 (cc @mansenfranzen ) we should have some .list operations as well, and map df.x[:,i] to a list_get function.

[bergkvist]

@JovanVeljanoski Has this been implemented now, since the issue was closed? If it has been, what did the syntax land on, and since which version is it available?

[maartenbreddels]

Not sure why this was closed, @JovanVeljanoski I think we should have an issue about supporting lists, and link that that, there is also a discussion on this IIRC.

[JovanVeljanoski]

Because the main issue is accessing and operations of lists, and I believe it is tracked elsewhere. But we can keep this open as well.

[longtng]

I have faced a similar issue as I would like to extract the first 10-chars of each cell from a column. In Pandas just sth like this:
df["COMPANY_CODE"] = df["COMPANY_CODE"][:10]
But it does not work in vaex this way as vaex return issues:
TypeError: 'StringOperations' object is not subscriptable

[maartenbreddels]

Hi,

I think you want https://vaex.io/docs/api.html#vaex.expression.StringOperations.slice

[longtng]

Hi,

I think you want https://vaex.io/docs/api.html#vaex.expression.StringOperations.slice

superb, thank you very much