Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request]: Add the ability to use the drop function with doric columns #363

Closed
Amalicia opened this issue Jun 9, 2023 · 3 comments · Fixed by #366
Closed

[Feature request]: Add the ability to use the drop function with doric columns #363

Amalicia opened this issue Jun 9, 2023 · 3 comments · Fixed by #366
Labels
API enhancement New feature or request good first issue Good for newcomers

Comments

@Amalicia
Copy link
Contributor

Amalicia commented Jun 9, 2023

Feature suggestion

The following functionality is not supported:

df.drop(colString("col1))

Whilst the same effect can be achieved via select, having the ability to drop is useful in certain cases.

Whilst looking into this I found out the Spark doe not support dropping multiple columns via col notation or similar and only allows dropping columns via string column names. It might be a nice addition to doric to have a function defined similar to the following:

def drop(col: DoricColumn[_]*): DataFrame

Current behaviour

When trying to use the drop function with a doric column we get the following error:

Cannot resolve overloaded method 'drop`
@Amalicia Amalicia added the enhancement New feature or request label Jun 9, 2023
@Amalicia
Copy link
Contributor Author

Amalicia commented Jun 9, 2023

As mentioned, I'm not fully sure if this is functionality required in doric given we can achieve it with select. However if this is something you do feel will add value please let me know as I have the changes locally for both functions 😃

@alfonsorr
Copy link
Member

Hi!

The idea is good, but the original logic of drop is that it never throws an error if the column doesn't exist. We can have an alternative version that checks if it exists and returns an error in case it doesn't.

@eruizalo eruizalo added good first issue Good for newcomers API labels Jun 20, 2023
@Amalicia
Copy link
Contributor Author

Ahhhhh I didn't realise that was the case with drop, thanks for the heads up!

Throwing an error when the column doesn't exist sounds like a great idea, I'll work on adding that to what I currently have and go from there 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants