Missing transforms

[juliohm]

Below is a list of transforms to be implemented:

• Rename
• Filter
• DropMissing
• Replace
• Coalesce
• Coerce

Contributions are very welcome! Just comment on the issue if you plan to start working on some of these.

[Omar-Elrefaei]

Hi! I am going to try implementing a Rename transform.

I am kinda of an Beginner-Intermediate in julia. Incidentally I was reading about Transducers and watching Rich Hickey talk about them hours before I found this issue on the good-first-issues on Julia's homepage.

[juliohm]

Awesome @Omar-Elrefaei, welcome to the Julia community! :)

Below is the signature of the Rename transform we need:

struct Rename <: Stateless
  names::Dict{Symbol,Symbol}
end

This will create a default constructor that works like

Rename(Dict(:col1 => :newcol1, :col2 => :newcol2))

After we have this struct holding a dictionary with symbols, we need to implement apply and revert.

Let me know if you need any help.

[juliohm]

By the way, what web page is mentioning our issues as good first issues?

[Omar-Elrefaei]

Thanks for the warm welcome @juliohm.

At the bottom of julialang.org there is a link to good first issues

[juliohm]

Interesting, wasn't aware of this link, thanks.

[juliohm]

@Omar-Elrefaei any update on this? If you can start working on a PR it will be easier for us to review while you evolve the solution. Many first-time contributors try to develop a feature on their own in isolation just to discover that radical changes to the design are necessary later. If you open a PR and continue pushing commits to the branch, GitHub will update and will allow reviewers guide you in the process.

[Omar-Elrefaei]

Yes, I definitely agree. I do semi-regularly make that mistake of banging my head on the wall too much instead of just asking questions. Anyway PR is sent.

[juliohm]

Thank you @Omar-Elrefaei for adding a new transform to the list. Feel free to pick any other from the list. The Coerce transform should be simple to implement as well given that we can simply wrap the ScientificTypes.coerce function in a lazy way.

[juliohm]

We are only missing Replace, Coalesce and Coerce for a v1.0 release 👍🏽

[ceferisbarov]

What is the expected behavior of Coalesce and Coerce transforms? I am aware of functions and commands with same name in other languages, but not all of them behave in the same way, and not all of them are table transforms.

[ceferisbarov]

What would syntax of the Replace transform be like? Do we only provide what values to change:

Replace(changes::Dict{Symbol,Symbol})

Or do we also provide what columns to work on:

Replace(colnames, changes::Dict{Symbol,Symbol})

This wouldn't work well, because we have no way of specifying what values to replace in which column.

Pandas has a solution to this. to_replace parameter determines what values to change in which columns and value parameter determines what values to replace them with.

[juliohm]

What is the expected behavior of Coalesce and Coerce transforms? I am aware of functions and commands with same name in other languages, but not all of them behave in the same way, and not all of them are table transforms.

They are the equivalent of Base.coalesce and Scitypes.coerce. @eliascarv is already working on the Coalesce, which is related to Replace. For Coerce we just need to forward the arguments to Scitypes.coerce. Do you want to work on that @ceferisbarov ?

[ceferisbarov]

Thanks for the response. Yes, I do. @juliohm

[juliohm]

Great job everyone, this issue is now complete. Let's open new issues for specific transforms in the future.