Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PipeOpEncode doesn't work for features of type character() #749

Closed
bblodfon opened this issue Dec 11, 2023 · 2 comments
Closed

PipeOpEncode doesn't work for features of type character() #749

bblodfon opened this issue Dec 11, 2023 · 2 comments

Comments

@bblodfon
Copy link

The doc says that is should encode character() columns, but:

library(mlr3)
library(mlr3pipelines)

data = data.table::data.table(x = letters[1:3], y = factor(letters[1:3]))
task = as_task_classif(data, id = "task", target = "y")
poe = po("encode")
poe$train(list(task))[[1]]$data() # no encoding!
#>    y x
#> 1: a a
#> 2: b b
#> 3: c c

Created on 2023-12-11 with reprex v2.0.2

@bblodfon bblodfon changed the title po("encode") doesn't work for features of type character() PipeOpEncode doesn't work for features of type character() Dec 11, 2023
@mb706
Copy link
Collaborator

mb706 commented Dec 11, 2023

The semantics of a character feature (at least as far as mlr3pipelines sees it) are that it contains "free text", as opposed to levels of a fixed set of possible values (like factors). One would typically apply "nlp" methods on character features, e.g. using po("textvectorizer") to extract a bag of words representation.
If you really want to do factor encoding on character features, then your case is comparable to wanting to do factor encoding on numeric features: what you really want is for mlr3 to see the semantics of your feature in a non-standard way. The solution for this is to convert your feature, using po("colapply", applicator = as.factor). Does your usecase work with that?

@bblodfon
Copy link
Author

Martin the issue here is that many datasets that we use in R, have factors as characters (in terms of their type). So when users try to do (factor) encoding, they expect to get the feature called sex ("male" and "female") encoded but they don't. The colapply is a workaround for sure (+affect_columns needs to be configured properly, extra thing) but if docs says the PipeOp does work on character features but it actually doesn't work, well one of the two needs to be changed/updated :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants