Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start working on validation support #753

Closed
wants to merge 1 commit into from
Closed

Conversation

sebffischer
Copy link
Sponsor Member

@sebffischer sebffischer commented Jan 24, 2024

  • The GraphLearner / Graph should null the test roles unless there is a PipeOp that makes use of them. Otherwise we might end up making predictions that are not needed. Maybe we want to achieve this differently / change this in mlr3?

PipeOps to look at:

  • PipeOpChunk
  • PipeOpFeatureUnion
  • PipeOpImpute
  • PipeOpOVRSplit
  • PipeOpTuneThreshold (not really anything to do)
  • PipeOpLearner (not really anything to do)
  • PipeOpLearnerCV (not a lot to do)
  • PipeOpTrafo
  • PipeOpTaskPreproc: train and predict stage now need to work a little differently even for stuff like PipeOpPreprocSimple

@sebffischer sebffischer changed the title start workign on validation support start working on validation support Jan 24, 2024
@sebffischer
Copy link
Sponsor Member Author

This does not work because of the same reason bootstrapping does not work with mlr3pipelines.
Consider the case where a task has overlapping "use" and "test" rows, e.g. task$row_roles$use of c(1, 2) and task$row_roles$test and this task should be preprocessed by PipeOpTaskPreproc.

We want to treat the row 2 during train and 2 during predict differently, as we want to apply the training preprocessing to the former and the prediction preprocessing to the latter. This does not work, because the tasks's data backend has only 3 rows and we want to cbind a backend that has 4 rows.

@mb706
Copy link
Collaborator

mb706 commented Jan 26, 2024

#706

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants