Easily featurize the inputs in the Subset class
#71
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelogs
featurization_fnto theSubsetclass and propagates this parameter to theget_train_test_split()method.Subsetclass function properly.Checklist:
Was this PR discussed in an issue? It is recommended to first discuss a new feature into a GitHub issue before opening a PR.feature,fixortest(or ask a maintainer to do it for you).Discussion
Throughout user interviews, adding support for easily featurizing the inputs was a common feature request. This PR takes a first step in that direction.
Before:
After:
What about the targets?
There could be benchmarks in which the target would have to be featurized too (e.g. a reaction product). I went down the rabbit hole of also adding a featurization function for the targets. However, this opened up other difficult questions that were difficult to address.
BenchmarkSpecification.evaluate()does not have ay_trueparameter to minimize the chance of a user accidentally changing the test labels. However, if we allow a target transformation to be passed to theget_train_test_split()method, this means we need to keep track of that function to replicate the correcty_truevalues inevaluate().evaluate()? We probably can't, so then it seems important to throw an error if this happens.I can see ways around all of the above, but this is out-of-scope for now. We can revisit this once a benchmark actually needs it.