- New method to label columns in tasks:
- New method to add stratification variables:
- New helper function
partition()to split a task into a training and test set.
- New standardized getter
- New measures
"bic"to compute the Akaike Information Criterion or the Bayesian Information Criterion, respectively.
- New Resampling method:
- Removed public (but documented as internal) field
BenchmarkResultto simplify the API and avoid confusion. The converter
as.data.table()can be used instead.
- Measures now have formal hyperparameters. A popular example where this is
required is the F1 score, now implemented with customizable
- Changed default of argument
- Fixed hashing of some measures.
- Removed experimental column role
uri. This role be split up into multiple roles by the
- Added a
- Renamed column
- Added converters
as_prediction_regr()to reverse the operation of
- Specifying a weight column during
learner$predict_newdata()is not mandatory anymore (#563).
Task$data()defaults to return only active rows and columns, instead of asserting to only return rows and columns. As a result, the
$data()method can now also be used to query inactive rows and cols from the
- New (experimental) column role
uriwhich is intended to point to external resources, e.g. images on the file system.
- New helper
set_threads()to control the number of threads during calls to external packages. All objects will be migrated to have threading disabled in their defaults to avoid conflicting parallelization techniques (#605).
- New option
mlr3.debug: avoid calls to
benchmark()to improve the readability of tracebacks.
- New experimental option
mlr3.allow_utf8_names: allow non-ascii characters in column names in tasks.
- Result containers
BenchmarkResultnow optionally remove the DataBackend of the Tasks in order to reduce file size and memory footprint after serialization. To remove the backends from the containers, set
benchmark(), respectively. Note that this behaviour will eventually will be the default for future releases.
- Prediction objects generated by
Learner$predict_newdata()now have row ids starting from 1 instead auto incrementing row ids of the training task.
as.data.table.DictionaryTasksnow returns an additional column
- Added flag
BenchmarkResult$score()to allow to work with failing learners more conveniently.
- New methods for
$set_row_rolesas a replacement for the deprecated and less flexible
- Learners can now have a timeout (#556).
- Removed S3 method
friedman.test.BenchmarkResult()in favor of the new
MeasureOOBErrornow has set property
- New learner property
"featureless"to tag learners which can operate on featureless tasks.
- Fixed [ResampleResult] ignoring argument
predict_setsfor returned [Prediction] objects.
- Compability with new version of
- Updated properties of featureless learners to apply it on all feature types (did not work on POSIXct columns).
- Fixed measures being calculated as
BenchmarkResultfor resamplings with a single iteration (#551).
- Fixed a bug where a broken heuristic disabled nested parallelization via
BenchmarkResultnow share a common interface to store the experiment results. Manual construction is still possible with helper function
- Fixed deep cloning of
- New measure
classif.prauc(area under precision-recall curve).
- Removed dependency on orphaned package
- Compact in-memory representation of R6 objects to save space when
saving objects via
- Objects in containers like
BenchmarkResultare now de-duplicated for an optimized serialization.
- Fixed data set
breast_cancer: all factor features are now correctly stored as ordered factors.
- Added a new utility function
- Added classification task
ResamplingLOOfor leave-one-out resampling.
- Regression now supports predict type
ResamplingBootstrapin combination with grouping (#514).
- Fixed plot method of
- Added hyperparameter
- Added new task generators (
- Columns in tasks generated by task generators have been renamed to be more consistent.
- Added a
plot()method for most task generators.
- Corrected data in task
future.applyis now imported (instead of suggested). This is necessary to ensure reproducibility: This way exactly the same result is calculated, independent of the parallel backend.
- Fixed a bug where prediction on new data for a task with blocking information raised an exception (#496).
- New binding:
- Some handy cheat sheets can now be downloaded from the project homepage.
- Added new measures
classif.bbrier(binary Brier score) and
classif.mbrier(multi-class Brier score).
- Added new Resampling:
- Added base class for unsupervised tasks:
- Added S3 methods to combine
- Fixed a bug where automatic generation of row ids could lead to duplicated ids
Switched to new
roxygen2documentation format for R6 classes.
benchmark()now support progress bars via the package
Row ids now must be numeric. It was previously allowed to have character row ids, but this lead to confusion and unnecessary code bloat. Row identifiers (e.g., to be used in plots) can still be part of the task, with row role
Row names can now be queried with
DataBackendMatrixnow supports to store an optional (numeric) dense part.
Added new method
ResampleResults to a subset of iterations.
character()-> object converters.
Empty test sets are now handled separately by learners (#421). An empty prediction object is returned for all learners.
The internal train and predict function of
Learnernow should be implemented as private method: instead of public methods
predict_internal, private methods
.predictare now encouraged.
It is now encouraged to move some internal methods from public to private:
Learner$train_internalshould now be private method
Learner$predict_internalshould now be private method
Measure$score_internalshould now be private method
$.score. The public methods will be deprecated in a future release.
Removed arguments from the constructor of measures
classif.costs. These can be set directly by
We have published an article about mlr3 in the Journal of Open Source Software: https://joss.theoj.org/papers/10.21105/joss.01903. See
citation("mlr3")for the citation info.
Learners returned by
BenchmarkResult$learnersare reset to encourage the safer alternative
BenchmarkResult$score()to access trained models.
Fix ordering of levels in
PredictionClassif$set_threshold()(triggered an assertion).
Switched from package
Measures can now calculate all scores using micro or macro averaging (#400).
Measures can now be configured to return a customizable performance score (instead of
NA) in case the score cannot be calculated.
Character columns are now treated differently from factor columns. In the long term,
character()columns are supposed to store text.
Fixed a bug triggered by integer grouping variables in
benchmark_grid()now accepts instantiated resamplings under certain conditions.
Task$set_row_roles()are now deprecated. Instead it is recommended for now to work with the lists
Learner$predict_newdata()now works without argument
taskif the learner has been fitted with
Names of column roles have been unified (
"groups"have been renamed).
MeasureClassifFScoreand fixed a bug in the F1 performance calculation (#353). Thanks to @001ben for reporting.
Stratification is now controlled via a task column role (was a parameter of class
Added a S3
predict()method for class
Learnerto increase interoperability with other packages.
Many objects now come with a
$help()which opens the respective manual page.
It is now possible to predict and score results on the training set or on both training and test set. Learners can be instructed to predict on multiple sets by setting
"test"). Measures operate on all sets specified in their field
ResampleResult$predictions()are now methods instead of fields, and allow to extract predictions for different predict sets.
ResampleResult$performance()has been renamed to
BenchmarkResult$performance()has been renamed to
Changed API for (internal) constructors accepting
paradox::ParamSet(). Instead of passing the initial values separately, the initial values must now be set directly in the
Deprecated support of automatically creating objects from strings. Instead,
mlr3provides the following helper functions intended to ease the creation of objects stored in dictionaries:
BenchmarkResultnow ensures that the stored
ResampleResults are in a persistent order. Thus,
ResampleResults can now be addressed by their position instead of their hash.
New S3 generic
Removed the control object
- Initial upload to CRAN.