[R] Implement feature weights.

[trivialfis]

Add feature weights to R DMatrix. Please note that the "approx" tree method was rewritten very recently and doesn't support feature weights in released versions.

cc @hetong007 @tibshirani

Close #7657

[tibshirani]

I am a github novice: have you added the feature weights ? Is there a new version I can try? Thanks!

rob

[trivialfis]

Hi @tibshirani,

have you added the feature weights

Yes, it's added.

Is there a new version I can try

Not yet, unfortunately, we are still in the development phase and not ready for a new major release. You need to install xgboost from the source code, assuming you have a recent R toolchain:

git clone --recursive https://github.com/dmlc/xgboost.git
cd xgboost/R-package/
R CMD INSTALL .

Please let us know if there's anything we can help with.

[tibshirani]

Thanks Jiamang! Will try to compile it best rob

…

On Wed, Feb 16, 2022 at 10:49 AM Jiaming Yuan ***@***.***> wrote: Hi @tibshirani <https://github.com/tibshirani>, have you added the feature weights Yes, it's added. Is there a new version I can try Not yet, unfortunately, we are still in the development phase and not ready for a new major release. You need to install xgboost from the source code, assuming you have a recent R toolchain: git clone --recursive https://github.com/dmlc/xgboost.git cd xgboost/R-package/ R CMD INSTALL . Please let us know if there's anything we can help with. — Reply to this email directly, view it on GitHub <#7660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVXOYNHUMW5KH2OCYJL7KTU3PWTDANCNFSM5OPHF4HA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- I get so much email that I might not reply to an incoming email, just because it got lost. So don't hesitate to email me again. The probability of a reply should increase. Prof. Robert Tibshirani Depts of Biomedical Data Sciences, and Statistics Stanford Univ Stanford CA 94305 ***@***.*** http://www-stat.stanford.edu/~tibs

[tibshirani]

Jiamang i have successfully compiled it. But i do not see the argument for feature weights(?) thanks rob On Wed, Feb 16, 2022 at 10:54 AM robert tibshirani ***@***.***> wrote:

…

Thanks Jiamang! Will try to compile it best rob On Wed, Feb 16, 2022 at 10:49 AM Jiaming Yuan ***@***.***> wrote: > Hi @tibshirani <https://github.com/tibshirani>, > > have you added the feature weights > > Yes, it's added. > > Is there a new version I can try > > Not yet, unfortunately, we are still in the development phase and not > ready for a new major release. You need to install xgboost from the source > code, assuming you have a recent R toolchain: > > git clone --recursive https://github.com/dmlc/xgboost.git > cd xgboost/R-package/ > R CMD INSTALL . > > Please let us know if there's anything we can help with. > > — > Reply to this email directly, view it on GitHub > <#7660 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAVXOYNHUMW5KH2OCYJL7KTU3PWTDANCNFSM5OPHF4HA> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. > > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- I get so much email that I might not reply to an incoming email, just because it got lost. So don't hesitate to email me again. The probability of a reply should increase. Prof. Robert Tibshirani Depts of Biomedical Data Sciences, and Statistics Stanford Univ Stanford CA 94305 ***@***.*** http://www-stat.stanford.edu/~tibs

-- I get so much email that I might not reply to an incoming email, just because it got lost. So don't hesitate to email me again. The probability of a reply should increase. Prof. Robert Tibshirani Depts of Biomedical Data Sciences, and Statistics Stanford Univ Stanford CA 94305 ***@***.*** http://www-stat.stanford.edu/~tibs

[trivialfis]

Hi @tibshirani , I wrote a test for this PR, can it be a starting point?

library(xgboost)

context("feature weights")

test_that("training with feature weights works", {
  nrows <- 1000
  ncols <- 9
  set.seed(2022)
  x <- matrix(rnorm(nrows * ncols), nrow = nrows)
  y <- rowSums(x)
  weights <- seq(from = 1, to = ncols)

  test <- function(tm) {
    names <- paste0("f", 1:ncols)
    xy <- xgb.DMatrix(data = x, label = y, feature_weights = weights)
    params <- list(colsample_bynode = 0.4, tree_method = tm, nthread = 1)
    model <- xgb.train(params = params, data = xy, nrounds = 32)
    importance <- xgb.importance(model = model, feature_names = names)
    expect_equal(dim(importance), c(ncols, 4))
    importance <- importance[order(importance$Feature)]
    expect_lt(importance[1, Frequency], importance[9, Frequency])
  }

  for (tm in c("hist", "approx", "exact")) {
    test(tm)
  }
})

[tibshirani]

Thank you very much! Works for me if my project turns into paper, I'll be sure to acknowledge your help best rob

…

On Fri, Feb 18, 2022 at 3:08 AM Jiaming Yuan ***@***.***> wrote: Hi @tibshirani <https://github.com/tibshirani> , I wrote a test for this PR, can it be a starting point? library(xgboost) context("feature weights") test_that("training with feature weights works", { nrows <- 1000 ncols <- 9 set.seed(2022) x <- matrix(rnorm(nrows * ncols), nrow = nrows) y <- rowSums(x) weights <- seq(from = 1, to = ncols) test <- function(tm) { names <- paste0("f", 1:ncols) xy <- xgb.DMatrix(data = x, label = y, feature_weights = weights) params <- list(colsample_bynode = 0.4, tree_method = tm, nthread = 1) model <- xgb.train(params = params, data = xy, nrounds = 32) importance <- xgb.importance(model = model, feature_names = names) expect_equal(dim(importance), c(ncols, 4)) importance <- importance[order(importance$Feature)] expect_lt(importance[1, Frequency], importance[9, Frequency]) } for (tm in c("hist", "approx", "exact")) { test(tm) } }) — Reply to this email directly, view it on GitHub <#7660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVXOYKERPGYWONL4MTGFRDU3YSDPANCNFSM5OPHF4HA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- I get so much email that I might not reply to an incoming email, just because it got lost. So don't hesitate to email me again. The probability of a reply should increase. Prof. Robert Tibshirani Depts of Biomedical Data Sciences, and Statistics Stanford Univ Stanford CA 94305 ***@***.*** http://www-stat.stanford.edu/~tibs