[R-package] miscellaneous changes to comply with CRAN requirements

[jameslamb]

This PR attempts to address the most recent request for changes from CRAN, #3293 (comment), on our way to #629

See changes to cran-comments.md for details.

[jameslamb]

@guolinke here is a package built from this branch, that can be submitted:

lightgbm_3.0.0.tar.gz

[StrikerRUS]

LGTM! Just two minor comments that shouldn't block merging.

BTW, is it still pre-checks from CRAN? Because I don't see errors for Solaris that we haven't fixed yet.

[jameslamb]

BTW, is it still pre-checks from CRAN? Because I don't see errors for Solaris that we haven't fixed yet.

We made it past the pre-checks (which is just Debian Linux and Windows) and are now into human review.

I think it is possible that we'll be accepted, then fail the Solaris check, and then be told "hey you are failing the Solaris check, you have 30 days to fix it or we're taking you down".

It's also possible that they'll allow us to stay up indefinitely without passing that check. It isn't easy to understand what the exact expectation is 😬

There are some packages currently in ERROR status on Solaris that have been allowed to stay on CRAN: https://cran.r-project.org/web/checks/check_details_r-patched-solaris-x86.html

[StrikerRUS]

... or we're taking you down"

Sounds scary!

I hope that we will be able to adopt that patch and it will work.

[jameslamb]

... or we're taking you down"

Sounds scary!

I hope that we will be able to adopt that patch and it will work.

haha yeah, it can happen.

I'm going to try that patch this weekend (maybe even tonight) based on this advice: #629 (comment)

worst-case scenario, if that doesn't work we can figure out how to disable distributed training on Solaris. I doubt anyone trying to do distributed training in LightGBM is going to provision a cluster of Solaris 10 machines 😂

I think that should be the only thing the network code is needed for.

[guolinke]

@jameslamb
Dear maintainer,

package lightgbm_3.0.0.tar.gz does not pass the incoming checks automatically, please see the following pre-tests:
Windows: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.0.0_20200828_032044%2FWindows%2F00check.log&data=02%7C01%7Cguolin.ke%40microsoft.com%7C822df97f9a794c5659cc08d84b150b28%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637341901057745862&sdata=Y9kE%2FpOSK7VaxLisXcruFs8vdqH%2B7D75ZcSVRwOi524%3D&reserved=0
Status: 2 NOTEs
Debian: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.0.0_20200828_032044%2FDebian%2F00check.log&data=02%7C01%7Cguolin.ke%40microsoft.com%7C822df97f9a794c5659cc08d84b150b28%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637341901057745862&sdata=EjkSroKsewumq2f8mHt5BhOUE30KPAVfcKmIh9yKcGo%3D&reserved=0
Status: 2 NOTEs

Please fix all problems and resubmit a fixed version via the webform.
If you are not sure how to fix the problems shown, please ask for help on the R-package-devel mailing list:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-package-devel&data=02%7C01%7Cguolin.ke%40microsoft.com%7C822df97f9a794c5659cc08d84b150b28%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637341901057755854&sdata=bOgHMSqtOTVA0ZHVgjvMwdeLFLmBKcS%2F3Bs2hFz0U9I%3D&reserved=0
If you are fairly certain the rejection is a false positive, please reply-all to this message and explain.

More details are given in the directory:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwin-builder.r-project.org%2Fincoming_pretest%2Flightgbm_3.0.0_20200828_032044%2F&data=02%7C01%7Cguolin.ke%40microsoft.com%7C822df97f9a794c5659cc08d84b150b28%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637341901057755854&sdata=ltIZ2uMZK38H7%2FLc0X7CYD8gfyuXhQXF3S7v7OdrNgk%3D&reserved=0
The files will be removed after roughly 7 days.

No strong reverse dependencies to be checked.

Best regards,
CRAN teams' auto-check service

[guolinke]

more details:

Flavor: r-devel-linux-x86_64-debian-gcc, r-devel-windows-ix86+x86_64
Check: CRAN incoming feasibility, Result: NOTE
  Maintainer: 'Guolin Ke <guolin.ke@microsoft.com>'
  
  New submission
  
  Possibly mis-spelled words in DESCRIPTION:
    Guolin (13:52)
    Ke (13:48)
    LightGBM (14:20)
    al (13:62)
    et (13:59)

Flavor: r-devel-linux-x86_64-debian-gcc, r-devel-windows-ix86+x86_64
Check: top-level files, Result: NOTE
  Non-standard files/directories found at top level:
    'docs' 'lightgbm-hex-logo.png' 'lightgbm-hex-logo.svg'

[jameslamb]

ahhhh sorry! the image files are my mistake. I expect they'll allow us to keep those other words because they ard not misspellings.

I'll push a change here and attach a new package in a few minutes

[jameslamb]

Ok I've made some changes. Could you try this one? After you submit, could you reply-all to that email and tell them that we have submitted a fix for the unnecessary files, but that we think the "mispellings" note is a false positive?

lightgbm_3.0.0.tar.gz

[guolinke]

@jameslamb
still waiting for the human check...
BTW, do we need to wait for several days every time we update the new version to CRAN?

[guolinke]

I think we cannot block the release plan by cran.
If we still don't get the reply tomorrow, maybe we should release the 3.0.0 version for github/pypi first.

[jameslamb]

I think we cannot block the release plan by cran.
If we still don't get the reply tomorrow, maybe we should release the 3.0.0 version for github/pypi first.

I'm ok with that! Let's not hold up #3293 any longer. It's ok for CRAN to come a bit later.

BTW, do we need to wait for several days every time we update the new version to CRAN?

We shouldn't have to. In my experience, projects are scrutinized much more closely the first time they're submitted to CRAN. Once we've been accepted once, we should be able to get accepted automatically for future releases.

[jameslamb]

@StrikerRUS @guolinke I found this interesting site that checks the status of CRAN submissions!

https://lockedata.github.io/cransays/articles/dashboard.html

It has a dashboard that is the result of polling ftp://cran.r-project.org/incoming/. It shows we're stuck in newbies:

a specific queue for the manual inspection of first time CRAN submissions.

[jameslamb]

When I checked https://lockedata.github.io/cransays/articles/dashboard.html last night, there were 16 packages ahead of us (in "newbies" for longer than we've been).

Now there are only two. So I hope we'll hear back in the next 24 hours!

[jameslamb]

I've just uploaded a 3.0.0 source package, in response to #3350 .

https://github.com/microsoft/LightGBM/releases/download/v3.0.0/lightgbm-3.0.0-r-cran.tar.gz

I'll update it if CRAN asks for changes and we have to update this PR, but this was our users can install the 3.0.0 release of the R package easily.

I can prepare binaries in a day or two. And have not forgotten about doing #3283 so that happens automatically.

[jameslamb]

I just added a proposed CI job using sanitizers in #3439

[StrikerRUS]

@jameslamb Thanks a lot for this huge job with CRAN requirements! This is really great efforts!
Please take a look at some my minor comments and fix linting issue about using cat.

[StrikerRUS]

Looks great! Thank you!

[jameslamb]

Thanks for the reviews and efforts everyone! I don't think there are any more discussions that are unresolved, so I'll merge this when CI is successful.

I'll address valgrind checks, ubsan checks, and other things in follow-up PRs and hopefully by the end of November we'll be on CRAN!

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.