Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] Release v3.3.4 #5619

Closed
wants to merge 18 commits into from
Closed

Conversation

jameslamb
Copy link
Collaborator

@jameslamb jameslamb commented Dec 6, 2022

Similar to #5525, this is an unusual release. It is v3.3.4, but doesn't include all of the changes made since v3.3.3.

Instead, this PR just contains the changes on top of v3.3.4 needed to prevent the R package from being archived by CRAN.

Fixes #5618.

THIS PR SHOULD NEVER BE MERGED


Setup

I created a new branch, release/v3.3.3, which contains only:

  • LightGBM's source code as of tag v3.3.3
  • non-breaking CI configs and build scripts as of latest master (f0cfbff)
how I did that (click me)
# get the v3.3.3 changes from main LightGBM repo
git remote add upstream git@github.com:microsoft/LightGBM.git
git checkout master
git fetch upstream --tags

# create a new branch with the v3.3.3 changes + any CI fixes
git checkout v3.3.3
git checkout -b release/v3.3.3
git push upstream release/v3.3.3

# https://github.com/microsoft/LightGBM/pull/5484
git cherry-pick dc4794b62f2c09103e558b7a8325bed66083a5c3

# https://github.com/microsoft/LightGBM/pull/5510
git cherry-pick 1d6d3b37da10bf6dce9e2aaf1f10355a6c492614

# https://github.com/microsoft/LightGBM/pull/5504
git cherry-pick 223013421f6eb756eca7f669be6c21842277c058

# https://github.com/microsoft/LightGBM/pull/5527
git cherry-pick 82b7d31c783a93cc96cf7fea24098b866182762f

# https://github.com/microsoft/LightGBM/pull/5503
git cherry-pick 45dd49ec0d8e3bbea7adbf855bb48f261739cf21

# https://github.com/microsoft/LightGBM/pull/5563
git cherry-pick d8a6b8871b990950ab37cfa96d83e3bfc87680ba

# https://github.com/microsoft/LightGBM/pull/5570
git cherry-pick 629b3ce58e2dfecdc0e4376dbc780e35fd826f99

# https://github.com/microsoft/LightGBM/pull/5555
git cherry-pick 1a67a36fd62ce3fd4001a58f64116bc43c0f031a

# https://github.com/microsoft/LightGBM/pull/5602
git cherry-pick 60c92a969a0075cd271d55e40b611aaf52b061ed

# update macOS-latest job configurations, since Python 3.6 is no longer supported
git checkout master -- .github/workflows/python_package.yml

# create a new branch for this patch release
git checkout -b release/v3.3.4

# pull over the vsprintf change
git cherry-pick 2d4654a18f970c10d35e1e8e0e60dd102310c4ca

Tags and artifacts should be made directly from this release_v3.3.4 branch instead of master.

Release checklist:

Copied from #5525.

What about the changelog?

As described in #4930 (comment), doing this interferes with the automatic changelog generation we get from the release-draft bot.

So just like @StrikerRUS did in #4930 and I did in #5525, I've included the current draft changelog here, as well as the one saved in the comments on #4930.

changelog from v3.3.1 to when v3.3.2 was created (click me)

NOTE: I wrapped this in a markdown text block so GitHub stops re-evaluating the links.

## Changes

## 💡 New Features

- [R-package] added argument eval_train_metric to lgb.cv() (fixes #4911) @mayer79 (#4918)
- Add support for Visual Studio 2022 @StrikerRUS (#4889)
- Add C API function that returns all parameter names with their aliases @StrikerRUS (#4829)
- [python][sklearn] respect parameters for predictions in `init()` and `set_params()` methods @StrikerRUS (#4822)
- Add customized parser support @chjinche (#4782)
- [R-package] Add `print()` and `summary()` methods for Booster @david-cortes (#4686)
- Add 'nrounds' as an alias for 'num_iterations' (fixes #4743) @mikemahoney218 (#4746)
- [python-package] early stopping min_delta (fixes #2526) @jmoralez (#4580)
- [python][sklearn] respect objective aliases @StrikerRUS (#4758)
- [python][sklearn] add `n_estimators_` and `n_iter_` post-fit attributes @StrikerRUS (#4753)

## 🔨 Breaking

- [python] remove `early_stopping_rounds` argument of `train()` and `cv()` functions @StrikerRUS (#4908)
- [python] remove `evals_result` argument of `train()` function @StrikerRUS (#4882)
- [python][sklearn] do not replace empty dict with `None` for `evals_result_` @StrikerRUS (#4884)
- [python] Drop Python 3.6 support @StrikerRUS (#4891)
- [python] remove `verbose_eval` argument of `train()` and `cv()` functions @StrikerRUS (#4878)
- [python] remove `verbose` argument of `model_from_string()` method of Booster class @StrikerRUS (#4877)
- [python][sklearn] Remove `early_stopping_rounds` argument of `fit()` method @StrikerRUS (#4846)
- [R-package] remove support for '...' in `slice()` @jameslamb (#4872)
- [R-package] remove support for '...' in `lgb.Dataset()` @jameslamb (#4874)
- [R-package] remove support for '...' in `dim.lgb.Dataset()` @jameslamb (#4873)
- [R-package] remove support for '...' in lgb.train() @jameslamb (#4863)
- [R-package] remove support for '...' in `create_valid()` @jameslamb (#4865)
- [R-package] remove support for 'info' in Dataset @jameslamb (#4866)
- [R-package] remove Dataset getinfo() @jameslamb (#4864)
- [R-package] remove support for '...' in lgb.cv() @jameslamb (#4860)
- [R-package] remove Dataset setinfo() @jameslamb (#4854)
- [R-package] remove support for '...' in predict() @jameslamb (#4857)
- [R-package] remove support for '...' in Booster reset_parameter() @jameslamb (#4856)
- [python][sklearn] unify values of `best_iteration` for sklearn and standard APIs @StrikerRUS (#4845)
- [ci] migrate CI from macOS 10.14 to 10.15 and drop support of Mojave @StrikerRUS (#4849)
- [R-package] enable saving Booster with saveRDS() and loading it with readRDS() (fixes #4296) @david-cortes (#4685)
- [python][sklearn] remove `verbose` argument from `fit()` method @StrikerRUS (#4832)
- [python] remove `learning_rates` argument of `train()` function @StrikerRUS (#4831)
- [python] remove "auto" value of `ylabel` argument of `plot_metric()` function @StrikerRUS (#4818)
- [python] Remove `print_evaluation()` function @StrikerRUS (#4819)
- [python] Remove `silent` argument @StrikerRUS (#4800)

## 🚀 Efficiency Improvement

- clear memory allocated for sampled data when constructing Dataset from text file @xuchuanyin (#4890)
- [python] Faster categorical column names selection @Neronuser (#4787)
- [R-package] parallelize compilation in CMake-based builds @jameslamb (#4525)

## 🐛 Bug Fixes

- [R-package] respect 'verbose' argument in lgb.cv() (fixes #4667) @jameslamb (#4903)
- [R-package] Apply patch for R4.2 on Windows @shiyu1994 (#4923)
- [R-package] respect aliases for objective and metric and lgb.train() and lgb.cv() @jameslamb (#4913)
- [python] raise an informative error instead of segfaulting when custom objective produces incorrect output @yaxxie (#4815)
- [R-package] fix handling of duplicate parameters (fixes #4521) @jameslamb (#4914)
- [R-package] update parameter 'verbosity' based on keyword arg 'verbose' @jameslamb (#4899)
- [R-package] fix CVBooster reset_parameter() method (fixes #4900) @jameslamb (#4901)
- [python] reset storage in record evaluation callback each time before starting training @StrikerRUS (#4885)
- [python] reset storages in early stopping callback after finishing training @StrikerRUS (#4868)
- [R-package] fix `--no-build-vignettes` option for `build-cran-package.sh` @jameslamb (#4848)
- [python][docs] fix type hints for custom functions and remove vague `array-like` wording @StrikerRUS (#4816)
- Always respect forced splits, even when feature_fraction < 1.0 (fixes #4601) @tongwu-msft (#4725)
- Reset OpenMP thread number if num_threads <= 0 @hzy46 (#4704)

## 📖 Documentation

- [R-package] [docs] fix calculation of R test coverage (fixes #4919) @jameslamb (#4922)
- [docs] Update link to FLAML code example @qingyun-wu (#4892)
- [R-package] [docs] add Michael Mayer to DESCRIPTION @jameslamb (#4867)
- [docs] document that `pred_early_stop` can be used only in normal and raw scores prediction @StrikerRUS (#4823)
- [R-package] [docs] add intro vignette (#3946) @jameslamb (#4775)
- [docs] fix broken SynapseML link @StrikerRUS (#4795)
- [docs] [dask] Add return information to Dask fit() docs (fixes #4402) @jameslamb (#4716)
- [docs] add specific estimator names to sklearn fit() docs @jameslamb (#4774)
- [docs] [R-package] update cran-comments for v3.3.1 release @jameslamb (#4738)
- [ci] only use conda-forge when installing R packages in docs builds @jameslamb (#4767)
- [docs] improve docs about `nthreads` parameter @StrikerRUS (#4756)
- [docs] update link to LightGBM Ruby @StrikerRUS (#4740)
- [docs] fix broken SynapseML link @jameslamb (#4712)

## 🧰 Maintenance

- [R-package] [tests] remove uses of testthat::context() @jameslamb (#4915)
- [R-package][tests] remove uses of `testthat::expect_is()` @jameslamb (#4916)
- [R-package] remove unnecessary comments and fix typos in comments @jameslamb (#4902)
- [R-package] reduce verbosity in tests using lgb.train() @jameslamb (#4896)
- [tests][python-package] change boston dataset to synthetic dataset in tests that don't check score @jmoralez (#4895)
- [R-package] reduce verbosity in some unit tests @jameslamb (#4879)
- [R-package] remove unused callback cb.reset.parameter @jameslamb (#4871)
- [python-package][dask] handle failures parsing worker host names @jameslamb (#4852)
- [python-package] [dask] fix mypy error about worker_addresses @jameslamb (#4851)
- [python] remove workaround for `UnboundLocalError` in early stopping callback for Python 2 @StrikerRUS (#4855)
- [python-package] remove unused imports @jameslamb (#4850)
- [python-package] fix mypy errors in engine.py @jameslamb (#4839)
- [python-package] [dask] fix mypy errors from dask.py docstrings @jameslamb (#4844)
- [python-package] fix mypy errors in sklearn.py @jameslamb (#4837)
- [tests][dask] fix argument names in custom eval function in Dask test @StrikerRUS (#4833)
- [python-package] fix mypy errors in plotting.py @jameslamb (#4838)
- [python-package] fix mypy error about missing type hint in dask.py @jameslamb (#4840)
- [python-package] fix mypy errors in compat.py and setup.py @jameslamb (#4836)
- [docs][python] simplify mocking in docs @StrikerRUS (#4830)
- [python] add type hints to `_compare_params_for_warning()` and make it reusable @StrikerRUS (#4824)
- [R-package] Move R6 to Imports @david-cortes (#4812)
- [ci] Fix CI job for R artifact @StrikerRUS (#4811)
- [ci] fix getting status of optional workflows in PRs with a lot of comments @StrikerRUS (#4806)
- [python] add type hints for custom objective and metric functions in scikit-learn interface @jameslamb (#4547)
- [c_api] Improve ANSI compatibility by avoiding <stdbool.h> @drewmiller (#4697)
- [ci] ignore CMakeLint errors related to package names @StrikerRUS (#4801)
- [ci] fix CMakeLint errors related to function naming case @StrikerRUS (#4794)
- [ci] simplify processing of `CMAKE_CUDA_FLAGS` variable in `CMakeLists.txt` @StrikerRUS (#4799)
- [ci] fix CMakeLint `linelength` errors @StrikerRUS (#4796)
- [ci] use pure `endif()` and `endfunction()` without expression inside in `CMakeLists.txt` @StrikerRUS (#4798)
- cmake: use object library to avoid duplicate compilation. @cyfdecyf (#4489)
- Suppress categorical warning (fixes #3379) @hzy46 (#4768)
- [ci][tests][python] remove assertion for `filename` that is no longer true with new version of graphviz @StrikerRUS (#4778)
- [ci] Revert temp workaround for `freetype` version @StrikerRUS (#4776)
- [ci] increase timeout for valgrind job to 240 minutes @jameslamb (#4773)
- [python] improve warning message about aliases in `cv()` function @StrikerRUS (#4766)
- [ci] Temporary pin dask version at CI @StrikerRUS (#4770)
- [ci] use wch1/r-debug image in Solaris tests @jameslamb (#4765)
- [ci] upgrade actions/checkout to v2.4.0 @jameslamb (#4763)
- [docs][python] add `datatable` to the mocked modules during docs building process and sort them alphabetically @StrikerRUS (#4750)
- [R-package] allow use of custom R executable when building CRAN package @jameslamb (#4754)
- [R-package] remove temporary files created in configure.win @jameslamb (#4752)
- [ci] indicate support of Monterey @StrikerRUS (#4732)
- [python] Make dummy classes constructible with any arguments @StrikerRUS (#4749)
- [ci] Temporary pin freetype version at Windows CI @StrikerRUS (#4747)
- Remove checks for label when loading dataset from binary file because label is ignored in that case @StrikerRUS (#4737)
- [tests][python] add test for non-serializable callback @StrikerRUS (#4741)
- Improve warning wordings @StrikerRUS (#4731)
- [ci] Bump version for development @StrikerRUS (#4730)
- Add some warnings when loading dataset from binary file @StrikerRUS (#4724)
changelog from when v3.3.2 was created to date of v3.3.3 (click me)

NOTE: I wrapped this in a markdown text block so GitHub stops re-evaluating the links.

## Changes

## 💡 New Features

- [CUDA] Add multiclass objective for cuda_exp @shiyu1994 (#5473)
- [CUDA] Add feature interaction constraint for cuda_exp (fix #4785) @shiyu1994 (#5474)
- [CUDA] Add rank_xendcg objective for cuda_exp @shiyu1994 (#5472)
- [CUDA] Add fair regression objective for cuda_exp @shiyu1994 (#5469)
- [CUDA] Add lambdarank objective for cuda_exp @shiyu1994 (#5453)
- [CUDA] Add Huber regression objective for cuda_exp @shiyu1994 (#5462)
- [CUDA] Add L1 regression objective for cuda_exp @shiyu1994 (#5457)
- [CUDA] L2 regression objective for cuda_exp @shiyu1994 (#5452)
- [CUDA] Add binary objective for cuda_exp @shiyu1994 (#5425)
- [R-package] Add remainder of prediction funtions @david-cortes (#5312)
- [python-package] support saving and loading CVBooster (fixes #3556) @nyanp (#5160)
- feature: Add true streaming APIs to reduce client-side memory usage @svotaw (#5299)
- [python-package] highlight the path a sample takes through a tree in `plot_tree` and `create_tree_digraph` (fixes #4784) @jmoralez (#5119)
- reproducible parameter alias resolution for wrappers (fixes #5304) @jmoralez (#5338)
- [CUDA] Initial work for boosting and evaluation with CUDA @shiyu1994 (#5279)
- [python-package] add validate_features argument to refit @jmoralez (#5331)
- [python-package] check feature names in predict with dataframe (fixes #812) @jmoralez (#4909)
- [R-package] Add sparse feature contribution predictions @david-cortes (#5108)
- [python-package][R-package] allow using feature names when retrieving number of bins @jmoralez (#5116)
- [R-package] Keep row names in output from `predict` @david-cortes (#4977)
- [python] make `reset_parameter` callback pickleable @StrikerRUS (#5109)
- [python] make `record_evaluation` callback pickleable @StrikerRUS (#5107)
- [python] make `log_evaluation` callback pickleable @StrikerRUS (#5101)
- [python] allow to register any custom logger (fixes #4783) @RustingSword (#4880)
- Load initial scores with binary data files in CLI version @shiyu1994 (#4807)
- [R-package] Rename `weight` -> `weights` @david-cortes (#4975)
- [CUDA] New CUDA version Part 1 @shiyu1994 (#4630)
- [python] make `early_stopping` callback pickleable @Yard1 (#5012)
- [c-api][python-package][R-package] expose feature num bin @jmoralez (#5048)
- [python-package] [R-package] propagate the best iteration of cvbooster into the individual boosters @jmoralez (#5066)
- [python-package] add support for pandas nullable types @jmoralez (#4927)
- [R-package] Promote objective and init_score to top-level arguments in `lightgbm()` @david-cortes (#4976)
- [python] Start supporting Python 3.10 @StrikerRUS (#4893)
- [python-package] support customizing Dataset creation in Booster.refit() (fixes #3038) @TremaMiguel (#4894)
- [dask] add support for custom objective functions (fixes #3934) @jameslamb (#4920)

## 🔨 Breaking

- fix: Adjust LGBM_DatasetCreateFromSampledColumn to handle distributed data @svotaw (#5344)
- [python-package] allow custom weighing in fobj for scikit-learn API (closes #5027) @jmoralez (#5211)
- [R-package] Use `type` argument to control prediction types @david-cortes (#5133)
- [python-package] Use scikit-learn interpretation of negative `n_jobs` and change default to number of cores @david-cortes (#5105)
- [python-package] remove Booster.set_attr() and Booster.attr() @jameslamb (#5272)
- remove support for Solaris (fixes #5216) @jameslamb (#5226)
- [R-package] stop automatically calculating eval metrics on training data in lightgbm() @jameslamb (#5209)
- [R-package] remove lgb.unloader() @jameslamb (#5204)
- [python-package] remove 'fobj' in favor of passing custom objective function in params @TremaMiguel (#5052)
- [python-package] remove is_reshape argument in Booster.predict (fixes #5115) @jmoralez (#5117)
- [R-package] Remove `reshape` argument in `predict` @david-cortes (#4971)
- [R-package] Promote number of threads to top-level argument in `lightgbm()` and change default to number of cores @david-cortes (#4972)
- [R-package] Rename `data` -> `newdata` in `predict` @david-cortes (#4973)
- Build Windows artifacts in `windows-2019` image instead of `vs2017-win2016` @StrikerRUS (#5059)
- [python-package] use 2d collections for predictions, grads and hess in multiclass custom objective @jmoralez (#4925)
- [R-package] prefer params to keyword argument in lgb.train() @jameslamb (#5007)
- [R-package] remove behavior where lightgbm() saves model to disk @david-cortes (#4974)
- [python-package] make record_evaluation compatible with cv (fixes #4943) @jmoralez (#4947)

## 🚀 Efficiency Improvement

- [python-package] simplify Dataset processing of label @jameslamb (#5456)
- [python-package] make a shallow copy on dataframe rename (fixes #4596) @jmoralez (#5254)
- [python-package] make a shallow copy when replacing categorical features with codes (fixes #4596) @jmoralez (#5225)
- [R-package] reduce cost of repeated parameter alias checks @jameslamb (#5141)
- reduce duplicate computation in poisson, gamma, and tweedie objectives @lorentzenchr (#4950)

## 🐛 Bug Fixes

- include parameters from reference dataset on subset (fixes #5402) @jmoralez (#5416)
- [python-package] ignore training set on early stopping callback (fixes #5354) @jmoralez (#5412)
- [fix] change the destructor of ScoreUpdater to virtual (fixes #5400) @shiyu1994 (#5403)
- Add default definition for GetColWiseData and GetColWiseData @shiyu1994 (#5413)
- Fix potential overflow in linear trees @StrikerRUS (#5395)
- Use double precision in threaded calculation of linear tree coefficients (fixes #5226) @btrotta (#5368)
- [R-package] raise an informative error when custom objective produces incorrect output (fixes #5323) @jmoralez (#5329)
- Clear split info buffer in cost efficient gradient boosting before every iteration (fix partially #3679) @shiyu1994 (#5164)
- [c++][fix] check nullable of bin mappers in dataset_loader.cpp (fix #5221) @shiyu1994 (#5258)
- [python] Fix training on subset constructed without params @StrikerRUS (#5213)
- Check existence of inet_pton for win32 in CMakeLists.txt (fixes #5019) @shiyu1994 (#5159)
- Fix potential overflow "Multiplication result converted to larger type" @StrikerRUS (#5189)
- [R-package] ensure that callbacks respect verbosity from params @jameslamb (#5199)
- fix precision lost in tree's ToIfElse @Grass-CLP (#5187)
- fix some wrong format specifiers @StrikerRUS (#5190)
- [R-package] allow use of categorical_features in Dataset when raw data does not have column names (fixes #4374) @jmoralez (#5184)
- [c-api] check number of features when retrieving number of bins @jmoralez (#5183)
- [CUDA] Fix integer overflow in cuda row-wise data @shiyu1994 (#5167)
- [R-package] ensure values in params override keyword arguments to predict() (fixes #4670) @jameslamb (#5122)
- check nullable of bin_mappers in DatasetLoader::CheckCategoricalFeatureNumBin (fix #5145) @shiyu1994 (#5146)
- [CUDA] Fix row-wise histogram construction with dense data matrix @shiyu1994 (#5103)
- [fix] fix duplicate added initial scores for single-leaf trees @shiyu1994 (#5050)
- [python] fixes for supporting 2d numpy arrays for predictions, grads and hess in multiclass custom objective and eval @StrikerRUS (#5030)
- CUDATreeLearner: free GPU memory in destructor if any allocated @denmoroz (#4963)
- Use `delete[]` where appropriate instead of `delete` @david-cortes (#4984)
- Pass train dataset parser config to valid dataset loading parser @chjinche (#4985)
- [R-package] Fix custom objective detection in `print.lgb.Booster()` @StrikerRUS (#4941)
- gpu allocate memory overflow (fixes #4926) @jiapengwen (#4928)

## 📖 Documentation

- [R-package] [docs] use CRAN canonical form for package links @jameslamb (#5504)
- [docs] Fix link to Optuna's LightGBMTuner example @knshnb (#5519)
- [docs]fix a typo in docs/Features.rst @JiantingFeng (#5463)
- Use ROC-AUC metric for classification model in examples @Green-16 (#5440)
- [docs] Fix links @StrikerRUS (#5451)
- [R-package] Fix docstrings for predict functions @StrikerRUS (#5444)
- [R-package] [docs] clarify shape of predictions @jameslamb (#5384)
- [python-package] Improve readme on experimental cuda usage @thomasaarholt (#5386)
- [docs] [R-package] document how to regenerate roxygen docs @jameslamb (#5382)
- [R-package] [docs] fix typo in documentation on init_model @sebffischer (#5379)
- [docs] [R-package] upgrade to roxygen2 7.2.1 @jameslamb (#5381)
- [ci] make check-docs job compatible with rstcheck 6.x @jameslamb (#5388)
- [docs][python] Fix return types in docstrings @StrikerRUS (#5326)
- [docs] set language = 'en' in Sphinx config @jameslamb (#5322)
- [R-package] [docs] document difference between lightgbm() and lgb.train() (fixes #5203) @jameslamb (#5273)
- [docs] Use https links in docs @StrikerRUS (#5284)
- [R-package] [docs] upgrade docs to roxygen2==7.2.0 @jameslamb (#5251)
- [docs] Update broken links in docs @StrikerRUS (#5265)
- [docs] add Microsoft security policy (fixes #5235) @jameslamb (#5264)
- [R-package] update roles in DESCRIPTION @StrikerRUS (#5192)
- [docs] Fix formula in path smoothing docs (fixes #5139) @samFarrellDay (#5154)
- [docs] Document behaviour of the first linear estimator @Pablo-Davila (#5132)
- [docs] use 'docker run --rm' in valgrind instructions @jameslamb (#5127)
- [docs] use 'docker run --rm' in autoconf instructions @jameslamb (#5125)
- [docs] Improve rendering of class properties in docs @StrikerRUS (#5078)
- [python-package][docs] Booster eval methods accept list of callables @TremaMiguel (#5054)
- [docs] update categorical feature description in Advanced Topics @StrikerRUS (#5044)
- Correct documentation for sparse predictions @david-cortes (#4979)
- [docs][R-package] Use direct link for the list of supported objectives @StrikerRUS (#5029)
- [Docs] Weights non-negative for train data @TremaMiguel (#5013)
- [docs] improve docs for sklearn wrapper @StrikerRUS (#5026)
- [docs] clarify that categorical features will be converted to integers internally @jmoralez (#4959)
- [docs] clarify that custom eval functions are not only used on training data @jameslamb (#5011)
- [docs] document rounding behavior of floating point numbers in categorical features @StrikerRUS (#5009)
- Change docs for feval @akshitadixit (#5002)
- [docs] document `conda-forge` channel preference over `default` one and describe possible workaround for OpenMP conflicts in FAQ @StrikerRUS (#4994)
- Add Nyoka to README @StrikerRUS (#4992)
- [docs] minor docs improvements @StrikerRUS (#4938)

## 🧰 Maintenance

- [ci] run Appveyor checks on PRs targeting release/ branches @jameslamb (#5528)
- [ci] run CI on pull requests targeting release/ branches @jameslamb (#5527)
- [python-package] prefix is_numpy_1d_array with _ @Madnex (#5520)
- replace `pandas.Series.iteritems` with `pandas.Series.items` @jmoralez (#5506)
- [ci] prefer CPython in Windows test environment and use safer approach for cleaning up network (fixes #5509) @jameslamb (#5510)
- [ci] test against R 4.2 for macOS and Linux CI jobs @jameslamb (#5484)
- [R-package] improve safety of index selection in plotting @jameslamb (#5485)
- [ci] [R-package] ensure that MSVC jobs fail when tests fail (fixes #5439) @jameslamb (#5448)
- [python-package] add more hints in sklearn.py @jameslamb (#5460)
- [R-package] [ci] restore R 3.6 Windows cran CI job (fixes #5036) @jameslamb (#5479)
- Remove redundant whitespaces @ch3rn0v (#5480)
- fix references to 'object function' @jameslamb (#5468)
- [python] Fix typehints in Sequence API @StrikerRUS (#5465)
- fix: Revert leftover debugging in streaming test @svotaw (#5467)
- Fix CUDA `#ifndef` guards @StrikerRUS (#5466)
- [python-package] add type hints on Dataset constructors @jameslamb (#5458)
- Add streaming concurrency tests @svotaw (#5437)
- Rename Metadata num_classes to be more clear @svotaw (#5461)
- [R-package] fix function references in error messages @jameslamb (#5455)
- [python-package] fix type hints on ctypes array converters @jameslamb (#5446)
- [python-package] fix mypy errors about callbacks @jameslamb (#5450)
- [python-package] add type hints on empty initializations @jameslamb (#5445)
- [python-package] fix mypy error about type change in Dataset.feature_num_bin() @jameslamb (#5447)
- [ci][fix] Fix cuda_exp ci @shiyu1994 (#5438)
- [python] remove unused import @StrikerRUS (#5443)
- [ci] bump CUDA version from `11.7.0` to `11.7.1` at CI @StrikerRUS (#5442)
- [python-package] add type hints on Booster eval methods @jameslamb (#5433)
- update tree to if-else @jmoralez (#5422)
- [ci] bump CUDA version from `11.6.2` to `11.7.0` at CI  @StrikerRUS (#5287)
- [python-package] add more type hints on Dataset @jameslamb (#5431)
- [python-package] add more type hints on Booster @jameslamb (#5432)
- [ci] increase valgrind bytes lost limit to 352 @jameslamb (#5429)
- [python] Prefix `basic.is_numeric()` with _ @ak04p (#5421)
- [ci] increase valgrind timeout to 5 hours @jameslamb (#5414)
- [python-package] add more type hints in basic.py @jameslamb (#5407)
- [python-package] prefix NUMERIC_TYPES with _ to make it a internal object @thomasjpfan (#5409)
- [python-package] add type hints on Booster.save_model() @jameslamb (#5406)
- Minor CUDA cleanup @StrikerRUS (#5394)
- [ci] temporarily pin scipy version and fix Python linting error @StrikerRUS (#5398)
- [python-package] add type hints on some functions in basic.py @makquel (#5362)
- [python-package] add more type hints on Booster @jameslamb (#5360)
- [python-package] make library-loading stricter @jameslamb (#5357)
- [python-package] add type hints on Booster.update() @jameslamb (#5359)
- [python-package] add more type hints on basic.py @jameslamb (#5356)
- [python-package] add type hints on Dataset @jameslamb (#5353)
- [python-package] remove inner function _construct_dataset() in LGBMModel.fiit() @jameslamb (#5333)
- [python-package] add type hints on predict() methods @jameslamb (#5334)
- [python-package] add more type hints in basic.py @jameslamb (#5330)
- [ci][R-package] Minor refactoring for `lintr` code @StrikerRUS (#5327)
- [python-package] add more type hints in engine.py @jameslamb (#5301)
- [ci] [R-package] Add string_boundary_linter @CuriousCorrelation (#5324)
- [python-package] add more type hints on Booster @jameslamb (#5309)
- [ci] [R-package] Add paste_linter @CuriousCorrelation (#5320)
- [ci] [R-package] Add duplicate_argument_linter @SaumyaBhushan (#5310)
- [ci] [R-package] Add a few more linters @CuriousCorrelation (#5311)
- [ci] [R-package] Add class_equals linter @CuriousCorrelation (#5307)
- [python][sklearn] Simplify params handling in `predict()` with `_choose_param_value()` @StrikerRUS (#5308)
- [ci] [R-package] Add any_is_na_linter @CuriousCorrelation (#5306)
- [python-package] add some type hints on Booster @jameslamb (#5302)
- [python] preserve None in _choose_param_value() @jameslamb (#5289)
- [ci] Install PoCL in Docker for old Ubuntu 14.04 @StrikerRUS (#5286)
- [R-package] Specify concrete linting rule that is being ignored via `nolint` mark @StrikerRUS (#5300)
- [python-package] add type hints on cv() @jameslamb (#5271)
- [ci] [R-package] upgrade to lintr v3.0 (fixes #5228) @jameslamb (#5294)
- [ci] Pin lintr to <3.0 @jgiannuzzi (#5290)
- [ci] Run Linux OpenCL tests against POCL instead of the AMD App SDK @jgiannuzzi (#5282)
- [tests][python] Make test that checks original pandas data isn't modified more strict @StrikerRUS (#5267)
- [python] add more type hints in basic.py @jameslamb (#5255)
- [python] add more type hints on LGBMModel methods @jameslamb (#5239)
- [R-package] address linter warnings about portable paths @jameslamb (#5249)
- [R-package] [ci] silence more logs in tests (fixes #4862) @jameslamb (#5250)
- Remove leftovers after the drop of Solaris support @StrikerRUS (#5248)
- [python] add type hints on sklearn metric and eval wrappers @jameslamb (#5238)
- [R-package] silence logs in print(), show(), summary() tests @jameslamb (#5237)
- [R-package] standardize style for placement of braces @jameslamb (#5240)
- [R-package] remove semicolon in R code @jameslamb (#5232)
- [R-package] remove Solaris reference in test message @jameslamb (#5231)
- [R-package] silence more logs in unit tests @jameslamb (#5227)
- [ci] switch CRAN mirror to RStudio @jameslamb (#5230)
- Cleanup codeowners @StrikerRUS (#5215)
- [R-package] minor improvements on weight demo @jmoralez (#5212)
- [ci] fix R Hub token @jameslamb (#5210)
- [R-package] silence more logs in tests @jameslamb (#5208)
- [ci] Prevent Python downgrading to pypy on Windows (2) @StrikerRUS (#5198)
- [ci] Prevent Python downgrading to pypy on Windows @StrikerRUS (#5197)
- [ci] fix git checkout for comment-triggered CI jobs @jameslamb (#5169)
- [python] remove unused import from `basic.py` @StrikerRUS (#5188)
- [tests] replace `fobj` with `custom objective` in test comments and make tests stricter @StrikerRUS (#5173)
- [ci] Update version of Azure REST API @StrikerRUS (#5172)
- [ci] skip auto-injected tasks at Azure Pipelines @StrikerRUS (#5175)
- fix typo in CEGB method name @jameslamb (#5168)
- [ci] temporarily pin Azure Devops jobs to ubuntu 20.04 @jameslamb (#5174)
- [ci] fix git checkout for CI jobs (fixes #5151) @jameslamb (#5152)
- [ci] bump CUDA version from `11.5.1` to `11.6.2` at CI @StrikerRUS (#5149)
- [python] Use predefined constant in feature importance type comparison instead of raw int literal @StrikerRUS (#5148)
- [ci] update to R 4.1.3 and use macOS-latest for R jobs (fixes #4990) @jameslamb (#5137)
- [ci] have no-response bot post as github-actions user @jameslamb (#5136)
- [ci] fail R macOS CI jobs earlier when installations fail @jameslamb (#5129)
- [ci] use lee-dohm/no-response to close stale issues (fixes #5060) @jameslamb (#5120)
- [ci] Remove temp workaround for graphviz installation @StrikerRUS (#5126)
- [R-package] rename internal callback functions @mayer79 (#5123)
- [R-package] ensure boosting happens in tests on small datasets @jameslamb (#5121)
- [R-package] Add missed packages into dependencies list @StrikerRUS (#5118)
- [R-package] add assertions to test on lightgbm() weights @jameslamb (#5110)
- Log warnings for number of bins of categorical features @shiyu1994 (#4448)
- clarify no-meaningful-features warning in Dataset construction (fixes #5081) @jameslamb (#5083)
- [ci] Add text file with GitHub commit hash to nightly builds @StrikerRUS (#5082)
- [ci] remove Visual Studio 2017 CI job for R-package @StrikerRUS (#5079)
- [python-package] Better column dtype logging when column has "bad dtype" @hsorsky (#5065)
- [ci] update CODEOWNERS @jameslamb (#5063)
- [ci] fix current `master` fails with graphviz-related error @StrikerRUS (#5068)
- [ci] temporarily remove R3.6 CRAN CI build @jameslamb (#5049)
- [tests][python] move tests that use `train()` function defined in `engine.py` from `test_basic.py` to `test_engine.py` @StrikerRUS (#5034)
- [R-package] remove internal function lgb.check.obj() @jameslamb (#5021)
- [ci] use Python 3.8 for `bdist` CI job in old Ubuntu container @StrikerRUS (#5022)
- [R-package] raise informative errors directly when Booster creation fails @jameslamb (#5014)
- [ci] [docs] use miniforge for readthedocs builds (fixes #4954) @jameslamb (#4957)
- [ci] Enclose Python versions in quotes in config yml files @StrikerRUS (#5015)
- [python] enforce a floor of Python 3.6 (fixes #5004) @PyVCEchecker (#5006)
- Update versions of LightGBM dependencies @StrikerRUS (#4935)
- [tests][python] remove compatibility code for old versions in tests @StrikerRUS (#4978)
- [ci] use conda-forge in Windows CI jobs and Docker images @StrikerRUS (#4993)
- [ci] use `latest` tag again for Ubuntu 14 CI Docker @StrikerRUS (#5001)
- [ci] use conda-forge in Linux and macOS CI jobs @jameslamb (#4953)
- [ci] upgrade to R 4.1.2 in CI, change approach to macOS-latest R CI builds (fixes #4988) @jameslamb (#4989)
- [ci] bump CUDA version from `11.5.0` to `11.5.1` at CI @StrikerRUS (#4967)
- [ci] use `-1` for exit code in case of failure @StrikerRUS (#4939)
- [tests][R-package] use one `#` symbol for comments @StrikerRUS (#4940)
- [ci] bump CUDA version from 11.4.2 to 11.5.0 at CI @StrikerRUS (#4937)
- [ci] Bump version for development @StrikerRUS (#4933)
changelog from when v3.3.3 was created to today (click me)

NOTE: I wrapped this in a markdown text block so GitHub stops re-evaluating the links.

## Changes

## 💡 New Features

- [CUDA] Add binary logloss metric for new CUDA version @shiyu1994 (#5635)
- Decouple Boosting Types (fixes #3128) @lyf-00 (#4827)
- [CUDA] Add L2 metric for new CUDA version @shiyu1994 (#5633)
- [CUDA] Add rmse metric for new CUDA version @shiyu1994 (#5611)
- Build integrated OpenCL Linux wheels @jgiannuzzi (#5252)
- [CUDA] Add Poisson regression objective for cuda_exp and refactor objective functions for cuda_exp @shiyu1994 (#5486)
- [CUDA] Add multiclass_ova objective for cuda_exp @shiyu1994 (#5491)
- [python-package] add install option to enable printing of time costs @Remy-Luciani (#5497)
- [python-package][R-package] load parameters from model file (fixes #2613) @jmoralez (#5424)

## 🔨 Breaking

- [ci] migrate CI from macOS 10.15 to 11 (fixes #5391) @StrikerRUS (#5396)
- [ci] switch to manylinux_2_28 for Linux artifacts (fixes #5514, fixes #5589) @jameslamb (#5580)

## 🚀 Efficiency Improvement

- [python-package] replace .values usage with .to_numpy() @superlaut (#5612)

## 🐛 Bug Fixes

- Check feature indexes in forced split file (fixes #5517) @btrotta (#5653)
- fix feature index in Dataset::AddFeaturesFrom (fixes #5410) @jameslamb (#5650)
- [ci] [python-package] fix missing import, test that lightgbm can be imported with only required dependencies (fixes #5631) @jameslamb (#5632)
- Fix OpenMP thread allocation in Linux @svotaw (#5551)
- [R-package] correctly quote paths on Windows for CMake-based builds @jameslamb (#5607)
- [ci] [python-package] correct tag on x86_64 wheels @jameslamb (#5598)
- [tests][dask] fix workers without data test (fixes #5537) @jmoralez (#5544)
- prefer 'vsnprintf' to 'vsprintf' @jameslamb (#5561)
- [ci] fix R-package CI jobs and compatibility with OpenMP 15+ (fixes #5549, #5562) @jameslamb (#5563)

## 📖 Documentation

- [docs] update CPU dockerfiles (fixes #5550) @jameslamb (#5601)
- [docs] update cran-comments.md for v3.3.4 release @jameslamb (#5646)
- [docs] add postgresml to list of external repos @jameslamb (#5565)
- [docs] Improve docs: fix consistency of dots in C API and add notes about new ``time-costs`` Python-package build option @StrikerRUS (#5554)

## 🧰 Maintenance

- [ci][dask][gpu] Run Dask tests with LightGBM GPU version @jgiannuzzi (#5292)
- [python-package] prefix internal objects with '_' @jameslamb (#5654)
- [ci] speed up Windows jobs (fixes #5647) @jameslamb (#5648)
- [ci] automatically cancel GitHub Actions runs for outdated commits @jameslamb (#5651)
- [ci] use LightGBM CI image for building aarch64 wheels (fixes #5595) @jameslamb (#5622)
- [ci] allow ubuntu-latest to float for some GitHub Actions jobs @jameslamb (#5644)
- [ci] fix locale-setting in jobs running in ubuntu container @jameslamb (#5643)
- [ci] make GitHub Actions branch protection stricter (fixes #5501) @jameslamb (#5645)
- [ci] Use Ubuntu 22.04 as `ubuntu-latest` at CI (fixes #5186) @StrikerRUS (#5288)
- [tests][python-package] remove remaining tests using load_boston (fixes #4793) @jmoralez (#5581)
- [python-package] prefix c_int_array and c_float_array with _ @OmarManzoor (#5614)
- [ci] test against R 4.2.2 @jameslamb (#5621)
- [python-package] prefix basic.convert_from_sliced_object with _ @OmarManzoor (#5613)
- [ci] detect non-default dynamic symbols in check_dynamic_dependencies.py @jameslamb (#5610)
- [ci] add docs and specific error messages in check_dynamic_dependencies.py @jameslamb (#5582)
- [ci] update versions of GitHub Actions @jameslamb (#5590)
- [swig] switch to SWIG_ADD_LIBRARY() in CMakeLists.txt (fixes #5586) @shogohida (#5603)
- [ci] switch from MiKTeX to tinytex on Windows R jobs (fixes #5600) @jameslamb (#5602)
- [python-package] prefix several internal functions with _ @Madnex (#5545)
- [python-package] remove unused imports @jameslamb (#5559)
- [ci] prefer CPython in Linux and macOS test environment @StrikerRUS (#5555)
- [docs] bump development version to 3.3.3.99 @jameslamb (#5533)
- [ci] upgrade macOS_regular job to Python 3.9 (fixes #5569) @jameslamb (#5570)
- [tests] Fix cpp streaming data tests @StrikerRUS (#5481)
- [python-package] prefix is_numpy_column_array with _ @Madnex (#5531)
- renamed cur_cat => cur_cat_idx and added some comments @zyxue (#5522)
- [python-package] prefix cast_numpy_array_to_dtype with _ @Madnex (#5532)
- suppress alias warnings with verbosity<0 (fixes #4518) @jmoralez (#5253)
- renamed tmp_num_sample_values to non_na_cnt @zyxue (#5521)

@jameslamb
Copy link
Collaborator Author

/gha run r-configure

@jameslamb jameslamb removed the request for review from Laurae2 December 6, 2022 01:53
@@ -7,6 +7,7 @@ if [[ $OS_NAME == "macos" ]]; then
sudo xcode-select -s /Applications/Xcode_10.3.app/Contents/Developer || exit -1
fi
else # gcc
sudo xcode-select -s /Applications/Xcode_14.1.app/Contents/Developer || exit -1
Copy link
Collaborator Author

@jameslamb jameslamb Dec 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had to pull in this change from #5580 for GitHub Actions and Azure CI jobs on macOS-latest to work

@jameslamb
Copy link
Collaborator Author

jameslamb commented Dec 14, 2022

/gha run r-valgrind

Workflow R valgrind tests has been triggered! 🚀
https://github.com/microsoft/LightGBM/actions/runs/3698602005

Status: success ✔️.

@jameslamb
Copy link
Collaborator Author

Since all of the R tests are passing on this branch, and since we have less than 2 days left to get a new CRAN submission up, I've submitted the R package built from this branch to CRAN.

@shiyu1994 please check your email and click the confirmation link there.

image


I'll continue trying to get this branch building successfully so we can do a full release of the project (e.g. to PyPI, NuGet, GitHub releases page). But the CRAN update is the main reason for this, and the most urgent thing to get done.

And also I'm traveling tomorrow night 😅

@jameslamb
Copy link
Collaborator Author

ok I think this is ready for review! We can ignore the linting and cuda_exp jobs failing...cuda_exp didn't exist this far back on master, and I didn't want to add linting-only changes into this release.

@shiyu1994 @guolinke @jmoralez can you please review? If you all approve, I'll cut a release, push wheels to PyPI, etc. and then we can fully focus on v4.0.0

@shiyu1994
Copy link
Collaborator

@jameslamb Hi James, thanks for working on this. I got another email from CRAN on Jan 1st. We need to fix a compilation warning before Jan 15th (see #5661). Do you think we should fix that along with this PR.

@jameslamb
Copy link
Collaborator Author

Do you think we should fix that along with this PR.

@shiyu1994 no, I think it should be a new PR and a v3.3.5 release. Since we've already published the state of this PR to CRAN as v3.3.4.

Could you approve this PR if you're comfortable with releasing this branch? We still haven't pushed a v3.3.4 to PyPI, GitHub Releases, etc.... I can do that if you approve, then start on a v3.3.5.

Copy link
Collaborator

@shiyu1994 shiyu1994 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shiyu1994
Copy link
Collaborator

@jameslamb Thanks for your help! Feel free to contact me at anytime if there's something I can do.

@jameslamb
Copy link
Collaborator Author

Thank you! I'm going to do the following then:

  1. finish the release checklist items for v3.3.4
  2. close (but never merge) this PR
  3. investigate [R-package] CRAN issue with unqualified call to 'std::move' #5661 and put up a PR against master
  4. open a v3.3.5 release PR with those same changes to fix [R-package] CRAN issue with unqualified call to 'std::move' #5661

@jameslamb
Copy link
Collaborator Author

jameslamb commented Jan 4, 2023

Alright, continuing on this release! I'll continue editing this comment as things progress, so we have a single comment to link to with more details on what to do in the upcoming v3.3.5 release.

  1. copied the latest changelog notes from https://github.com/microsoft/LightGBM/releases up into the description of this PR, so we can produce an accurate changelog for v4.0.0

  2. pushed a new tag

git tag v3.3.4
git push upstream v3.3.4

(I use upstream as the git remote alias for LightGBM, and origin for my fork)

  1. triggered by that tag, Azure DevOps successfully created a release here on GitHub and attached all artifacts to it 🎉

(build link)

image

  1. edited the automatically-created v3.3.4 release, and published it (https://github.com/microsoft/LightGBM/releases/tag/v3.3.4)

  2. updated the stable tag to point to the same commit as v3.3.4

git fetch upstream --tags
git tag -d stable
git push upstream :refs/tags/stable
git tag stable v3.3.4
git push upstream stable

https://github.com/microsoft/LightGBM/tags

Screen Shot 2023-01-04 at 4 14 14 PM

  1. added v3.3.4 version on readthedocs

(build link)

Screen Shot 2023-01-04 at 4 34 11 PM

  1. uploaded v3.3.4 to PyPI

see https://pypi.org/project/lightgbm/

image

downloaded the files from the release, then uploaded them like this:

twine upload -u ${PYPI_USER} dist/*
  1. someone else put up a PR for Homebrew: lightgbm 3.3.4 Homebrew/homebrew-core#119801
  2. uploaded the package to NuGet

image

@jameslamb
Copy link
Collaborator Author

release v3.3.4 is now publicly available: https://github.com/microsoft/LightGBM/releases/tag/v3.3.4

@jameslamb jameslamb closed this Jan 4, 2023
@jameslamb
Copy link
Collaborator Author

This work is now complete. Thanks for the help everyone!

@jameslamb jameslamb deleted the release/v3.3.4 branch January 5, 2023 06:25
@jameslamb jameslamb mentioned this pull request Jan 7, 2023
13 tasks
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants