[jvm-package] refactor evaluation of xgboost4j-spark #1852

merged 2 commits into from Dec 9, 2016


None yet

3 participants


I refactor the eval method in XGBoostModel. There are some redundant code of two approaches, I use match pattern to do the same thing in less code.

codecov-io commented Dec 8, 2016 edited

Current coverage is 33.85% (diff: 100%)

Merging #1852 into master will decrease coverage by 0.01%

@@             master      #1852   diff @@
  Files            68         68          
  Lines          5281       5281          
  Methods         827        827          
  Messages          0          0          
  Branches        526        526          
- Hits           1789       1788     -1   
  Misses         3415       3415          
- Partials         77         78     +1   

Powered by Codecov. Last update 2b6aa77...01aa68c

fromradio and others added some commits Dec 8, 2016
@fromradio fromradio refactor duplicate evaluation implementation
@CodingCat CodingCat Merge branch 'master' into master

LGTM, will merge after travis done

@CodingCat CodingCat merged commit d9584ab into dmlc:master Dec 9, 2016

4 checks passed

codecov/patch Coverage not affected when comparing 2b6aa77...01aa68c
codecov/project Absolute coverage decreased by -0.01% but relative coverage increased by +66.12% compared to 2b6aa77
continuous-integration/appveyor/pr AppVeyor build succeeded
continuous-integration/travis-ci/pr The Travis CI build passed
@Laurae2 Laurae2 added a commit to Laurae2/xgboost that referenced this pull request Dec 11, 2016
@Laurae2 Laurae2 R json dump (#1)
* [jvm-packages] Scala implementation of the Rabit tracker. (#1612)

* [jvm-packages] Scala implementation of the Rabit tracker.

A Scala implementation of RabitTracker that is interface-interchangable with the
Java implementation, ported from `tracker.py` in the
[dmlc-core project](https://github.com/dmlc/dmlc-core).

* [jvm-packages] Updated Akka dependency in pom.xml.

* Refactored the RabitTracker directory structure.

* Fixed premature stopping of connection handler.

Added a new finite state "AwaitingPortNumber" to explicitly wait for the
worker to send the port, and close the connection. Stopping the actor
prematurely sends a TCP RST to the worker, causing the worker to crash
on AssertionError.

* Added interface IRabitTracker so that user can switch implementations.

* Default timeout duration changes.

* Dependency for Akka tests.

* Removed the main function of RabitTracker.

* A skeleton for testing Akka-based Rabit tracker.

* waitFor() in RabitTracker no longer throws exceptions.

* Completed unit test for the 'start' command of Rabit tracker.

* Preliminary support for Rabit Allreduce via JNI (no prepare function support yet.)

* Fixed the default timeout duration.

* Use Java container to avoid serialization issues due to intermediate wrappers.

* Added tests for Allreduce/model training using Scala Rabit tracker.

* Added spill-over unit test for the Scala Rabit tracker.

* Fixed a typo.

* Overhaul of RabitTracker interface per code review.

  - Removed methods start() waitFor() (no arguments) from IRabitTracker.
  - The timeout in start(timeout) is now worker connection timeout, as tcp
    socket binding timeout is less intuitive.
  - Dropped time unit from start(...) and waitFor(...) methods; the default
    time unit is millisecond.
  - Moved random port number generation into the RabitTrackerHandler.
  - Moved all Rabit-related classes to package ml.dmlc.xgboost4j.scala.rabit.

* More code refactoring and comments.

* Unified timeout constants. Readable tracker status code.

* Add comments to indicate that allReduce is for tests only. Removed all other variants.

* Removed unused imports.

* Simplified signatures of training methods.

 - Moved TrackerConf into parameter map.
 - Changed GeneralParams so that TrackerConf becomes a standalone parameter.
 - Updated test cases accordingly.

* Changed monitoring strategies.

* Reverted monitoring changes.

* Update test case for Rabit AllReduce.

* Mix in UncaughtExceptionHandler into IRabitTracker to prevent tracker from hanging due to exceptions thrown by workers.

* More comprehensive test cases for exception handling and worker connection timeout.

* Handle executor loss due to unknown cause: the newly spawned executor will attempt to connect to the tracker. Interrupt tracker in such case.

* Per code-review, removed training timeout from TrackerConf. Timeout logic must be implemented explicitly and externally in the driver code.

* Reverted scalastyle-config changes.

* Visibility scope change. Interface tweaks.

* Use match pattern to handle tracker_conf parameter.

* Minor clarification in JNI code.

* Clearer intent in match pattern to suppress warnings.

* Removed Future from constructor. Block in start() and waitFor() instead.

* Revert inadvertent comment changes.

* Removed debugging information.

* Updated test cases that are a bit finicky.

* Added comments on the reasoning behind the unit tests for testing Rabit tracker robustness.

* Add benchmarks, fix GCC build (#1848)

* refactor duplicate evaluation implementation (#1852)

* [R-package] JSON tree dump interface

* [R-package] precision bugfix in xgb.attributes

* [R-package] bugfix for cb.early.stop called from xgb.cv

* [R-package] a bit more clarity on labels checking in xgb.cv

* [R-package] test JSON dump for gblinear as well

* whitespace lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment