[Ray] Fix web serialize lambda #2512
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
qinxuye
added
to be backported
qinxuye
pushed a commit
to qinxuye/mars
that referenced
this pull request
Oct 15, 2021
(cherry picked from commit 3b61957)
wjsi
pushed a commit
that referenced
this pull request
Oct 15, 2021
qinxuye
added
backported already
chaokunyang
added a commit
to chaokunyang/mars
that referenced
this pull request
May 31, 2022
Merge branch merge_github_2524 of git@gitlab.alipay-inc.com:ray-project/mars.git into master https://code.alipay.com/ray-project/mars/pull_requests/58?tab=diff Signed-off-by: 捕牛 <hejialing.hjl@antgroup.com> * [Ray] Support reconstructing worker (mars-project#2413) * Make cmdline support third party modules (mars-project#2454) Co-authored-by: hanguang <zhusiyuan.zsy@alibaba-inc.com> * Support visualizing subtask graphs on Mars Web (mars-project#2426) * Fix timeout error when waiting for a submitted task (mars-project#2457) * Print the error message when error happens in `TaskProcessor` (mars-project#2458) * Add nightly builds for docker images (mars-project#2456) * Fix misuse of `name` parameter in DataFrame align (mars-project#2469) * Fix hang when start sub pool fails (mars-project#2468) * Refine and unify subtask detail APIs (mars-project#2465) * Fix coverage for Azure pipeline (mars-project#2474) * Split tileable information and subtask graph into two tabs (mars-project#2480) * Support specified vineyard socket and skip the launching vineyardd process (mars-project#2481) * Basic reschedule subtask (mars-project#2467) * Compatible with scikit-learn 1.0 (mars-project#2486) Co-authored-by: hekaisheng <kaisheng.hks@alibaba-inc.com> * Fix wrong translation in cluster deployment. (mars-project#2489) * Fix bug that failed to execute query when there are multiple arguments (mars-project#2490) * Include tileable property in detail api (mars-project#2493) * Fix version of statsmodels to pass CI (mars-project#2497) * Implements `glm.LogisticRegression` (mars-project#2466) * Implements bagging sampling (mars-project#2496) * Refine MarsDMatrix & support more parameters for XGB classifier and regressor (mars-project#2498) * Fix output of df.groupby(as_index=False).size() (mars-project#2507) * Add preliminary implementations for ufunc methods (mars-project#2510) * Add doc for reading csv in oss (mars-project#2514) * [Ray] Fix serializing lambdas in web (mars-project#2512) * Add `make_regression` support for learn module (mars-project#2515) * Fix reduction result on empty series (mars-project#2520) * Fix df.loc when df is empty (mars-project#2524) * fix start subpool * fix test_kill_and_wait_timeout * fix autoscale timeout * fix ray larger clsuter fixture * Update ci ray package to 1.2.2 * remove python3.6 3.8 .39 ut and upgrade ray 3.7 image * echo python path * fix json decode error * fix bundle release timeout * fix remove placement group timeout * fix no_restart * fix ci * fix autoscale
chaokunyang
added a commit
to chaokunyang/mars
that referenced
this pull request
May 31, 2022
Merge branch merge_github_2524 of git@gitlab.alipay-inc.com:ray-project/mars.git into master https://code.alipay.com/ray-project/mars/pull_requests/58?tab=diff Signed-off-by: 捕牛 <hejialing.hjl@antgroup.com> * [Ray] Support reconstructing worker (mars-project#2413) * Make cmdline support third party modules (mars-project#2454) Co-authored-by: hanguang <zhusiyuan.zsy@alibaba-inc.com> * Support visualizing subtask graphs on Mars Web (mars-project#2426) * Fix timeout error when waiting for a submitted task (mars-project#2457) * Print the error message when error happens in `TaskProcessor` (mars-project#2458) * Add nightly builds for docker images (mars-project#2456) * Fix misuse of `name` parameter in DataFrame align (mars-project#2469) * Fix hang when start sub pool fails (mars-project#2468) * Refine and unify subtask detail APIs (mars-project#2465) * Fix coverage for Azure pipeline (mars-project#2474) * Split tileable information and subtask graph into two tabs (mars-project#2480) * Support specified vineyard socket and skip the launching vineyardd process (mars-project#2481) * Basic reschedule subtask (mars-project#2467) * Compatible with scikit-learn 1.0 (mars-project#2486) Co-authored-by: hekaisheng <kaisheng.hks@alibaba-inc.com> * Fix wrong translation in cluster deployment. (mars-project#2489) * Fix bug that failed to execute query when there are multiple arguments (mars-project#2490) * Include tileable property in detail api (mars-project#2493) * Fix version of statsmodels to pass CI (mars-project#2497) * Implements `glm.LogisticRegression` (mars-project#2466) * Implements bagging sampling (mars-project#2496) * Refine MarsDMatrix & support more parameters for XGB classifier and regressor (mars-project#2498) * Fix output of df.groupby(as_index=False).size() (mars-project#2507) * Add preliminary implementations for ufunc methods (mars-project#2510) * Add doc for reading csv in oss (mars-project#2514) * [Ray] Fix serializing lambdas in web (mars-project#2512) * Add `make_regression` support for learn module (mars-project#2515) * Fix reduction result on empty series (mars-project#2520) * Fix df.loc when df is empty (mars-project#2524) * fix start subpool * fix test_kill_and_wait_timeout * fix autoscale timeout * fix ray larger clsuter fixture * Update ci ray package to 1.2.2 * remove python3.6 3.8 .39 ut and upgrade ray 3.7 image * echo python path * fix json decode error * fix bundle release timeout * fix remove placement group timeout * fix no_restart * fix ci * fix autoscale
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What do these changes do?
The problem is register_ray_serializers() will replace some mars serializers by ray serializers. Then the serialization.core.serialize will return headers with some unpickable objects, e.g. lambda.
This PR replaces the pickle by cloudpickle in mars.utils for serializing headers.
Related issue number
Fixes #2511