Skip to content
This repository has been archived by the owner on Aug 17, 2023. It is now read-only.

Fix for failing Training Job on py 3.8+ #541

Merged
merged 2 commits into from Sep 10, 2020

Conversation

izapolsk
Copy link
Contributor

@izapolsk izapolsk commented Sep 9, 2020

I've encountered the issue similar to described in #452.
It occurs when user uses function preprocessor + cloudpicke version containing cloudpickle_fast.py + python 3.8+.

The root cause is that newer versions of cloudpickle contain cloudpickle_fast.py and that file isn't copied by preprocessor during image preparation.

So, this PR aims to append mentioned above file along with the rest of cloudpickle package.

By the way, what's the reason for copying that package into image ? Wouldn't it be better to copy egg there f.e. and install it on start up ?

@google-cla
Copy link

google-cla bot commented Sep 9, 2020

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@kubeflow-bot
Copy link

This change is Reviewable

@k8s-ci-robot
Copy link
Contributor

Hi @izapolsk. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@izapolsk
Copy link
Contributor Author

izapolsk commented Sep 9, 2020

@googlebot I signed it!

@izapolsk
Copy link
Contributor Author

izapolsk commented Sep 9, 2020

/assign @xauthulei

@xauthulei
Copy link
Member

/test

@k8s-ci-robot
Copy link
Contributor

@xauthulei: The /test command needs one or more targets.
The following commands are available to trigger jobs:

  • /test kubeflow-fairing-presubmit

Use /test all to run all jobs.

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@xauthulei
Copy link
Member

/ok-to-test

@xauthulei
Copy link
Member

Thanks @izapolsk

@xauthulei
Copy link
Member

Currently it report this issue:

Files with lint issues: /build/lib/kubeflow/fairing/preprocessors/function.py, /kubeflow/fairing/preprocessors/function.py

You could install the Pylint checking plugin in your python development env to catch the errors/warning here.

@jinchihe
Copy link
Member

@izapolsk This is caused by line-too-long, easy to fix. Thanks

INFO|2020-09-10T01:45:19|util.py:72| ************* Module kubeflow.fairing.preprocessors.function
INFO|2020-09-10T01:45:19|util.py:72| C: 69, 0: Line too long (118/100) (line-too-long)
INFO|2020-09-10T01:45:19|util.py:72| 
INFO|2020-09-10T01:45:19|util.py:72| -----------------------------------
INFO|2020-09-10T01:45:19|util.py:72| Your code has been rated at 9.77/10

@izapolsk
Copy link
Contributor Author

sorry, my default setting is =<120

@xauthulei
Copy link
Member

/lgtm

@xauthulei
Copy link
Member

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xauthulei

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 476b4bb into kubeflow:master Sep 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants