Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix #9315] Add downloadDirectory support to function k8s runtime #9377

Conversation

freeznet
Copy link
Contributor

@freeznet freeznet commented Jan 30, 2021

Fixes #9315

Motivation

k8s runtime not using downloadDirectory defined from functions_worker.yml, so user cannot self define the download directory for k8s runtime.

Modifications

  • add downloadDirectory from WorkerConfig
  • add backward compatibles
  • add tests

Verifying this change

  • Make sure that the change passes the CI checks.

@freeznet freeznet force-pushed the freeznet/fix-downloadDirectory-not-applied-to-k8s-runtime branch 2 times, most recently from 69ff4ce to 10dc0bd Compare February 1, 2021 05:40
@freeznet freeznet changed the title [WIP][fix #9315] Add downloadDirectory support to function k8s runtime [fix #9315] Add downloadDirectory support to function k8s runtime Feb 1, 2021
@freeznet
Copy link
Contributor Author

freeznet commented Feb 1, 2021

/pulsarbot run-failure-checks

@sijie sijie added this to the 2.8.0 milestone Feb 1, 2021
@sijie
Copy link
Member

sijie commented Feb 1, 2021

@nlu90 Can you review this pull request?

@freeznet freeznet force-pushed the freeznet/fix-downloadDirectory-not-applied-to-k8s-runtime branch from 10dc0bd to 992aa0e Compare February 1, 2021 13:04
@freeznet
Copy link
Contributor Author

freeznet commented Feb 1, 2021

/pulsarbot run-failure-checks

1 similar comment
@freeznet
Copy link
Contributor Author

freeznet commented Feb 1, 2021

/pulsarbot run-failure-checks

@freeznet
Copy link
Contributor Author

freeznet commented Feb 2, 2021

/pulsarbot run-failure-checks

4 similar comments
@freeznet
Copy link
Contributor Author

freeznet commented Feb 2, 2021

/pulsarbot run-failure-checks

@freeznet
Copy link
Contributor Author

freeznet commented Feb 2, 2021

/pulsarbot run-failure-checks

@freeznet
Copy link
Contributor Author

freeznet commented Feb 3, 2021

/pulsarbot run-failure-checks

@freeznet
Copy link
Contributor Author

freeznet commented Feb 3, 2021

/pulsarbot run-failure-checks

@freeznet freeznet force-pushed the freeznet/fix-downloadDirectory-not-applied-to-k8s-runtime branch from 3c7d0cf to e907cc0 Compare February 3, 2021 15:00
@freeznet
Copy link
Contributor Author

freeznet commented Feb 4, 2021

/pulsarbot run-failure-checks

Copy link
Member

@nlu90 nlu90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except for the minor comment

@freeznet freeznet force-pushed the freeznet/fix-downloadDirectory-not-applied-to-k8s-runtime branch 2 times, most recently from f6afec5 to ab8ef03 Compare February 5, 2021 00:48
@freeznet
Copy link
Contributor Author

freeznet commented Feb 5, 2021

/pulsarbot run-failure-checks

@freeznet
Copy link
Contributor Author

freeznet commented Feb 7, 2021

/pulsarbot run-failure-checks

@freeznet freeznet force-pushed the freeznet/fix-downloadDirectory-not-applied-to-k8s-runtime branch from 30e5ac0 to 2fa7434 Compare February 7, 2021 14:23
@freeznet freeznet force-pushed the freeznet/fix-downloadDirectory-not-applied-to-k8s-runtime branch from d568a44 to a3a8406 Compare February 8, 2021 12:57
@freeznet
Copy link
Contributor Author

freeznet commented Feb 9, 2021

/pulsarbot run-failure-checks

@sijie sijie merged commit 3af98bc into apache:master Feb 9, 2021
@codelipenghui
Copy link
Contributor

@freeznet Could you help push a PR based on branch-2.7?

freeznet added a commit to freeznet/pulsar that referenced this pull request Feb 19, 2021
…me (apache#9377)

Fixes apache#9315

### Motivation

k8s runtime not using `downloadDirectory` defined from `functions_worker.yml`, so user cannot self define the download directory for k8s runtime.

### Modifications

- add `downloadDirectory` from `WorkerConfig`
- add backward compatibles
- add tests

### Verifying this change

- [x] Make sure that the change passes the CI checks.
# Conflicts:
#	pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/PackagesImpl.java
#	pulsar-functions/runtime/src/test/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeTest.java
@freeznet
Copy link
Contributor Author

@freeznet Could you help push a PR based on branch-2.7?

@codelipenghui I have created a new pr #9619, PTAL, thanks.

freeznet added a commit to freeznet/pulsar that referenced this pull request Feb 22, 2021
…me (apache#9377)

Fixes apache#9315

### Motivation

k8s runtime not using `downloadDirectory` defined from `functions_worker.yml`, so user cannot self define the download directory for k8s runtime.

### Modifications

- add `downloadDirectory` from `WorkerConfig`
- add backward compatibles
- add tests

### Verifying this change

- [x] Make sure that the change passes the CI checks.
# Conflicts:
#	pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/PackagesImpl.java
#	pulsar-functions/runtime/src/test/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeTest.java
codelipenghui pushed a commit that referenced this pull request Feb 22, 2021
…9619)

Fixes #9315

### Motivation

per request from @codelipenghui in #9377, this pr is cherry pick #9377 to branch 2.7 and resolved conflicts.

### Modifications

- cherry pick pr
- resolve conflicts in tests
- remove `package` related code
@zymap
Copy link
Member

zymap commented Mar 3, 2021

We have another PR for fixing this issue on branch 2.7. So I will remove the release/2.7.1 label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pulsar-functions kuberenetesRuntime does not use downloadDirectory for placement of download files.
6 participants