From bed970676cf9a5509fea66c9dd9086ed1d59c345 Mon Sep 17 00:00:00 2001 From: Walid Ghallab Date: Sat, 16 Mar 2024 02:39:53 +0100 Subject: [PATCH] Fix documentation for parallel processing work queue tasks. Changes in details: - Add instructions to run redis server, since without it the rest of the steps won't work. - For work-queue jobs, instruct the user to leave the completions number unset (as setting it to 1 won't run more than one job in parallel, and is different when left unset). - Change the package installation to python3, as python package is no longer available --- .../tasks/job/coarse-parallel-processing-work-queue.md | 2 +- .../docs/tasks/job/fine-parallel-processing-work-queue.md | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md b/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md index 7afaf2fc4cae4..e64c8b3d6172d 100644 --- a/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md +++ b/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md @@ -84,7 +84,7 @@ Next install the `amqp-tools` so you can work with message queues. The next commands show what you need to run inside the interactive shell in that Pod: ```shell -apt-get update && apt-get install -y curl ca-certificates amqp-tools python dnsutils +apt-get update && apt-get install -y curl ca-certificates amqp-tools python3 dnsutils ``` Later, you will make a container image that includes these packages. diff --git a/content/en/docs/tasks/job/fine-parallel-processing-work-queue.md b/content/en/docs/tasks/job/fine-parallel-processing-work-queue.md index b865f49fd742d..f600f60df051e 100644 --- a/content/en/docs/tasks/job/fine-parallel-processing-work-queue.md +++ b/content/en/docs/tasks/job/fine-parallel-processing-work-queue.md @@ -59,6 +59,12 @@ You could also download the following files directly: - [`rediswq.py`](/examples/application/job/redis/rediswq.py) - [`worker.py`](/examples/application/job/redis/worker.py) +To start a single instance of Redis, you need to create the redis pod and redis service: + +```shell +kubectl apply -f https://k8s.io/examples/application/job/redis/redis-pod.yaml +kubectl apply -f https://k8s.io/examples/application/job/redis/redis-service.yaml +``` ## Filling the queue with tasks @@ -171,7 +177,7 @@ Since the workers themselves detect when the workqueue is empty, and the Job con know about the workqueue, it relies on the workers to signal when they are done working. The workers signal that the queue is empty by exiting with success. So, as soon as **any** worker exits with success, the controller knows the work is done, and that the Pods will exit soon. -So, you need to set the completion count of the Job to 1. The job controller will wait for +So, you need to leave the completion count of the Job unset. The job controller will wait for the other pods to complete too. ## Running the Job