Skip to content

Commit

Permalink
Fix documentation for parallel processing work queue tasks.
Browse files Browse the repository at this point in the history
Changes in details:
- Add instructions to run redis server, since without it the rest of the steps won't work.
- For work-queue jobs, instruct the user to leave the completions number unset (as setting it to 1 won't run more than one job in parallel, and is different when left unset).
- Change the package installation to python3, as python package is no longer available
  • Loading branch information
walidghallab committed Mar 18, 2024
1 parent 652214f commit bed9706
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 2 deletions.
Expand Up @@ -84,7 +84,7 @@ Next install the `amqp-tools` so you can work with message queues.
The next commands show what you need to run inside the interactive shell in that Pod:

```shell
apt-get update && apt-get install -y curl ca-certificates amqp-tools python dnsutils
apt-get update && apt-get install -y curl ca-certificates amqp-tools python3 dnsutils
```

Later, you will make a container image that includes these packages.
Expand Down
Expand Up @@ -59,6 +59,12 @@ You could also download the following files directly:
- [`rediswq.py`](/examples/application/job/redis/rediswq.py)
- [`worker.py`](/examples/application/job/redis/worker.py)

To start a single instance of Redis, you need to create the redis pod and redis service:

```shell
kubectl apply -f https://k8s.io/examples/application/job/redis/redis-pod.yaml
kubectl apply -f https://k8s.io/examples/application/job/redis/redis-service.yaml
```

## Filling the queue with tasks

Expand Down Expand Up @@ -171,7 +177,7 @@ Since the workers themselves detect when the workqueue is empty, and the Job con
know about the workqueue, it relies on the workers to signal when they are done working.
The workers signal that the queue is empty by exiting with success. So, as soon as **any** worker
exits with success, the controller knows the work is done, and that the Pods will exit soon.
So, you need to set the completion count of the Job to 1. The job controller will wait for
So, you need to leave the completion count of the Job unset. The job controller will wait for
the other pods to complete too.

## Running the Job
Expand Down

0 comments on commit bed9706

Please sign in to comment.