WIP gem to make active job work with kubernetes jobs.
Roadmap for V1.0:
- ActiveJob.perform_later create a Job in k8s that will execute the job
- ActiveJob.perform_later with delay create a Job in k8s in suspended mode, a task will enable it as soon as the time is reached
- Limiting the number of concurrent jobs (if there are more jobs they will be created in suspended mode)
Future:
- Metrics (because everyone like metrics)
- Retry options
Add this line to your application's Gemfile:
gem 'active_job_k8s'And then execute:
$ bundle install
Or install it yourself as:
$ gem install active_job_k8s
Configure the KubeClient as documented here: https://github.com/ManageIQ/kubeclient and instantiate the adapter.
Example: Local with default kind cluster:
kubeclient_config = Kubeclient::Config.read(ENV['KUBECONFIG'] || File.join(Dir.home, '/.kube/config'))
config.active_job.queue_adapter = ActiveJob::QueueAdapters::K8sAdapter.new(
kubeclient_context: kubeclient_config.context('kind-kind')
)Example: production configuration:
Rails.application.configure do
auth_options = {
bearer_token_file: "/var/run/secrets/kubernetes.io/serviceaccount/token"
}
ssl_options = {}
if File.exist?("/var/run/secrets/kubernetes.io/serviceaccount/ca.crt")
ssl_options[:ca_file] = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
end
api_endpoint = "https://kubernetes.default.svc"
api_version = "v1"
namespace = File.read("/var/run/secrets/kubernetes.io/serviceaccount/namespace")
context = Kubeclient::Config::Context.new(
api_endpoint,
api_version,
ssl_options,
auth_options,
namespace
)
default_manifest = YAML.safe_load(
<<~MANIFEST
apiVersion: batch/v1
kind: Job
metadata:
name: scheduled-job-name
namespace: default
spec:
template:
spec:
restartPolicy: Never
containers:
- name: app-job
image: image_of_the_rails_application
imagePullPolicy: IfNotPresent
command:
- /bin/bash
args:
- '-c'
- bundle exec rails active_job_k8s:run_job
MANIFEST
)
config.active_job.queue_adapter = ActiveJob::QueueAdapters::K8sAdapter.new(
kubeclient_context: context,
default_manifest: default_manifest
)
endThe adapter should be capable to query the kubernetes cluster, this is the role to create inside the namespace, it will add api capacity o jobs to the default ServiceAccount
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: jobs-edit
namespace: YOURN-NAMESPACE
rules:
- apiGroups:
- "batch"
resources:
- jobs
verbs:
- get
- list
- create
- patch
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: jobs-edit
namespace: YOURN-NAMESPACE
subjects:
- kind: ServiceAccount
name: default
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jobs-editInside you job you should describe the initial manifest for the KubernetesJob
Example:
class HelloWorldJob < ApplicationJob
queue_as :default
def perform(*args)
Rails.logger.debug "Hello World"
end
#[Optional] if not present it will be taken from scheduler configuration
# def manifest
# .....
# end
endThe command will be inserted by the gem if not present.
command: ["rails"]
args: ["active_job_k8s:run_job"]To the name of the Job we will append a timestamp to make it uniq
Requirements: kind
After checking out the repo, run bin/setup to install the environment (cluster and dummy app).
Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/active_job_k8s. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.
The gem is available as open source under the terms of the MIT License.
Everyone interacting in the ActiveJobK8s project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.