Skip to content
Browse files

Base `JobBatch#complete?` on the presence of `completed_at`.

The old logic was designed to figure out when to mark the
job batch as complete by setting the timestamp -- but
that has been moved into the lua script. Basing `complete?`
on the job jid sets is potentially risky since we have
discussed making some of the job batch redis keys expire
sooner than others. If the job jid set keys vanished from
redis the old logic would wrongly report the job batch
was not complete even though it was. The new logic
relies only on the `meta` key which is the primary key
used by a job batch to track its state.
  • Loading branch information...
1 parent f032f2e commit 17ea826dd6cd2546fcab979d328842dab13372b3 @myronmarston myronmarston committed
Showing with 6 additions and 17 deletions.
  1. +1 −5 lib/plines/job_batch.rb
  2. +5 −12 spec/unit/plines/job_batch_spec.rb
View
6 lib/plines/job_batch.rb
@@ -132,7 +132,7 @@ def complete_job(qless_job)
end
def complete?
- _complete?(pending_job_jids.length, completed_job_jids.length)
+ !!completed_at
end
def resolve_external_dependency(dep_name)
@@ -319,10 +319,6 @@ def update_external_dependency(dep_name, meth, jids)
end
end
- def _complete?(pending_size, complete_size)
- pending_size == 0 && complete_size > 0
- end
-
def time_from(meta_entry)
date_string = meta[meta_entry]
Time.iso8601(date_string) if date_string
View
17 spec/unit/plines/job_batch_spec.rb
@@ -501,23 +501,16 @@ def create_batch_with_job
end
describe "#complete?" do
- it 'returns false when there are no pending or completed jobs' do
+ it 'returns true if the job batch has a completed_at timestamp' do
batch = JobBatch.create(qless, pipeline_module, "foo", {})
- expect(batch).not_to be_complete
+ batch.meta['completed_at'] = Time.now.getutc.iso8601
+ expect(batch).to be_complete
end
- it 'returns false when there are pending jobs and completed jobs' do
+ it 'returns false if it lacks a completed_at timestamp' do
batch = JobBatch.create(qless, pipeline_module, "foo", {})
- batch.pending_job_jids << "a"
- batch.completed_job_jids << "b"
expect(batch).not_to be_complete
end
-
- it 'returns true when there are only completed jobs' do
- batch = JobBatch.create(qless, pipeline_module, "foo", {})
- batch.completed_job_jids << "b"
- expect(batch).to be_complete
- end
end
describe "#pending_qless_jobs" do
@@ -742,7 +735,7 @@ def update_dependency(batch, name)
end
it 'returns true for a complete job batch' do
- batch.completed_job_jids << "a"
+ batch.meta['completed_at'] = Time.now.getutc.iso8601
expect(batch.in_terminal_state?).to be_true
end

0 comments on commit 17ea826

Please sign in to comment.
Something went wrong with that request. Please try again.