Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert C2M to worker Queue #1062

Merged
merged 6 commits into from Aug 9, 2018
Merged

Convert C2M to worker Queue #1062

merged 6 commits into from Aug 9, 2018

Conversation

atz
Copy link
Contributor

@atz atz commented Aug 7, 2018

Note: we already abandoned order when we moved order from scopes to class methods. The service classes still called the scopes and not the class methods, so no ordering was happening.

That's more obvious now, since moving to a queue means asynchronicity inherently. Order enqueued does not guarantee order of completion. Also presumably parallelization provides better enough performance and reliability that we aren't worried about some objects never making it through.

Massive cleanup to twisty code. No entanglement of class methods and instance methods. A factory for fixtures!

Follow on work would include:

  • cleanup README, maybe wiki
  • add new workers to resque-pool(s) required
  • change how we schedule C2M, since the "every saturday" stuff is not particularly relevant anymore

@atz atz force-pushed the c2m_quuuuu branch 2 times, most recently from baf9b83 to 4b5c471 Compare August 7, 2018 23:10
@coveralls
Copy link

coveralls commented Aug 7, 2018

Coverage Status

Coverage decreased (-0.2%) to 97.709% when pulling 4ff63aa on c2m_quuuuu into a73c2c4 on master.

Copy link
Contributor

@SaravShah SaravShah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me. just one copy pasta comment!

raise ArgumentError, 'CompleteMoab param required' unless job.arguments.first.is_a?(CompleteMoab)
end

# @param [CompleteMoab] complete_moab object to checksum
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assuming this is copy pasta

atz added 6 commits August 8, 2018 16:44
- Now uses factory that is fixture-aware.
- Use single `AuditResults` instance double for all cases.
- Stop intercepting `UnrelatedModel.new(...)` and just set the double
object directly into the subject where needed.  Then set expecations on
the model's getter (instead of on the double independently).
- More better `before` blocks
@@ -56,8 +53,6 @@ class CompleteMoab < ApplicationRecord
' AND (last_checksum_validation + (fixity_ttl * INTERVAL \'1 SECOND\')) < CURRENT_TIMESTAMP'\
' OR last_checksum_validation IS NULL'
)
# possibly counter-intuitive: the .order sorts so that null values come first (because IS NOT NULL evaluates
# to 0 for nulls, which sorts before 1 for non-nulls, which are then sorted by last_checksum_validation)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nitpick (leftovers from the prior PR that got rid of sort order on the scope): it'd be nice to move these comments to the class methods that are now provided as ordering wrappers.

@jmartin-sul jmartin-sul merged commit cfa4172 into master Aug 9, 2018
@jmartin-sul jmartin-sul deleted the c2m_quuuuu branch August 9, 2018 02:07
@jmartin-sul
Copy link
Member

merged, happy to do my nitpick as a follow-on if others find it agreeable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants