Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle virtual object creation in a background job #422

Merged
merged 6 commits into from Sep 19, 2019
Merged

Conversation

mjgiarlo
Copy link
Member

@mjgiarlo mjgiarlo commented Sep 18, 2019

Connects to #336
Deployment blocked by sul-dlss/puppet#4470

  • Add ActiveRecord and Postgres as dependencies. This makes it possible to reliably stash information about background jobs in a persistent store, which will be used to run virtual object creation in the background.
  • Include factory_bot_rails as a dependency which will make it easier to spin
    up fixture-like factories when testing with database entities later.
  • Create a model and a table for background job results. The model holds any output from the job, an HTTP status code, and a processing status.
  • Add factory_bot_rails to the test suite. This allows us to more efficiently test database-backed entities, specifically BackgroundJobResults.
  • Add new controller and view for background job results.
  • Use a background job to create virtual objects. Use the new job from the CreateVirtualObjectsController. In the job, record progress and results in the application database

Why was this change made?

To be able to process large batches of virtual objects outside of the request/response cycle.

Was the API documentation (openapi.json) updated?

Yes.

Copy link
Contributor

@jcoyne jcoyne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR message says you updated openapi.json, but I don't see that in the changed files.

app/controllers/virtual_objects_controller.rb Outdated Show resolved Hide resolved
app/controllers/virtual_objects_controller.rb Outdated Show resolved Hide resolved
app/jobs/create_virtual_objects_job.rb Outdated Show resolved Hide resolved
app/models/background_job_result.rb Outdated Show resolved Hide resolved
spec/support/auth_helper.rb Outdated Show resolved Hide resolved
@mjgiarlo
Copy link
Member Author

@jcoyne 💬

The PR message says you updated openapi.json, but I don't see that in the changed files.

Included now.

@mjgiarlo mjgiarlo force-pushed the bg_virtobj_336 branch 7 times, most recently from 23991be to 8699868 Compare September 18, 2019 21:36
Connects to #336

This makes it possible to reliably stash information about background jobs in a persistent store, which will be used to run virtual object creation in the background.

Includes `factory_bot_rails` as a dependency which will make it easier to spin
up fixture-like factories when testing with database entities later.
Connects to #336

The model holds any output from the job, an HTTP status code, and a processing status.
Connects to #336

This allows us to test database-backed entities, specifically BackgroundJobResults, more efficently.
Connects to #336

And use the new job from the CreateVirtualObjectsController. In the job, record progress and results in the application database.
class CreateBackgroundJobResults < ActiveRecord::Migration[5.2]
def up
execute <<-SQL
CREATE TYPE background_job_result_status AS ENUM (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! We should port this to workflow-service.

openapi.json Outdated Show resolved Hide resolved
openapi.json Outdated Show resolved Hide resolved
openapi.json Outdated Show resolved Hide resolved
@jcoyne jcoyne merged commit 7bfbfdc into master Sep 19, 2019
@jcoyne jcoyne deleted the bg_virtobj_336 branch September 19, 2019 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants