-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This adds a status field to `PreIngestFile`s that defaults to 'preingest'. The `AttachFilesToWorkJob` class is preprended with logic that updates the status to 'attached'. The logic requires that `deduplication_key` is present in the CSV so that field is added to the default mapper and validator. There is a helper which outputs a success icon (a green checkmark) if the `PreIngestFile` has an 'attached' status. This is used for the `PreIngestWork` if all the statuses for the `PreIngestFile` are 'attached'.
- Loading branch information
Showing
19 changed files
with
180 additions
and
12 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
module AttachFilesToWorkJobPrepends | ||
# @param [ActiveFedora::Base] work - the work object | ||
# @param [Array<Hyrax::UploadedFile>] uploaded_files - an array of files to attach | ||
def perform(work, uploaded_files, **work_attributes) | ||
super | ||
if work.deduplication_key | ||
uploaded_files.each do |uploaded_file| | ||
# Remove existing PreIngestFile for this FileSet | ||
pre_ingest_work = Zizia::PreIngestWork.find_by(deduplication_key: work.deduplication_key) | ||
pre_ingest_file = pre_ingest_work.pre_ingest_files.where('status != "attached"') | ||
.find_by(filename: File.basename(uploaded_file.file.path)) | ||
pre_ingest_file.status = 'attached' | ||
pre_ingest_file.save | ||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
AttachFilesToWorkJob.prepend(AttachFilesToWorkJobPrepends) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
class AddStatusToPreIngestFile < ActiveRecord::Migration[5.1] | ||
def change | ||
add_column :zizia_pre_ingest_files, :status, :string, default: 'preingest' | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 4 additions & 4 deletions
8
spec/dummy/spec/fixtures/csv_import/csv_files_with_problems/extra - headers.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
rights_statement,another_header_2,rights statement,title,files,label,relative_path,import url,resource type,creator,contributor,abstract or summary,keyword,license,publisher,date created,subject,language,identifier,location,related url,bibliographic_citation,source,visibility | ||
http://rightsstatements.org/vocab/InC/1.0/,,http://rightsstatements.org/vocab/InC/1.0/,Work 1 Title,cat.jpg,,,,,creator,,,cat,,,,,,,,,,,open | ||
http://rightsstatements.org/vocab/InC/1.0/,,http://rightsstatements.org/vocab/InC/1.0/,Work 2 Title,cat.jpg,,,,,creator,,,cat,,,,,,,,,,,open | ||
http://rightsstatements.org/vocab/InC/1.0/,,http://rightsstatements.org/vocab/InC/1.0/,Work 3 Title,cat.jpg,,,,,creator,,,cat,,,,,,,,,,,open | ||
deduplication_key,rights_statement,another_header_2,rights statement,title,files,label,relative_path,import url,resource type,creator,contributor,abstract or summary,keyword,license,publisher,date created,subject,language,identifier,location,related url,bibliographic_citation,source,visibility | ||
1,http://rightsstatements.org/vocab/InC/1.0/,,http://rightsstatements.org/vocab/InC/1.0/,Work 1 Title,cat.jpg,,,,,creator,,,cat,,,,,,,,,,,open | ||
2,http://rightsstatements.org/vocab/InC/1.0/,,http://rightsstatements.org/vocab/InC/1.0/,Work 2 Title,cat.jpg,,,,,creator,,,cat,,,,,,,,,,,open | ||
3,http://rightsstatements.org/vocab/InC/1.0/,,http://rightsstatements.org/vocab/InC/1.0/,Work 3 Title,cat.jpg,,,,,creator,,,cat,,,,,,,,,,,open |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
identifier,license,deduplication_key,visibility,location,keyword,rights statement,creator,title,files | ||
abc/123,https://creativecommons.org/licenses/by/4.0/,abc/123,PUBlic,http://www.geonames.org/5667009/montana.html|~|http://www.geonames.org/6252001/united-states.html,Clothing stores $z California $z Los Angeles|~|Interior design $z California $z Los Angeles,http://rightsstatements.org/vocab/InC/1.0/,"Connell, Will, $d 1898-1961","Interior view of The Bachelors haberdashery designed by Julius Ralph Davidson, Los Angeles, circa 1929",dog.jpg | ||
identifier,type,license,deduplication_key,visibility,location,keyword,rights statement,creator,title,files | ||
abc/123,work,https://creativecommons.org/licenses/by/4.0/,abc/123,PUBlic,http://www.geonames.org/5667009/montana.html|~|http://www.geonames.org/6252001/united-states.html,Clothing stores $z California $z Los Angeles|~|Interior design $z California $z Los Angeles,http://rightsstatements.org/vocab/InC/1.0/,"Connell, Will, $d 1898-1961","Interior view of The Bachelors haberdashery designed by Julius Ralph Davidson, Los Angeles, circa 1929",dog.jpg |
2 changes: 2 additions & 0 deletions
2
spec/dummy/spec/fixtures/csv_import/good/all_fields_multi.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
identifier,type,license,deduplication_key,visibility,location,keyword,rights statement,creator,title,files | ||
abc/123,work,https://creativecommons.org/licenses/by/4.0/,abc/123,PUBlic,http://www.geonames.org/5667009/montana.html|~|http://www.geonames.org/6252001/united-states.html,Clothing stores $z California $z Los Angeles|~|Interior design $z California $z Los Angeles,http://rightsstatements.org/vocab/InC/1.0/,"Connell, Will, $d 1898-1961","Interior view of The Bachelors haberdashery designed by Julius Ralph Davidson, Los Angeles, circa 1929",dog.jpg|~|cat.jpg |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
# frozen_string_literal: true | ||
require 'rails_helper' | ||
include Warden::Test::Helpers | ||
|
||
RSpec.describe 'Importing records from a CSV file', :perform_jobs, :clean, type: :system, js: true do | ||
before do | ||
allow(CharacterizeJob).to receive(:perform_later) | ||
ENV['IMPORT_PATH'] = File.join(fixture_path, 'images') | ||
end | ||
|
||
let(:csv_file) { File.join(fixture_path, 'csv_import', 'good', 'all_fields_multi.csv') } | ||
|
||
context 'logged in as an admin user' do | ||
let(:collection) { FactoryBot.build(:collection, title: ['Testing Collection']) } | ||
let(:admin_user) { FactoryBot.create(:admin) } | ||
|
||
let(:admin_set_id) { AdminSet.find_or_create_default_admin_set_id } | ||
let(:permission_template) { Hyrax::PermissionTemplate.find_or_create_by!(source_id: admin_set_id) } | ||
let(:workflow) { Sipity::Workflow.create!(active: true, name: 'test-workflow', permission_template: permission_template) } | ||
|
||
before do | ||
# Create a single action that can be taken | ||
Sipity::WorkflowAction.create!(name: 'submit', workflow: workflow) | ||
|
||
# Grant the user access to deposit into the admin set. | ||
Hyrax::PermissionTemplateAccess.create!( | ||
permission_template_id: permission_template.id, | ||
agent_type: 'user', | ||
agent_id: admin_user.user_key, | ||
access: 'deposit' | ||
) | ||
|
||
collection.save! | ||
login_as admin_user | ||
end | ||
|
||
it 'starts the import' do | ||
visit '/csv_imports/new' | ||
expect(page).to have_content 'Testing Collection' | ||
expect(page).not_to have_content '["Testing Collection"]' | ||
|
||
# Fill in and submit the form | ||
select 'Testing Collection', from: "csv_import[fedora_collection_id]" | ||
select 'Update Existing Metadata, create new works', from: "csv_import[update_actor_stack]" | ||
attach_file('csv_import[manifest]', csv_file, make_visible: true) | ||
|
||
expect(page).to have_content('You sucessfully uploaded this CSV: all_fields_multi.csv') | ||
|
||
click_on 'Preview Import' | ||
|
||
# We expect to see the title of the collection on the page | ||
expect(page).to have_content 'Testing Collection' | ||
|
||
expect(page).to have_content 'This import will process 1 row(s).' | ||
|
||
# There is a link so the user can cancel. | ||
expect(page).to have_link 'Cancel', href: '/csv_imports/new?locale=en' | ||
|
||
expect(page).not_to have_content 'deduplication_key' | ||
|
||
# After reading the warnings, the user decides | ||
# to continue with the import. | ||
click_on 'Start Import' | ||
|
||
# The show page for the CsvImport | ||
expect(page).to have_content 'all_fields_multi.csv' | ||
expect(page).to have_content 'Start time' | ||
|
||
# We expect to see the title of the collection on the page | ||
expect(page).to have_content 'Testing Collection' | ||
expect(Work.count).to eq 1 | ||
|
||
# Ensure that all the fields got assigned as expected | ||
work = Work.where(title: "*haberdashery*").first | ||
expect(work.title.first).to match(/haberdashery/) | ||
expect(Zizia::PreIngestFile.last.status).to eq('attached') | ||
|
||
expect(Zizia::PreIngestWork.find_by(deduplication_key: work.deduplication_key).pre_ingest_files.count).to eq(2) | ||
|
||
# Everything should have the attached status now | ||
expect(Zizia::PreIngestWork.find_by(deduplication_key: work.deduplication_key) | ||
.pre_ingest_files.first.status).to eq('attached') | ||
expect(Zizia::PreIngestWork.find_by(deduplication_key: work.deduplication_key).pre_ingest_files.last.status).to eq('attached') | ||
expect(Zizia::PreIngestWork.find_by(deduplication_key: work.deduplication_key).status).to eq('attached') | ||
visit('/csv_import_details/index') | ||
click_on '1' | ||
expect(page).to have_content 'View Files' | ||
click_on 'View Files' | ||
expect(page.html).not_to match(/glyphicon-question-sign/) | ||
expect(page.html).to match(/glyphicon-ok-sign/) | ||
expect(page).to have_content('dog.jpg') | ||
expect(page).to have_content('cat.jpg') | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# frozen_string_literal: true | ||
require 'rails_helper' | ||
|
||
RSpec.describe Zizia::ApplicationJob do | ||
let(:application_job) { described_class.new } | ||
|
||
context 'when including Zizia' do | ||
it 'has its own ApplicationJob class' do | ||
expect(application_job).to be_kind_of(ActiveJob::Base) | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters