Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allows CSV grad data to have extra quotes #2322

Merged
merged 1 commit into from
Jun 21, 2023
Merged

Allows CSV grad data to have extra quotes #2322

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions app/services/graduation_service.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,14 @@ def run
update_registrar_feed(approved_etds, publishable_etds)
end

DOWNCASE_CONVERTER = ->(header) { header.downcase }

# Read registrar data from CSV or JSON source file
# responsible for massaging the input file into the same json structure
# @return [Hash] a hash of registrar keys pointing at the associated student graduation records
def parse_registrar_file
grad_records = @registrar_feed.graduation_records
case grad_records.content_type
when 'text/csv', 'application/vnd.ms-excel'
registrar_csv = CSV.parse(grad_records.download, headers: true, header_converters: DOWNCASE_CONVERTER)
registrar_csv = CSV.parse(grad_records.download, headers: true, header_converters: :downcase, liberal_parsing: true)
registrar_csv.map { |row| [row['etd record key'], row.to_hash] }.to_h
when 'application/json'
JSON.parse(grad_records.download)
Expand Down
2 changes: 1 addition & 1 deletion spec/fixtures/registrar_feeds/registrar_sample.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
etd record key,public person id,directory last name,directory first name,directory middle name,preferred email address,home address 1,home address 2,home address 3,home address city,home address state,home address postal code,home address country code,home address country descr,ferpa suppression flag,acad career code,acad career descr,acad program code,acad program descr,primary acad plan code,primary acad plan descr,secondary acad plan code,secondary acad plan descr,program status descr,degree code,degree status descr,degree status date
P0000001-GSAS-PHD,P0000001,Doe,John, ,jdoe@example.com,123 Fake St, , ,Atlanta,GA,30301,USA,United States,N,GSAS,School of Graduate Studies,PHD,Doctor of Philosophy,BBSPHD,Biological and Biomedical Sci., , ,AC,PHD, ,
P0000002-UCOL-LIBAS,P0000002,Smith,Jane,Cinderlla,jsmith@example.com,321 Ash Way, , ,Atlanta,GA,30301,USA,United States,N,UCOL,Undergraduate Emory College,LIBAS,Liberal Arts & Sciences,POLISCIBA,Political Science,LACSND,Latin Amer. & Caribbean Stu.,AC,BS,Awarded,2017-05-18
P0000003-UCOL-LIBAS,P0000003,Hood,Riding,Red,rhood@example.com,12 Nana Ct,"","",Atlanta,GA,30301,USA,United States,N,UCOL,Undergraduate Emory College,LIBAS,Liberal Arts & Sciences,MATHCSBS,Mathematics & Computer Science, , ,CM,BS,Awarded,2017-03-16
P0000003-UCOL-LIBAS,P0000003,Hood,Riding,Red,rhood@example.com,"12 "Nana" Ct","","",Atlanta,GA,30301,USA,United States,N,UCOL,Undergraduate Emory College,LIBAS,Liberal Arts & Sciences,MATHCSBS,Mathematics & Computer Science, , ,CM,BS,Awarded,2017-03-16
P0000004-THEO-MDV,P0000004,Smith,Jim,James,jim.s@example.com,123 Fake Dr,"","",Atlanta,GA,30301,USA,United States,N,THEO,Theology,MDV,Master of Divinity,MDVDIVIN,Divinity, , ,CM,MDV,Awarded,2018-01-12
P0000004-THEO-THD,P0000004,Smith,Jim,James,jim.s@example.com,123 Fake Dr,"","",Atlanta,GA,30301,USA,United States,N,THEO,Theology,THD,Doctor of Theology,THDCOUNSEL,Pastoral Counseling, , ,AC,THD,Awarded,2020-05-23
P0000005-GSAS-PHD,P0000005,Anderson,John, ,janders@example.com,123 Fake Dr,"","",Atlanta,GA,30301,USA,United States,N,GSAS,School of Graduate Studies,PHD,Doctor of Philosophy,CHEMPHD,Chemistry, , ,CM,PHD,Awarded,2020-05-25
Expand Down
8 changes: 6 additions & 2 deletions spec/services/graduation_service_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
describe "#parse_registrar_file" do
context 'with JSON data' do
let(:feed) { FactoryBot.create(:json_registrar_feed) }
it "exrtracts records successfully" do
it "extracts records successfully" do
parsed_data = grad_service.parse_registrar_file
expect(parsed_data)
.to include('P0000006-UBUS-BBA' =>
Expand All @@ -56,13 +56,17 @@

context 'with CSV data' do
let(:feed) { FactoryBot.create(:registrar_feed) }
it 'exrtracts records successfully' do
it 'extracts records successfully' do
parsed_data = grad_service.parse_registrar_file
expect(parsed_data)
.to include('P0000006-UBUS-BBA' =>
hash_including('public person id' => 'P0000006',
'directory last name' => 'Dieu-le-Veut',
'degree status date' => '2022-05-25'))
expect(parsed_data)
.to include('P0000003-UCOL-LIBAS' =>
hash_including('public person id' => 'P0000003',
'home address 1' => "\"12 \"Nana\" Ct\""))
end
end

Expand Down