Fixes - #270 Implemented mapping from DAMS4 to SHARE V2 API and updat… #460

hweng · 2018-06-07T17:25:25Z

Fixes #270

Local Checklist

What does this PR do?

Added support for mapping from DAMS4 to SHARE Version 2 schema.
Updated push function due to SHARE Version 2 API changes.
Refactored old SHARE mapping code to conform to Rubocop suggested expression, syntax and formatting.

@ucsdlib/developers - please review

houndci-bot · 2018-06-07T17:25:38Z

lib/dams/controller_helper.rb

+        'tags': osf_mads_fields(document),
+        'extra': osf_extra(document)
+      }
+      json_data = {"jsonData": field_map}


Lint/UselessAssignment: Useless assignment to variable - json_data.
Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T17:25:38Z

lib/dams/controller_helper.rb

-		  }
-		  json_data = {"jsonData": field_map}
-		end
+    def export_to_API(document)


Naming/MethodName: Use snake_case for method names.

houndci-bot · 2018-06-07T17:25:38Z

lib/dams/controller_helper.rb

+          dams_data.each do |datum| 
+            osf_data << datum
+          end
+        end 


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T17:25:38Z

lib/dams/controller_helper.rb

+        if dams_data.kind_of?(String)
+          osf_data << dams_data 
+        elsif dams_data.kind_of?(Array)
+          dams_data.each do |datum| 


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T17:25:38Z

lib/dams/controller_helper.rb

+        dams_data = document["#{field_name}"]
+        if dams_data.kind_of?(String)
+          osf_data << dams_data 
+        elsif dams_data.kind_of?(Array)


Style/ClassCheck: Prefer Object#is_a? over Object#kind_of?.

houndci-bot · 2018-06-07T17:25:39Z

lib/dams/controller_helper.rb

-		  end
-		  osf_data
-		end
+    def osf_mads_fields(document)


Metrics/MethodLength: Method has too many lines. [27/15]

houndci-bot · 2018-06-07T17:25:39Z

lib/dams/controller_helper.rb

      end
-			osf_data 
-		end
+      osf_data 


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T17:25:39Z

lib/dams/controller_helper.rb

-      	langs.each do |lang|  
-      		osf_data <<	lang
-      	end
+        langs.each do |lang|  


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T17:25:39Z

lib/dams/controller_helper.rb

-			dams_data = document["#{field_name}"]
+    def osf_languages(document)
+      field_name = "language_tesim"
+      dams_data = document["#{field_name}"]


Style/UnneededInterpolation: Prefer to_s over string interpolation.

houndci-bot · 2018-06-07T17:25:39Z

lib/dams/controller_helper.rb

-			field_name = "language_tesim"
-			dams_data = document["#{field_name}"]
+    def osf_languages(document)
+      field_name = "language_tesim"


Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols.

houndci-bot · 2018-06-07T17:47:56Z

lib/dams/controller_helper.rb

+        'tags': osf_mads_fields(document),
+        'extra': osf_extra(document) 
+      }
+      {'jsonData': field_map}


Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T17:47:56Z

lib/dams/controller_helper.rb

+        'languages': osf_languages(document),
+        'date_published': osf_date_published(document),
+        'tags': osf_mads_fields(document),
+        'extra': osf_extra(document) 


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T17:47:56Z

lib/dams/controller_helper.rb

-		  json_data = {"jsonData": field_map}
-		end
+    def export_to_api(document)
+      field_map = { 


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T17:47:56Z

lib/dams/controller_helper.rb

+      mads_field ||= YAML.load(ERB.new(IO.read(File.join(Rails.root, 'config', 'mads_field.yml'))).result)
+      field_names = mads_field.fetch('key')
+      field_names.each do |field_name|
+        dams_data = document["#{field_name}"]


Style/UnneededInterpolation: Prefer to_s over string interpolation.

houndci-bot · 2018-06-07T17:47:56Z

lib/dams/controller_helper.rb

-		end
+    def osf_mads_fields(document)
+      osf_data = []
+      mads_field ||= YAML.load(ERB.new(IO.read(File.join(Rails.root, 'config', 'mads_field.yml'))).result)


Security/YAMLLoad: Prefer using YAML.safe_load over YAML.load.
Rails/FilePath: Please use Rails.root.join('path', 'to') instead.

houndci-bot · 2018-06-07T17:47:57Z

lib/dams/controller_helper.rb

+
+    def osf_date_published(document)
+      field_name = 'date_json_tesim'
+      dams_data = document["#{field_name}"]


Style/UnneededInterpolation: Prefer to_s over string interpolation.

houndci-bot · 2018-06-07T17:47:57Z

lib/dams/controller_helper.rb

+
+      if dams_data != nil
+        url = "http://library.ucsd.edu/dc/collection/#{dams_data}"
+        osf_data = {'canonicalUri': url, 'providerUris': url}


Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T17:47:57Z

lib/dams/controller_helper.rb

+      dams_data = document["#{field_name}"]
+      osf_data = {}
+
+      if dams_data != nil


Style/NonNilCheck: Prefer !expression.nil? over expression != nil.

houndci-bot · 2018-06-07T17:47:58Z

lib/dams/controller_helper.rb

+
+    def osf_uris(document)
+      field_name = 'id'
+      dams_data = document["#{field_name}"]


Style/UnneededInterpolation: Prefer to_s over string interpolation.

houndci-bot · 2018-06-07T17:47:58Z

lib/dams/controller_helper.rb

+        end
+      end
+      osf_data
+    end 


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T18:11:53Z

lib/dams/controller_helper.rb

+        'tags': osf_mads_fields(document),
+        'extra': osf_extra(document)
+      }
+      {'jsonData': field_map}


Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T18:11:53Z

lib/dams/controller_helper.rb

+      mads_field ||= YAML.safe_load(ERB.new(IO.read(Rails.root.join('config', 'mads_field.yml'))).result)
+      field_names = mads_field.fetch('key')
+      field_names.each do |field_name|
+        dams_data = document["#{field_name}"]


Style/UnneededInterpolation: Prefer to_s over string interpolation.

houndci-bot · 2018-06-07T18:11:53Z

lib/dams/controller_helper.rb

+      dams_data = document['date_json_tesim']
+      osf_data = ''
+
+      if !dams_data.nil?


Style/NegatedIf: Favor unless over if for negative conditions.

houndci-bot · 2018-06-07T18:11:54Z

lib/dams/controller_helper.rb

+    def osf_date_published(document)
+      dams_data = document['date_json_tesim']
+      osf_data = ''
+


Layout/TrailingWhitespace: Trailing whitespace detected.

houndci-bot · 2018-06-07T18:11:54Z

lib/dams/controller_helper.rb

+      dams_data = document['id']
+      osf_data = {}
+
+      if !dams_data.nil?


Style/NegatedIf: Favor unless over if for negative conditions.

houndci-bot · 2018-06-07T18:11:55Z

lib/dams/controller_helper.rb

-		  osf_data = (osf_data.is_a?(Time) || osf_data.is_a?(DateTime)) ? osf_data : Time.now
-		end
+    def agent_type(type)
+      type = 'principalinvestigator' if type == 'principal investigator' || type == 'Principal Investigator'


Style/MultipleComparison: Avoid comparing a variable with multiple items in a conditional, use Array#include? instead.

houndci-bot · 2018-06-07T18:11:55Z

lib/dams/controller_helper.rb

+          end
+        end
+      end
+      osf_data << {agent_type: 'Publisher', type: 'Organization', 'name': 'UC San Diego Library Digital Collections'}


Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T18:11:55Z

lib/dams/controller_helper.rb

+          relationships = JSON.parse(datum)
+          relationships.each do |key, value|
+            value.each do |v|
+              osf_data << {agent_type: agent_type(key), type: 'Person', 'name': v}


Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T18:11:55Z

lib/dams/controller_helper.rb

+      dams_data = document['relationship_json_tesim']
+      osf_data =[]
+
+      if !dams_data.nil?


Style/NegatedIf: Favor unless over if for negative conditions.

houndci-bot · 2018-06-07T18:11:55Z

lib/dams/controller_helper.rb

-		end
+    def osf_related_agents(document)
+      dams_data = document['relationship_json_tesim']
+      osf_data =[]


Layout/SpaceAroundOperators: Surrounding space missing for operator =.

houndci-bot · 2018-06-07T18:27:02Z

lib/dams/controller_helper.rb

+        dams_data.each do |datum|
+          date = JSON.parse(datum)
+          if date['type'] == 'issued'
+            d_date = date['beginDate']|| ''


Layout/SpaceAroundOperators: Surrounding space missing for operator ||.

houndci-bot · 2018-06-07T18:27:02Z

lib/dams/controller_helper.rb

+          end
+        end
+      end
+      osf_data = (osf_data.blank?) ? osf_data << { 'name': 'UC San Diego Library' } : osf_data


Style/RedundantParentheses: Don't use parentheses around a method call.
Style/TernaryParentheses: Omit parentheses for ternary conditions.

houndci-bot · 2018-06-07T18:27:02Z

lib/dams/controller_helper.rb

+          relationships = JSON.parse(datum)
+          relationships.each do |key, value|
+            value.each do |v|
+              osf_data << {"name": v}


Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.

houndci-bot · 2018-06-07T18:27:02Z

lib/dams/controller_helper.rb

+      unless dams_data.nil?
+        dams_data.each do |datum|
+          relationships = JSON.parse(datum)
+          relationships.each do |key, value|


Lint/UnusedBlockArgument: Unused block argument - key. If it's necessary, use _ or _key as an argument name to indicate that it won't be used.

houndci-bot · 2018-06-07T18:27:02Z

lib/dams/controller_helper.rb

-		end
+    def osf_contributors(document)
+      dams_data = document['relationship_json_tesim']
+      osf_data =[]


Layout/SpaceAroundOperators: Surrounding space missing for operator =.

houndci-bot · 2018-06-07T18:27:03Z

lib/dams/controller_helper.rb

+        dams_data.each do |datum|
+          title = JSON.parse(datum)
+          osf_data = title['name'] ? title['name'] : ''
+          osf_data += title['name'] && !title['translationVariant'].blank? ? ' : ' : ''


Rails/Present: Use title['translationVariant'].present? instead of !title['translationVariant'].blank?.

houndci-bot · 2018-06-07T18:27:03Z

lib/dams/controller_helper.rb

-		end
+    def osf_title(document)
+      dams_data= document['title_json_tesim']
+      osf_data=''


Layout/SpaceAroundOperators: Surrounding space missing for operator =.

houndci-bot · 2018-06-07T18:27:03Z

lib/dams/controller_helper.rb

-		  osf_data
-		end
+    def osf_title(document)
+      dams_data= document['title_json_tesim']


Layout/SpaceAroundOperators: Surrounding space missing for operator =.

houndci-bot · 2018-06-07T18:27:03Z

lib/dams/controller_helper.rb

-		  end
-		  osf_data
-		end
+    def osf_title(document)


Metrics/CyclomaticComplexity: Cyclomatic complexity for osf_title is too high. [8/7]
Metrics/MethodLength: Method has too many lines. [18/15]

houndci-bot · 2018-06-07T18:27:03Z

app/controllers/dams_resource_controller.rb

@@ -224,8 +224,15 @@ def osf_delete
  def osf_push
    @document = get_single_doc_via_search(1, {:q => "id:#{params[:id]}"} )
    authorize! :show, @document
-    document = ShareNotify::PushDocument.new("http://library.ucsd.edu/dc/collection/#{params[:id]}", osf_date(@document))
+    document = ShareNotify::PushDocument.new("http://library.ucsd.edu/dc/collection/#{params[:id]}")
+    document.type = "DataSet" if @document["unit_code_tesim"].first == "rdcp"


Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols.

coveralls · 2018-06-07T18:29:26Z

Coverage decreased (-0.2%) to 65.893% when pulling c9e0605 on feature/SHAREV2_mapping into dfa8b3e on develop.

houndci-bot · 2018-06-07T18:42:15Z

lib/dams/controller_helper.rb

+          date = JSON.parse(datum)
+          if date['type'] == 'issued'
+            d_date = date['beginDate'] || ''
+            osf_data = DateTime.new(d_date.to_i,1,1) if d_date.match( '^\d{4}$' )


Style/DateTime: Prefer Date or Time over DateTime.
Rails/TimeZone: Do not use DateTime.new without zone. Use one of Time.zone.local, DateTime.current, DateTime.new.in_time_zone, DateTime.new.utc, DateTime.new.getlocal, DateTime.new.iso8601, DateTime.new.jisx0301, DateTime.new.rfc3339, DateTime.new.to_i, DateTime.new.to_f instead.
Layout/SpaceAfterComma: Space missing after comma.
Performance/RedundantMatch: Use =~ in places where the MatchData returned by #match will not be used.
Layout/SpaceInsideParens: Space inside parentheses detected.

houndci-bot · 2018-06-07T18:42:15Z

lib/dams/controller_helper.rb

+      type = 'principalinvestigator' if ['principal investigator', 'Principal Investigator'].include? type
+      agent_dataset ||= YAML.safe_load(ERB.new(IO.read(Rails.root.join('config', 'share_agent_type.yml'))).result)
+      share_agent_type = agent_dataset.fetch('key')
+      type = (share_agent_type.include? type) ? type : 'Contributor'


Lint/UselessAssignment: Useless assignment to variable - type.
Style/TernaryParentheses: Omit parentheses for ternary conditions.

houndci-bot · 2018-06-07T18:42:15Z

lib/dams/controller_helper.rb

+          end
+        end
+      end
+      osf_data = osf_data.blank? ? osf_data << { 'name': 'UC San Diego Library' } : osf_data


Rails/Presence: Use osf_data.presence || osf_data << { 'name': 'UC San Diego Library' } instead of osf_data.blank? ? osf_data << { 'name': 'UC San Diego Library' } : osf_data.

houndci-bot · 2018-06-07T19:17:23Z

lib/dams/controller_helper.rb

+          end
+        end
+      end
+      osf_data = (osf_data.is_a?(Time) || osf_data.is_a?(DateTime)) ? osf_data : Time.now


Style/TernaryParentheses: Omit parentheses for ternary conditions.
Rails/TimeZone: Do not use Time.now without zone. Use one of Time.zone.now, Time.current, Time.now.in_time_zone, Time.now.utc, Time.now.getlocal, Time.now.iso8601, Time.now.jisx0301, Time.now.rfc3339, Time.now.to_i, Time.now.to_f instead.

houndci-bot · 2018-06-07T19:17:23Z

lib/dams/controller_helper.rb

+          date = JSON.parse(datum)
+          if date['type'] == 'issued'
+            d_date = date['beginDate'] || ''
+            osf_data = Time.utc(d_date.to_i, 1, 1) if d_date.match('^\d{4}$')


Performance/RedundantMatch: Use =~ in places where the MatchData returned by #match will not be used.

houndci-bot · 2018-06-07T19:17:24Z

lib/dams/controller_helper.rb

+    def agent_type(type)
+      type = 'principalinvestigator' if ['principal investigator', 'Principal Investigator'].include? type
+      agent_dataset ||= YAML.safe_load(ERB.new(IO.read(Rails.root.join('config', 'share_agent_type.yml'))).result)
+      (agent_dataset.fetch('key').include? type) ? type : 'Contributor'


Style/TernaryParentheses: Omit parentheses for ternary conditions.

houndci-bot · 2018-06-07T19:17:24Z

lib/dams/controller_helper.rb

+          end
+        end
+      end
+      osf_data = osf_data.presence || osf_data << { 'name': 'UC San Diego Library' } 


Layout/TrailingWhitespace: Trailing whitespace detected.

…ed push function. Refactor code to pass rubocop sytle check. Refactor the mapping code to use Rails.root.join to fetch config file. Refactor code to Array#include? for comparison and use to_s over string interpolation Refactored argument name and remove all trailing whitespace Refactor to use Time instead of DateTime and use Rails/Presence Refactor the regex and reduce the title method Complexity refactor for Style/TernaryParentheses Fixing failed tests.

hweng · 2018-06-08T20:03:35Z

@mcritchlow I moved to this this PR as there are fair amount of refactoring for old code according to Rubocop. And the agent_type, mads_type was moved to config file as you suggested.

hweng · 2018-06-08T20:05:26Z

@ucsdlib/developers - All Hound Rubocop check passed, please review. Thanks!

mcritchlow

LGTM 👍

VivianChu · 2018-06-08T21:58:59Z

👍

lsitu · 2018-06-11T16:09:34Z

👍

houndci-bot reviewed Jun 7, 2018

View reviewed changes

hweng force-pushed the feature/SHAREV2_mapping branch from 0dcec0e to c9e0605 Compare June 7, 2018 21:49

hweng mentioned this pull request Jun 8, 2018

Fixes - #270 Implemented mapping from DAMS4 to SHARE V2 API and updated push function. #454

Closed

mcritchlow approved these changes Jun 8, 2018

View reviewed changes

lsitu merged commit 7008f85 into develop Jun 11, 2018

lsitu deleted the feature/SHAREV2_mapping branch June 11, 2018 16:09

Fixes - #270 Implemented mapping from DAMS4 to SHARE V2 API and updat… #460

Fixes - #270 Implemented mapping from DAMS4 to SHARE V2 API and updat… #460

Conversation

hweng commented Jun 7, 2018 • edited Loading

Local Checklist

What does this PR do?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Jun 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hweng commented Jun 8, 2018

hweng commented Jun 8, 2018

mcritchlow left a comment

Choose a reason for hiding this comment

VivianChu commented Jun 8, 2018

lsitu commented Jun 11, 2018

hweng commented Jun 7, 2018 •

edited

Loading

coveralls commented Jun 7, 2018 •

edited

Loading