Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
3e69afd
MMT-2231 Add revisions page to published UMM-T page (#610)
ryanmiller-1 Jul 8, 2020
fe0543a
MMT-2311: Added a translate_collections.rake file and added nokogiri/…
Jul 8, 2020
5e6a0a8
MMT-2288 Update jquery version that is being package for the preview …
ryanmiller-1 Jul 9, 2020
994cbb1
MMT-2230 (#608)
ryanmiller-1 Jul 9, 2020
e1394af
added display parameter to rake task
Jul 9, 2020
31b1c59
MMT-2229 Adding delete for published tools
william-valencia Jul 10, 2020
ebb408b
MMT-2311: Added a translate_collections.rake file and added nokogiri/…
Jul 8, 2020
6018f8e
MMT-2311: added display parameter to rake task
Jul 9, 2020
cac7733
Merge branch 'MMT-2311-1' of https://github.com/nasa/mmt into MMT-2311-1
Jul 10, 2020
22a9fb8
MMT-2229 Updating not current provider modal
ryanmiller-1 Jul 10, 2020
db5af95
MMT-2229 Fixing the tool_permissions_spec
william-valencia Jul 13, 2020
47bf0b2
MMT-2229 Removing duplicate test
william-valencia Jul 13, 2020
daaa4cf
Fixing spacing for tools_permissions_spec.rb
william-valencia Jul 13, 2020
f58c6ad
MMT-2234 (#607)
ryanmiller-1 Jul 13, 2020
4fa3f5c
Hotfix/MMT-2231-1 Updating revisions helpers to sort by revision date…
ryanmiller-1 Jul 13, 2020
3d6788b
Delete package-lock.json
ChristianTrummer99 Jul 13, 2020
c9ea01d
MMT-2311: made changes as per PR change requests
Jul 13, 2020
2b2fa4c
MMT-2311: made changes per PR requests, added package-lock to gitignore
Jul 13, 2020
9eec788
MMT-2229 Modifying delete_tool_spec due to comments and fixing tool_p…
william-valencia Jul 14, 2020
e1eedaa
MMT-2229 Removing unneeded delete
william-valencia Jul 14, 2020
55a52ae
MMT-2229 Removing unneeded native_id_3
william-valencia Jul 14, 2020
0885415
MMT-2311: added comments to each method definition
Jul 14, 2020
f05fef7
Merge branch 'master' into MMT-2229-Delete-Published-Tools
william-valencia Jul 14, 2020
ff40024
Merge pull request #614 from nasa/MMT-2229-Delete-Published-Tools
william-valencia Jul 14, 2020
ae1845c
MMT-2311: Added nokogiri array-path-parsing feature; removed unused f…
Jul 15, 2020
42371bb
MMT-2311: made changes according to change requests
Jul 15, 2020
c50d767
MMT-2311: more changes as per PR change requests
Jul 15, 2020
23e5cb7
MMT-2311: added comment
Jul 16, 2020
04ce818
MMT-2232/2233 Revision viewing and reversion for tools (#617)
ryanmiller-1 Jul 17, 2020
40af5e4
MMT-2311: made changes according to PR change requests
Jul 17, 2020
c479ee2
MMT-2311: more changes per PR change requests
Jul 20, 2020
52d5859
Hotfix/MMT-2232-1 Fixing tests bamboo started failing (#619)
ryanmiller-1 Jul 21, 2020
841b757
MMT-2311: Pull fixes for the selenium issues; Merge branch 'master' i…
Jul 21, 2020
d8f7561
MMT-2311: fixed dif10 asterisk issues
Jul 22, 2020
d60994a
MMT-2311: Adjusted removal of namespaces
Jul 22, 2020
55423ba
MMT-2311: not to be pushed
Jul 27, 2020
9030b9b
MMT-2311: updated the comparison methods/logic to be congruent with t…
Aug 3, 2020
a2688c4
Merge branch 'feature/MMT-2326' into hotfix/MMT-2311-1
ChristianTrummer99 Aug 4, 2020
d5e920d
MMT-2311-1: removed loss_report_helper file (it's only supposed to be…
Aug 4, 2020
636207b
Merge branch 'hotfix/MMT-2311-1' of https://github.com/nasa/mmt into …
Aug 4, 2020
345bab4
MMT-2311-1: removed unnecessary variable
Aug 4, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/routes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
get '/collections/:id/create_delete_proposal' => 'collections#create_delete_proposal', as: 'create_delete_proposal_collection'
get '/collections/:id/create_update_proposal' => 'collections#create_update_proposal', as: 'create_update_proposal_collection'


resource :variable_generation_processes_search, only: [:new]

resource :variable_generation_process, only: [:create, :update] do
Expand Down
189 changes: 106 additions & 83 deletions lib/tasks/compare_xml_collections.rake
Original file line number Diff line number Diff line change
Expand Up @@ -25,129 +25,152 @@ namespace :collection do
native_converted_hash = Hash.from_xml(back_to_native.body)
native_converted_xml = back_to_native.body

if args.format == 'dif10'

if args.format.include?('dif') || args.format.include?('iso')
nokogiri_original = Nokogiri::XML(native_original_xml) { |config| config.strict.noblanks } .remove_namespaces!
nokogiri_converted = Nokogiri::XML(native_converted_xml) { |config| config.strict.noblanks } .remove_namespaces!
else
nokogiri_original = Nokogiri::XML(native_original_xml) { |config| config.strict.noblanks }
nokogiri_converted = Nokogiri::XML(native_converted_xml) { |config| config.strict.noblanks }
end

ignored_paths = Array.new
arr_paths = Array.new # This array is used to keep track of the paths that lead to arrays that have already been mapped

nokogiri_original.diff(nokogiri_converted, {:added => true, :removed => true}) do |change,node|
split_path = node.parent.path.split('[')
if node.parent.path.include?('[') && !ignored_paths.include?(split_path[0])
ignored_paths << split_path[0]
array_comparison(split_path[0], native_original_hash, native_converted_hash).each do |item|
puts("#{item[0]}: #{item[1]}".ljust(60) + item[2]) if args.disp == 'show'
puts("#{item[0]}: " + item[2]) if args.disp == 'hide'
end
elsif !ignored_paths.include?(split_path[0]) && !path_leads_to_list?(node.parent.path, native_original_hash, native_converted_hash)
element = node.to_xml
path = node.parent.path.split('[')[0]
arr_path = top_level_arr_path(path, native_original_hash, native_converted_hash)

if arr_path && path_not_checked?(arr_path, arr_paths)
arr_paths << arr_path
array_comparison(arr_path, native_original_hash, native_converted_hash).each { |item| add_to_report(item[0], item[1], item[2], args.disp) }
elsif path_not_checked?(path, arr_paths)
if is_xml?(node)
element = Hash.from_xml(node.to_xml)
element = Hash.from_xml(element)
hash_map(element).each do |item|
puts("#{change}: #{item['value']}".ljust(60) + node.parent.path+'/'+item['path']) if args.disp == 'show'
puts("#{change}: " + node.parent.path+'/'+item['path']) if args.disp == 'hide'
arr_path = top_level_arr_path("#{path}/#{item['path']}", native_original_hash, native_converted_hash)
if arr_path && path_not_checked?("#{path}/#{item['path']}", arr_paths)
if path_not_checked?(arr_path, arr_paths)
arr_paths << arr_path
array_comparison(arr_path, native_original_hash, native_converted_hash).each { |item| add_to_report(item[0], item[1], item[2], args.disp) }
end
elsif path_not_checked?("#{path}/#{item['path']}", arr_paths)
add_to_report(change, item['value'], "#{path}/#{item['path']}", args.disp)
end
end
elsif (attr,val = is_attribute?(node))
add_to_report(change, val, "#{path}/#{attr}" , args.disp)
else
puts("#{change}: #{node.to_xml}".ljust(60) + node.parent.path) if args.disp == 'show'
puts("#{change}: " + node.parent.path) if args.disp == 'hide'
add_to_report(change, element, path, args.disp)
end
end
end
end

def path_leads_to_list?(path, org_hash, conv_hash)
# this method takes a path string (and the full original and converted hashes) and outputs true if the path string contains a list; else false
org_hash = hash_navigation(path, org_hash)
conv_hash = hash_navigation(path, conv_hash)

if path.include?("[") && path.include?("]")
bool = true
elsif org_hash.is_a?(Hash) && conv_hash.is_a?(Hash)
# the number of keys must be 1 because all arrays in echo10, dif10, and iso19115 are tagged similar to:
# <Contacts><Contact>contact</Contact></Contacts> and so all array-containing tags will be the plural
# of the array name. This clause serves to identify array-containing tags when their paths aren't properly
# displayed by nokogiri
bool = true if org_hash.keys.length == 1 && org_hash[org_hash.keys[0]].is_a?(Array)
bool = true if conv_hash.keys.length == 1 && conv_hash[conv_hash.keys[0]].is_a?(Array)
elsif org_hash.is_a?(Array) || conv_hash.is_a?(Array)
bool = true
else
bool = false
end
bool
def add_to_report(change, element, path, display)
@counter ||= 0 and @counter += 1
# this function serves to preclude complex nests from forming in loss_report_output the
# following 'if' structure is intended to increase readability by eliminating nests
puts "#{@counter}.".ljust(4)+"#{change}: #{element}".ljust(60) + path + "\n" if display == 'show'
puts "#{@counter}.".ljust(4)+"#{change}: ".ljust(3) + path + "\n" if display == 'hide'
end

def is_xml?(node)
if node.to_xml.include?('<' && '</' && '>') then return true
else return false end
# checks if the node being passed is xml
# may be beneficial to add more checks
node.to_xml.include?('<' && '</' && '>') ? true : false
end

def hash_navigation(path, hash)
# Passed a path string and the hash being navigated. This method parses the path string and
# returns the array/value at the end of the path
path.split('/').each do |key|
if hash.is_a?(Array)
return hash
elsif hash.key?(key) && hash.is_a?(Hash)
hash = hash[key]
end
def is_attribute?(node)
# this method checks if the node being passed is an attribute change;
# TODO: it may be beneficial to add more conditions to improve accuracy
if node.to_xml.include?('=') && !node.to_xml.include?(' = ')
attr_val = Array.new
node.to_xml.split('=').each {|item| attr_val << item.strip.delete('\\"')}
attr_val
else
false
end
hash
end

def hash_map(hash)
buckets = Array.new
hash.each do |key,val|
if val.is_a? Hash
hash_map(val).each do |item|
item['path'] = key + '/' + item['path']
buckets << item
end
else
buckets << {'path'=> key, 'value'=> val}
end
def path_not_checked?(arr_path, arr_paths)
# this method checks the arr_paths array to see if the path being added to
# the report has already been previously evaluated and added
arr_paths.each { |path| return false if arr_path.include?(path) }
true
end

def top_level_arr_path(path, orig_h, conv_h)
# if an array is passed that passes through an array ie. /Contacts/Contact[0]/Role/Name
# this method would return /Contacts/Contact because Contact is the outermost array (or false if the path doesn't contain an array)
pre_translation_array, pre_translation_path = hash_navigation(path, orig_h)
post_translation_array, post_translation_path = hash_navigation(path, conv_h)

return false if pre_translation_array == false && post_translation_array == false
return pre_translation_path if pre_translation_array.is_a?(Array)
return post_translation_path if post_translation_array.is_a?(Array)

# the number of keys must be 1 because all arrays in echo10, dif10, and iso19115 are tagged similar to:
# <Contacts><Contact>contact</Contact></Contacts> and so all array-containing tags will be the plural
# of the array name. This clause serves to identify array-containing tags when their paths aren't properly
# displayed by nokogiri
if pre_translation_array.is_a?(Hash) && pre_translation_array.keys.length == 1 && pre_translation_array[pre_translation_array.keys[0]].is_a?(Array)
return "#{pre_translation_path}/#{pre_translation_array.keys[0]}"
elsif post_translation_array.is_a?(Hash) && post_translation_array.keys.length == 1 && post_translation_array[post_translation_array.keys[0]].is_a?(Array)
return "#{post_translation_path}/#{post_translation_array.keys[0]}"
end
buckets
path_contains_array = false
end

def array_comparison(path, original_hash, converted_hash)
def hash_navigation(path, hash)
# Passed a path string and the hash being navigated. This method parses the path string and
# returns the array/value at the end of the path
current_path = String.new
path.split('/').each do |key|
if hash.is_a?(Array)
return hash, current_path
elsif hash.key?(key) && hash.is_a?(Hash)
current_path += "/#{key}"
hash = hash[key]
elsif !hash.key?(key) && key != ''
return path_exists = false, "#{current_path}/#{key}"
end
end
return hash, current_path
end

def hash_map(hash)
buckets = Array.new
hash.each do |key,val|
if val.is_a? Hash then hash_map(val).each do |item|
item['path'] = key + '/' + item['path']
buckets << item end
else
buckets << {'path'=> key, 'value'=> val}
end
end
buckets
end

def array_comparison(path, original_hash, converted_hash)
pre_translation_array = hash_navigation(path, original_hash)
post_translation_array = hash_navigation(path, converted_hash)

# in the case that a one-item array is parsed as a regular key-value pair instead of an array, an Array wrapper is placed around key-val pair
# so that the following for loops can be executed without error
pre_translation_array.is_a?(Array) ? lost_items_arr = pre_translation_array.clone : lost_items_arr = Array.wrap(pre_translation_array)
pre_translation_array = Array.wrap(pre_translation_array)
post_translation_array.is_a?(Array) ? added_itmes_arr = post_translation_array.clone : added_itmes_arr = Array.wrap(post_translation_array)
post_translation_array = Array.wrap(post_translation_array)

# as defined above, the lost_items_arr and added_itmes_arr are copies of pre_translation_array and post_translation_array, respectively.
# The *_arr values are edited during the comparison between the pre_translation_array and post_translation_array arrays
# and so the *_array arrays are used to maintain a full version of each array for indexing the items in the following lines.

for conv_item in post_translation_array
for org_item in pre_translation_array
if org_item == conv_item
lost_items_arr.delete(org_item)
added_itmes_arr.delete(conv_item)
break
end
end
end
pre_translation_array == false ? pre_translation_array = Array.new : pre_translation_array = Array.wrap(pre_translation_array)
post_translation_array == false ? post_translation_array = Array.new : post_translation_array = Array.wrap(post_translation_array)

output = Array.new
lost_items_arr.each do |item|
(pre_translation_array - post_translation_array).each do |item|
path_with_index = path + "[#{pre_translation_array.index(item)}]"
# the following line is used to eliminate indexing confusion when there is more than one occurrence of a particular item in an array
pre_translation_array[pre_translation_array.index(item)] = item.to_s + ' item indexed'
output << ['-', item, path_with_index]
end


added_itmes_arr.each do |item|
(post_translation_array - pre_translation_array).each do |item|
path_with_index = path + "[#{post_translation_array.index(item)}]"
# the following line is used to eliminate indexing confusion when there is more than one occurrence of a particular item in an array
post_translation_array[post_translation_array.index(item)] = item.to_s + ' item indexed'
output << ['+', item, path_with_index]
end
output
Expand Down