Skip to content

Commit

Permalink
Normalise the costs_awarded field
Browse files Browse the repository at this point in the history
  • Loading branch information
auxesis committed Nov 5, 2016
1 parent ce88f01 commit 7c4e362
Showing 1 changed file with 18 additions and 3 deletions.
21 changes: 18 additions & 3 deletions scraper.rb
Expand Up @@ -48,14 +48,27 @@ def normalise_date(value)
return value
else
puts "[debug] Unhandled date: #{value.inspect}"
raise
end
end

def normalise_trading_name(value)
def normalise_costs_awarded(value)
case
when value.class == String
normalise_string(value)
when value.class == Float
value
else
puts "[debug] Unhandled costs_awarded value: #{value.inspect}"
raise
end
end

def normalise_string(value)
case value
when nil
nil
when /N\\A/
when /^\s*N[\\\/]A\s*$/
nil
else
value.strip
Expand All @@ -76,7 +89,9 @@ def build_prosecution(row)
when 'date_of_conviction'
value = normalise_date(value)
when 'trading_name'
value = normalise_trading_name(value)
value = normalise_string(value)
when 'costs_awarded'
value = normalise_costs_awarded(value)
else
# Remove all leading and trailing whitespace, remove unicode spaces
value.scrub! if value.is_a? String
Expand Down

0 comments on commit 7c4e362

Please sign in to comment.