adding support for shapefiles that contain UTF8

README.textile

@@ -4,7 +4,7 @@ This is an executable rubygem that creates "GeoCouch":http://www.github.com/vmx/
 
 h2. Installation
 
-You will need access to a GeoCouch server. For instructions on setting up your own local GeoCouch, check out "my related blog post":http://maxogden.com.
+You will need access to a GeoCouch server. For instructions on setting up your own local GeoCouch, check out "my related blog post":http://maxogden.com/#/2010-08-14-loading-shapefiles-into-geocouch.html.
 
 Install dependencies:
 @unzip@, @sed@, @ogr2ogr@
@@ -13,6 +13,10 @@ If you are on Mac OS X, you will already have @unzip@ and @sed@ installed. To ge
 
 Note: @ogr2ogr@ is usually included with the @gdal@ package (sometimes called @gdal-bin@).
 
+@shp2geocouch@ depends on the following gems:
+* @httparty@ for sending data to CouchDB
+* @couchrest@ for interacting with CouchDB
+
 Install the @shp2geocouch@ gem:
 
 @gem install shp2geocouch@

Rakefile

@@ -10,6 +10,8 @@ begin
     gem.email = "max@maxogden.com"
     gem.homepage = "http://github.com/maxogden/shp2geocouch"
     gem.authors = ["Max Ogden"]
+    gem.add_dependency 'httparty'
+    gem.add_dependency 'couchrest'
   end
   Jeweler::GemcutterTasks.new
 rescue LoadError

bin/shp2geocouch

@@ -2,6 +2,7 @@
 require 'httparty'
 require 'couchrest'
 require 'optparse'
+require 'iconv'
 
 class ShapefileToGeoCouch
   attr_accessor :path, :extension, :name, :couch_url, :cleanup, :verbose
@@ -74,9 +75,11 @@ class ShapefileToGeoCouch
   end
 
   def upload
+    ic = Iconv.new('UTF-8//IGNORE', 'UTF-8') # disregard all UTF8 characters
     puts "Bulk loading data into GeoCouch... view progress at #{@couch_url}/_utils" if @verbose
     File.open(bulk).each_line do |line|
-      HTTParty.post(database_url + '/_bulk_docs', :body => '{"docs": [' + line[0..-3] + "]}", :headers => {'content-type' => "application/json"})
+      valid_string = ic.iconv(line[0..-3] + '  ')[0..-3] # http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/
+      HTTParty.post(database_url + '/_bulk_docs', :body => '{"docs": [' + valid_string + "]}", :headers => {'content-type' => "application/json"})
      end
   end