Permalink
Browse files

adding support for shapefiles that contain UTF8

  • Loading branch information...
1 parent f9a7358 commit af94dee95160e5d0c7cb09ab67279a9d9a3fa82e @maxogden committed Sep 24, 2010
Showing with 11 additions and 2 deletions.
  1. +5 −1 README.textile
  2. +2 −0 Rakefile
  3. +4 −1 bin/shp2geocouch
View
@@ -4,7 +4,7 @@ This is an executable rubygem that creates "GeoCouch":http://www.github.com/vmx/
h2. Installation
-You will need access to a GeoCouch server. For instructions on setting up your own local GeoCouch, check out "my related blog post":http://maxogden.com.
+You will need access to a GeoCouch server. For instructions on setting up your own local GeoCouch, check out "my related blog post":http://maxogden.com/#/2010-08-14-loading-shapefiles-into-geocouch.html.
Install dependencies:
@unzip@, @sed@, @ogr2ogr@
@@ -13,6 +13,10 @@ If you are on Mac OS X, you will already have @unzip@ and @sed@ installed. To ge
Note: @ogr2ogr@ is usually included with the @gdal@ package (sometimes called @gdal-bin@).
+@shp2geocouch@ depends on the following gems:
+* @httparty@ for sending data to CouchDB
+* @couchrest@ for interacting with CouchDB
+
Install the @shp2geocouch@ gem:
@gem install shp2geocouch@
View
@@ -10,6 +10,8 @@ begin
gem.email = "max@maxogden.com"
gem.homepage = "http://github.com/maxogden/shp2geocouch"
gem.authors = ["Max Ogden"]
+ gem.add_dependency 'httparty'
+ gem.add_dependency 'couchrest'
end
Jeweler::GemcutterTasks.new
rescue LoadError
View
@@ -2,6 +2,7 @@
require 'httparty'
require 'couchrest'
require 'optparse'
+require 'iconv'
class ShapefileToGeoCouch
attr_accessor :path, :extension, :name, :couch_url, :cleanup, :verbose
@@ -74,9 +75,11 @@ class ShapefileToGeoCouch
end
def upload
+ ic = Iconv.new('UTF-8//IGNORE', 'UTF-8') # disregard all UTF8 characters
puts "Bulk loading data into GeoCouch... view progress at #{@couch_url}/_utils" if @verbose
File.open(bulk).each_line do |line|
- HTTParty.post(database_url + '/_bulk_docs', :body => '{"docs": [' + line[0..-3] + "]}", :headers => {'content-type' => "application/json"})
+ valid_string = ic.iconv(line[0..-3] + ' ')[0..-3] # http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/
+ HTTParty.post(database_url + '/_bulk_docs', :body => '{"docs": [' + valid_string + "]}", :headers => {'content-type' => "application/json"})
end
end

0 comments on commit af94dee

Please sign in to comment.