Permalink
Browse files

Switch to use ActiveSupport::Inflector.transliterate for transliterat…

…ion to ASCII

This also

- Removes reliance on iconv (fixes deprecation warnings when running under Ruby 1.9)
- No longer works with Ruby 1.8
- This changes the way you configure conversion between encodings
  • Loading branch information...
1 parent 63e7f88 commit 39c961db1cab55739f218339d20c5419c7c330ea @mocoso mocoso committed with mocoso May 24, 2012
View
11 README.md
@@ -13,7 +13,7 @@ application.
The current version of CSV Builder works with:
* Rails 3.x
-* Ruby 1.8 or 1.9
+* Ruby 1.9
* Unicorn _is required for streaming_ see [the example streaming app](https://github.com/fawce/test_csv_streamer) for more details.
The legacy version (1.1.x) was originally developed and tested for Rails 2.1. See [the legacy
@@ -48,14 +48,9 @@ your controller's action method e.g.
@filename = 'report.csv'
-You can set the input encoding and output encoding by setting `@input_encoding` and `@output_encoding` instance
-variables. These default to 'UTF-8' and 'LATIN1' respectively. e.g.
+You can set `@csv_options` instance variable to define options for CSV generation. For example:
- @output_encoding = 'UTF-8'
-
-You can set `@csv_options` instance variable to define options for FasterCSV generator. For example:
-
- @csv_options = { :force_quotes => true, :col_sep => ';' }
+ @csv_options = { :force_quotes => true, :col_sep => ';', :encoding => Encoding::UTF_8 }
You can optionally stream your results line by line as they are generated. Results will stream if the underlying Rack server supports streaming, otherwise the results will be buffered and sent when the template finishes rendering. Just set `@streaming` to true:
View
4 csv_builder.gemspec
@@ -50,7 +50,6 @@ Gem::Specification.new do |s|
s.homepage = %q{http://github.com/dasil003/csv_builder}
s.licenses = [%q{MIT}]
s.require_paths = [%q{lib}]
- s.requirements = [%q{iconv}]
s.rubygems_version = %q{1.8.6}
s.summary = %q{CSV template handler for Rails}
@@ -59,6 +58,7 @@ Gem::Specification.new do |s|
if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
s.add_runtime_dependency(%q<actionpack>, [">= 3.0.0"])
+ s.add_runtime_dependency('activesupport', [">= 3.0.0"])
s.add_runtime_dependency('fastercsv') if RUBY_VERSION.to_f < 1.9
s.add_development_dependency(%q<rails>, [">= 3.0.0"])
s.add_development_dependency(%q<rspec>, ["~> 2.5"])
@@ -67,6 +67,7 @@ Gem::Specification.new do |s|
s.add_development_dependency(%q<sqlite3>, [">= 0"])
else
s.add_dependency(%q<actionpack>, [">= 3.0.0"])
+ s.add_dependency('activesupport', [">= 3.0.0"])
s.add_dependency('fastercsv') if RUBY_VERSION.to_f < 1.9
s.add_dependency(%q<rails>, [">= 3.0.0"])
s.add_dependency(%q<rspec>, ["~> 2.5"])
@@ -76,6 +77,7 @@ Gem::Specification.new do |s|
end
else
s.add_dependency(%q<actionpack>, [">= 3.0.0"])
+ s.add_dependency('activesupport', [">= 3.0.0"])
s.add_dependency('fastercsv') if RUBY_VERSION.to_f < 1.9
s.add_dependency(%q<rails>, [">= 3.0.0"])
s.add_dependency(%q<rspec>, ["~> 2.5"])
View
1 lib/csv_builder.rb
@@ -11,7 +11,6 @@ module CsvBuilder
end
require 'action_view'
-require 'iconv'
require 'csv_builder/transliterating_filter'
require 'csv_builder/template_handler'
require 'csv_builder/railtie'
View
74 lib/csv_builder/template_handler.rb
@@ -17,18 +17,18 @@ module CsvBuilder # :nodoc:
#
# @filename = 'report.csv'
#
- # You can also set the input encoding and output encoding by setting
- # <tt>@input_encoding</tt> and <tt>@output_encoding</tt> instance variables.
- # These default to 'UTF-8' and 'LATIN1' respectively. e.g.
+ # You can also set the output encoding by setting <tt>:encoding</tt> on the
+ # <tt>@csv_options</tt> hash.
+ # E.g.
#
- # @output_encoding = 'UTF-8'
+ # @csv_options = { :encoding => Encoding::US_ASCII }
if defined?(Rails) and Rails.version < '3'
class TemplateHandler < ActionView::Template::Handler
include ActionView::Template::Handlers::Compilable
end
end
-
+
# The ruby csv class will try to infer a separator to use, if the csv options
# do not set it. ruby's csv calls pos, eof?, read, and rewind to check the first line
# of the io to infer a separator. Rails' output object does not support these methods
@@ -40,59 +40,77 @@ class Yielder
def initialize(yielder)
@yielder = yielder
end
-
+
# always indicate that we are at the start of the io stream
def pos
return 0
end
-
+
# always indicate that we have reached the end of the file
def eof?
return true
end
-
+
#do nothing, we haven't moved forward
def rewind
end
-
+
#despite indicating that we have no data with pos and eof, we still need to return a newline
#otherwise CSV will enter an infinite loop with read.
def read(arg1)
return "\n"
end
-
+
# this is the method that ultimately yields to the block with output.
# the block is passed by Rails into the Streamer class' each method.
- # Streamer provides a Proc to this class, which simply invokes yield
+ # Streamer provides a Proc to this class, which simply invokes yield
# from within the context of the each block.
def <<(data)
@yielder.call data
end
-
+
end
-
+
# Streamer implements an each method to facilitate streaming back through the Rails stack. It requires
# the template to be passed to it as a proc. An instance of this class is returned from the template handler's
# compile method, and will receive calls to each. Data is streamed by yielding back to the containing block.
class Streamer
- def initialize(template_proc)
+ def initialize(template_proc, csv_options, transliterate)
@template_proc = template_proc
+ self.csv_options = csv_options
+ self.transliterate = transliterate
end
-
+
def each
yielder = CsvBuilder::Yielder.new(Proc.new{|data| yield data})
- csv_stream = CsvBuilder::CSV_LIB.new(yielder, @csv_options || {})
- csv = CsvBuilder::TransliteratingFilter.new(csv_stream, @input_encoding || 'UTF-8', @output_encoding || 'LATIN1')
+ csv_stream = CsvBuilder::CSV_LIB.new(yielder, csv_options)
+
+ if transliterate
+ csv = CsvBuilder::TransliteratingFilter.new csv_stream
+ else
+ csv = csv_stream
+ end
+
@template_proc.call(csv)
end
+
+ private
+ attr_accessor :csv_options, :transliterate
end
-
+
class TemplateHandler
def self.call(template)
-
+
<<-EOV
begin
-
+ if @csv_options && @csv_options[:encoding] == Encoding::US_ASCII
+ csv_options = @csv_options.except(:encoding)
+ transliterate = true
+ else
+ csv_options = @csv_options || {}
+ transliterate = false
+ end
+
unless defined?(ActionMailer) && defined?(ActionMailer::Base) && controller.is_a?(ActionMailer::Base)
@filename ||= "\#{controller.action_name}.csv"
if controller.request.env['HTTP_USER_AGENT'] =~ /msie/i
@@ -107,15 +125,19 @@ def self.call(template)
response.headers["Content-Transfer-Encoding"] = "binary"
end
end
-
+
if @streaming
template = Proc.new {|csv|
#{template.source}
}
- CsvBuilder::Streamer.new(template)
- else
- output = CsvBuilder::CSV_LIB.generate(@csv_options || {}) do |faster_csv|
- csv = CsvBuilder::TransliteratingFilter.new(faster_csv, @input_encoding || 'UTF-8', @output_encoding || 'LATIN1')
+ CsvBuilder::Streamer.new(template, csv_options, transliterate)
+ else
+ output = CsvBuilder::CSV_LIB.generate(csv_options) do |faster_csv|
+ if transliterate
+ csv = CsvBuilder::TransliteratingFilter.new(faster_csv)
+ else
+ csv = faster_csv
+ end
#{template.source}
end
output
@@ -126,7 +148,7 @@ def self.call(template)
end
EOV
end
-
+
def compile(template)
self.class.call(template)
end
View
26 lib/csv_builder/transliterating_filter.rb
@@ -1,23 +1,17 @@
-#encoding: utf-8
-
+# Transliterates CSV using ActiveSupport::Inflector.transliterate if the desired
+# output encoding is ASCII
+#
+# This should do a reasonable job of converting the input into ASCII even if
+# some characters don't have a good match
class CsvBuilder::TransliteratingFilter
- # Transliterate into the required encoding if necessary
- def initialize(faster_csv, input_encoding = 'UTF-8', output_encoding = 'LATIN1')
- @faster_csv = faster_csv
-
- # TODO: do some checking to make sure iconv works correctly in
- # current environment. See ActiveSupport::Inflector#transliterate
- # definition for details
- #
- # Not using the more standard //IGNORE//TRANSLIT because it raises
- # Iconv::IllegalSequence for some inputs
- @iconv = Iconv.new("#{output_encoding}//TRANSLIT//IGNORE", input_encoding) if input_encoding != output_encoding
+ def initialize(csv)
+ @csv = csv
end
- # Transliterate before passing to FasterCSV so that the right characters (e.g. quotes) get escaped
def <<(row)
- @faster_csv << if @iconv then row.map { |value| @iconv.iconv(value.to_s) } else row end
+ # Transliterate before passing to FasterCSV so that the right characters (e.g.
+ # quotes) get escaped
+ @csv << row.map { |value| ActiveSupport::Inflector.transliterate value }
end
-
alias :add_row :<<
end
View
2 spec/controllers/csv_builder_spec.rb
@@ -25,7 +25,7 @@ def complex
def encoding
respond_to do |format|
- format.csv { @output_encoding = params[:encoding] }
+ format.csv { @csv_options = { :encoding => Encoding.find(params[:encoding]) } }
end
end

0 comments on commit 39c961d

Please sign in to comment.