Make alternative of FileUtils

[tagomoris]

Updated: StringUtil.match_regexp is now easy to replace with Fluent::Plugin::Base#string_safe_encoding.
There are still Fluent::FileUtil.writable? and #writable_p?.

Fluent::Plugin::StringUtil.match_regexp (implementation was moved to Fluent::Compat::StringUtil now) has some implementation problem:

• it provides encode-safety about regular expression matching, but method name doesn't show it
• it uses $log, but should use plugin logger
• scrubbing strings is very universal process: not only for regular expression matching

My idea is to add a method like this:

def ensure_safe_encoding(str)
  begin
    yield str
  rescue ArgumentError => e
    raise e unless e.message.include?("invalid byte sequence in".freeze)
    log.info "invalid byte sequence is replaced in `#{string}`"
    str = str.scrub('?')
    retry
  end
end

We can use this method like below:

m = ensure_safe_encoding(line){|s| s =~ /my pattern/}
# or other methods or ...
chars  = ensure_safe_encoding(line){|s| s.split }

Blockers for implementation are:

• Naming problem of the method
• Namespace? lib/fluent? lib/fluent/util? or...
• Should it be a method of Fluent::Plugin::Base? plugin helper? (If true, we'll have so many helpers...) or just Mixin? (We're eliminating mixins, but...)

[tagomoris]

match_regexp is used in:

• lib/fluent/plugin/parser.rb
• lib/fluent/plugin/filter_grep.rb

[nurse]

You can write just like following. Note that String#valid_encoding? uses a cache in VALUE if the string has already checked the validity.

unless string.valid_encoding?
  log.info "invalid byte sequence is replaced in `#{string}`"
  string = string.scrub('?')
end

[tagomoris]

There's a same issue about FileUtil.

• FileUtil.writable? does different check from File.writable?... we should use another method name