Fluentd 1.10.0 cannot launch when ruby command is placed in a directory which constains spaces

[ashie]

Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.

Describe the bug

Fluentd 1.10.0 cannot launch when ruby command is placed in a directory which contains spaces.

-e.g.) C:\Program Files (x86)\td-agent\bin\ruby

To Reproduce

• Download a MSI installer which is built by td-agent-builder
  • https://github.com/clear-code/td-agent-builder/actions/runs/63706658
• Run td-agent-3.6.0-x64.msi in the zip package
• Install it into C:\Program Files (x86) or C:\foo bar or something which contains one or more spaces.
• Run td-agent from Td-agent Command Prompt

Expected behavior

Fluentd can launch from a directory which constains spaces.

Your Environment

• fluentd 1.10.0
• Windows 10

Your Configuration

• https://github.com/clear-code/td-agent-builder/blob/master/td-agent/templates/etc/td-agent/td-agent.conf
  • Same with https://github.com/treasure-data/omnibus-td-agent/blob/master/templates/etc/td-agent/td-agent.conf

Your Error Log

2020-03-26 23:13:19 +0900 [info]: parsing config file is succeeded path="C:\\Program Files (x86)\\td-agent\\bin\\\\..\\etc\\td-agent\\td-agent.conf"
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-elasticsearch' version '4.0.3'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-kafka' version '0.12.3'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-parser-winevt_xml' version '0.2.1'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-prometheus' version '1.7.3'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.0.1'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.2.0'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-s3' version '1.3.0'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-td' version '1.1.0'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.4'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-webhdfs' version '1.2.4'
2020-03-26 23:13:19 +0900 [info]: gem 'fluent-plugin-windows-eventlog' version '0.5.0'
2020-03-26 23:13:19 +0900 [info]: gem 'fluentd' version '1.10.0'
2020-03-26 23:13:19 +0900 [info]: gem 'fluentd' version '1.9.2'
2020-03-26 23:13:20 +0900 [warn]: [output_td] Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2020-03-26 23:13:20 +0900 [info]: using configuration file: <ROOT>
  <match td.*.*>
    @type tdlog
    @id output_td
    apikey xxxxxx
    auto_create_table
    <buffer>
      @type "file"
      path "/var/log/td-agent/buffer/td"
    </buffer>
    <secondary>
      @type "file"
      path "/var/log/td-agent/failed_records"
      <buffer time>
        path /var/log/td-agent/failed_records
      </buffer>
    </secondary>
  </match>
  <match debug.**>
    @type stdout
    @id output_stdout
  </match>
  <source>
    @type forward
    @id input_forward
  </source>
  <source>
    @type http
    @id input_http
    port 8888
  </source>
  <source>
    @type debug_agent
    @id input_debug_agent
    bind "127.0.0.1"
    port 24230
  </source>
</ROOT>
2020-03-26 23:13:20 +0900 [info]: starting fluentd-1.10.0 pid=16228 ruby="2.4.9"
C:/Program Files (x86)/td-agent/lib/ruby/2.4.0/open3.rb:199:in `spawn': No such file or directory - C:/Program Files (x86)/td-agent/bin/ruby.exe -Eascii-8bit:ascii-8bit -h (Errno::ENOENT)
        from C:/Program Files (x86)/td-agent/lib/ruby/2.4.0/open3.rb:199:in `popen_run'
        from C:/Program Files (x86)/td-agent/lib/ruby/2.4.0/open3.rb:95:in `popen3'
        from C:/Program Files (x86)/td-agent/lib/ruby/2.4.0/open3.rb:258:in `capture3'
        from C:/Program Files (x86)/td-agent/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0-x64-mingw32/lib/fluent/supervisor.rb:901:in `build_spawn_command'
        from C:/Program Files (x86)/td-agent/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0-x64-mingw32/lib/fluent/supervisor.rb:672:in `supervise'
        from C:/Program Files (x86)/td-agent/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0-x64-mingw32/lib/fluent/supervisor.rb:558:in `run_supervisor'
        from C:/Program Files (x86)/td-agent/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0-x64-mingw32/lib/fluent/command/fluentd.rb:330:in `<top (required)>'
        from C:/Program Files (x86)/td-agent/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from C:/Program Files (x86)/td-agent/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from C:/Program Files (x86)/td-agent/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0-x64-mingw32/bin/fluentd:8:in `<top (required)>'
        from C:/Program Files (x86)/td-agent/bin/fluentd:23:in `load'
        from C:/Program Files (x86)/td-agent/bin/fluentd:23:in `<main>'

Additional context

I'm now developing a new build system for td-agent. In this work I've allowed to install td-agent into any folder by this commit :fluent/fluent-package-builder@c3dfc08
When I tested installing it into C:\Program Files\ or somewhere which contains spaces, I got this error.

Following quick hack fixes the issue for me:

diff --git a/lib/fluent/supervisor.rb b/lib/fluent/supervisor.rb
index b02b5f62..0bbbea9b 100644
--- a/lib/fluent/supervisor.rb
+++ b/lib/fluent/supervisor.rb
@@ -897,7 +897,7 @@ module Fluent

       # Adding `-h` so that it can avoid ruby's command blocking
       # e.g. `ruby -Eascii-8bit:ascii-8bit` will block. but `ruby -Eascii-8bit:ascii-8bit -h` won't.
-      cmd = fluentd_spawn_cmd.join(' ')
+      cmd = fluentd_spawn_cmd.collect{ |arg| "\"#{arg}\"" }.join(' ')
       _, e, s = Open3.capture3("#{cmd} -h")
       if s.exitstatus != 0
         $log.error('Invalid option is passed to RUBYOPT', command: cmd, error: e)

[repeatedly]

For patch, how about using Fluent.windows? to separate the code?
It avoids unexpected change on non-windows environment.