Logstash tcp input hanging/buffering behavior

[surein]

I configured TCP Input client to external host. The connection is successful. All the events over tcp socket are getting appended to a single line.After force shutdown of logstash i can see the messages getting filtered.

Config:
input  
{
        tcp{
                mode => "client"
                host => "xx.xx.xx.xx"
                port => xxxx
           }
}
filter{
 grok {
    match => { "message" => "xxxxxxxxxxxxxxxxxxxxxxxxxxx" }
  }
}
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

Log:

Flushing buffer at interval {:instance=>"#<LogStash::Outputs::ElasticSearch::Buffer:0x12cc8b6 @operations_mutex=#Mutex:0x1eb3c63, @max_size=500, @operations_lock=#Java::JavaUtilConcurrentLocks::ReentrantLock:0x1d4a3c6, @submit_proc=#Proc:0x18c34cc@/home/suri/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:57, @logger=#<Cabin::Channel:0x1060dba @metrics=#<Cabin::Metrics:0x1b6518f @metrics_lock=#Mutex:0xf9c917, @metrics={}, @channel=#<Cabin::Channel:0x1060dba ...>>, @subscriber_lock=#Mutex:0x1bb8d0, @Level=:debug, @subscribers={12678=>#<Cabin::Subscriber:0xb21e7f @output=#<Cabin::Outputs::IO:0xb7d74f @io=#<IO:fd 1>, @lock=#Mutex:0x1052d71>, @options={}>}, @DaTa={}>, @last_flush=2016-07-27 01:36:11 +0530, @flush_interval=1, @stopping=#Concurrent::AtomicBoolean:0x582b1d, @buffer=[], @flush_thread=#<Thread:0xc3808f run>>", :interval=>1, :level=>:debug, :file=>"logstash/outputs/elasticsearch/buffer.rb", :line=>"90", :method=>"interval_flush"}

# ^CSIGINT received. Shutting down the agent. {:level=>:warn, :file=>"logstash/agent.rb", :line=>"322", :method=>"trap_sigint"}

After Force Shutdown

I can see the the tcp event getting filtered

{
                "message" => "xxxxxxxxxxxxxxxxxxx",
               "@version" => "1",
             "@timestamp" => "2016-07-26T07:36:11.770Z",
                   "host" => "10.105.73.71",
                   "port" => 37002,
           "RequestEvent" => "27",
                 "DcName" => "TON0",
                  "GSSNo" => "GSS2",
                  "PlcNo" => "PLC03",
               "CartonID" => "????????????????????",
                 "Weight" => "000.00",
                    "PID" => "00001",
                     "ts" => "2016-07-26 13:06:11.769",
              "AssnEvent" => "28",
         "AssEventDcName" => "TON0",
          "AssEventGSSNo" => "GSS2",
         "AssnEventPlcNo" => "PLC03",
    "AssnEventlaneNumber" => "00004",
           "totalAssigns" => "000000037",
      "AssnEventCartonID" => "????????????????????",
           "AssnEventPID" => "00001",
            "targetPlcNo" => "PLC03",
       "targetlaneNumber" => "00004",
            "AssnEventTs" => "2016-07-26 13:06:11.770"
}

[rukai-kooboo]

Hi Suyograo,
I also meet this issue, I can see the tcp message, but I can't see it is pushed to Elastic Search.

Does it matter with the @maxsize=500?

I test log size <500 can be sent to ES successfully, but >500 can't be.

[jakelandis]

Closing this as non-actionable.

Both the TCP input and Elasticsearch output have undergone major changes since this issue was logged.

If this is still an issue please open a new issue and include the plugin versions in use.