phpfpm plugin doesn't return any data

[sofixa]

I have tried using the phpfpm plugin, and altough it runs successfully, it doesn't return any data.
Here's the output from the command:

~$ /opt/telegraf/telegraf -config /etc/opt/telegraf/telegraf.conf -test -debug -filter phpfpm

* Plugin: phpfpm, Collection 1

Without -test i have:

2016/01/10 21:46:30 Gathered metrics, ({30s} interval), from 1 plugins in 13.174625ms

So, in theory, it should work, but in InfluxDB i have absolutely nothing.
PHP-FPM is working, and i have used it, and when running a script that shows the status of PHP-FPM i get plenty of data:

~# ./phpfpmstatus.sh /var/run/php5-fpm.sock
X-Powered-By: PHP/5.5.9-1ubuntu4.14
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache, no-store, must-revalidate, max-age=0
Content-Type: text/plain

pool:                 www
process manager:      dynamic
start time:           10/Jan/2016:21:50:58 +0000
start since:          111
accepted conn:        15
listen queue:         0
max listen queue:     0
listen queue len:     0
idle processes:       1
active processes:     1
total processes:      2
max active processes: 2
max children reached: 0
slow requests:        0

What could be the problem? What metrics should the phpfpm plugin collect?

[sparrc]

@sofixa your test output isn't in fact working, it should be showing measurements

@kureikain any ideas why he wouldn't be getting any metrics?

[v9n]

@sparrc My apologize for not answering sooner.

@sofixa Hi, Can you post your example config for fpm. It seems you are using unixsocket method.

My guess is this a permission issue. Telegraf may run under telegraf user. And it doesn't have permission to access socket. From your output, I can see that you are running under root to get stats.

Here is some information for further output(to verify user):

ps aux | grep telegaf
ps aux | grep fpm

also, you can try to run this command under same account that you run telegraf to see if you are able to read it. Don't run it under root as you did

$  ./phpfpmstatus.sh /var/run/php5-fpm.sock

If you use http method, then you don't have to worry about permission because it doesn't read the socket directly.

To verify, you may try to run telegraf under root account again and see if it collects any points.

When using unixsocket, I add telegraf user to www-data group so that I can read socket.

telegraf@axcoto:/opt/telegraf$ groups
telegraf www-data

[sofixa]

Hi, thanks for your response.
I thought it might be a rights issue before, when i was receiving an error, so my unix socket is in 666:

ubuntu@aws:~$ ls -la /var/run/php5-fpm.sock
srw-rw-rw- 1 www-data www-data 0 Jan 11 20:15 /var/run/php5-fpm.sock

My config is simple:

urls = ["localhost:/var/run/php5-fpm.sock"]

Note:without the localhost: bit i was getting an error, which wasn't due to a rights issue, since i have been running my sock as 666.
Furthermore, i have also ran telegraf under root, for test and debug, and it doesn't work either.

And sadly i can't use the http method, since my site has lots of redirections, and i'd have to spend some considerable time trying to find which one i'd need to add an exception to so that my status page stays visible(it's a Wordpress site with i18n and CDN caused redirections).

[lassizci]

I have noticed the same with fcgi/cgi. Works through nginx:

location /status {
        access_log off;
        allow 127.0.0.1;
        deny all;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_read_timeout 5s;
        fastcgi_param SCRIPT_FILENAME /status;
        include fastcgi_params;
}

I have tried with all of these, one at a time or course:
[[plugins.phpfpm]]
urls = ["fcgi://127.0.0.1:9000/status"]
#urls = ["cgi://127.0.0.1:9000/status"]
#urls = ["http://127.0.0.1/status"]

only the http url returns metrics. I get no errors, just the line * Plugin: phpfpm, Collection 1 and nothing after that.

[v9n]

@lassizci fcgi ideally should works. I don't write that part of code. Will look into it. Thanks for your Nginx configuration to limit access.

@sofixa As @lassizci pointed out, you can use deny all and allow before it to allow access from some ip such as 127.0.0.1.

Note:without the localhost: bit i was getting an error, which wasn't due to a rights issue, since i have been running my sock as 666.

Will push a fix for this.

I still not sure why it won't work even when you run under root. Will you able to compile this file and run it, it will output some info, I probably should return those error report into the plugin itself.

File main.go:

Content:

package main

// FastCGI client to request via socket

// Copyright 2012 Junqing Tan <ivan@mysqlab.net> and The Go Authors
// Use of this source code is governed by a BSD-style
// Part of source code is from Go fcgi package

// Fix bug: Can't recive more than 1 record untill FCGI_END_REQUEST 2012-09-15
// By: wofeiwo

import (
    "bufio"
    "bytes"
    "encoding/binary"
    "errors"
    "io"
    "log"
    "net"
    "strconv"
    "sync"
)

const FCGI_LISTENSOCK_FILENO uint8 = 0
const FCGI_HEADER_LEN uint8 = 8
const VERSION_1 uint8 = 1
const FCGI_NULL_REQUEST_ID uint8 = 0
const FCGI_KEEP_CONN uint8 = 1

const (
    FCGI_BEGIN_REQUEST uint8 = iota + 1
    FCGI_ABORT_REQUEST
    FCGI_END_REQUEST
    FCGI_PARAMS
    FCGI_STDIN
    FCGI_STDOUT
    FCGI_STDERR
    FCGI_DATA
    FCGI_GET_VALUES
    FCGI_GET_VALUES_RESULT
    FCGI_UNKNOWN_TYPE
    FCGI_MAXTYPE = FCGI_UNKNOWN_TYPE
)

const (
    FCGI_RESPONDER uint8 = iota + 1
    FCGI_AUTHORIZER
    FCGI_FILTER
)

const (
    FCGI_REQUEST_COMPLETE uint8 = iota
    FCGI_CANT_MPX_CONN
    FCGI_OVERLOADED
    FCGI_UNKNOWN_ROLE
)

const (
    FCGI_MAX_CONNS  string = "MAX_CONNS"
    FCGI_MAX_REQS   string = "MAX_REQS"
    FCGI_MPXS_CONNS string = "MPXS_CONNS"
)

const (
    maxWrite = 6553500 // maximum record body
    maxPad   = 255
)

type header struct {
    Version       uint8
    Type          uint8
    Id            uint16
    ContentLength uint16
    PaddingLength uint8
    Reserved      uint8
}

// for padding so we don't have to allocate all the time
// not synchronized because we don't care what the contents are
var pad [maxPad]byte

func (h *header) init(recType uint8, reqId uint16, contentLength int) {
    h.Version = 1
    h.Type = recType
    h.Id = reqId
    h.ContentLength = uint16(contentLength)
    h.PaddingLength = uint8(-contentLength & 7)
}

type record struct {
    h   header
    buf [maxWrite + maxPad]byte
}

func (rec *record) read(r io.Reader) (err error) {
    if err = binary.Read(r, binary.BigEndian, &rec.h); err != nil {
        return err
    }
    if rec.h.Version != 1 {
        return errors.New("fcgi: invalid header version")
    }
    n := int(rec.h.ContentLength) + int(rec.h.PaddingLength)
    if _, err = io.ReadFull(r, rec.buf[:n]); err != nil {
        return err
    }
    return nil
}

func (r *record) content() []byte {
    return r.buf[:r.h.ContentLength]
}

type FCGIClient struct {
    mutex     sync.Mutex
    rwc       io.ReadWriteCloser
    h         header
    buf       bytes.Buffer
    keepAlive bool
}

func NewClient(h string, args ...interface{}) (fcgi *FCGIClient, err error) {
    var conn net.Conn
    if len(args) != 1 {
        err = errors.New("fcgi: not enough params")
        return
    }
    switch args[0].(type) {
    case int:
        addr := h + ":" + strconv.FormatInt(int64(args[0].(int)), 10)
        conn, err = net.Dial("tcp", addr)
    case string:
        laddr := net.UnixAddr{Name: args[0].(string), Net: h}
        conn, err = net.DialUnix(h, nil, &laddr)
    default:
        err = errors.New("fcgi: we only accept int (port) or string (socket) params.")
    }
    fcgi = &FCGIClient{
        rwc:       conn,
        keepAlive: false,
    }
    return
}

func (client *FCGIClient) writeRecord(recType uint8, reqId uint16, content []byte) (err error) {
    client.mutex.Lock()
    defer client.mutex.Unlock()
    client.buf.Reset()
    client.h.init(recType, reqId, len(content))
    if err := binary.Write(&client.buf, binary.BigEndian, client.h); err != nil {
        return err
    }
    if _, err := client.buf.Write(content); err != nil {
        return err
    }
    if _, err := client.buf.Write(pad[:client.h.PaddingLength]); err != nil {
        return err
    }
    _, err = client.rwc.Write(client.buf.Bytes())
    return err
}

func (client *FCGIClient) writeBeginRequest(reqId uint16, role uint16, flags uint8) error {
    b := [8]byte{byte(role >> 8), byte(role), flags}
    return client.writeRecord(FCGI_BEGIN_REQUEST, reqId, b[:])
}

func (client *FCGIClient) writeEndRequest(reqId uint16, appStatus int, protocolStatus uint8) error {
    b := make([]byte, 8)
    binary.BigEndian.PutUint32(b, uint32(appStatus))
    b[4] = protocolStatus
    return client.writeRecord(FCGI_END_REQUEST, reqId, b)
}

func (client *FCGIClient) writePairs(recType uint8, reqId uint16, pairs map[string]string) error {
    w := newWriter(client, recType, reqId)
    b := make([]byte, 8)
    for k, v := range pairs {
        n := encodeSize(b, uint32(len(k)))
        n += encodeSize(b[n:], uint32(len(v)))
        if _, err := w.Write(b[:n]); err != nil {
            return err
        }
        if _, err := w.WriteString(k); err != nil {
            return err
        }
        if _, err := w.WriteString(v); err != nil {
            return err
        }
    }
    w.Close()
    return nil
}

func readSize(s []byte) (uint32, int) {
    if len(s) == 0 {
        return 0, 0
    }
    size, n := uint32(s[0]), 1
    if size&(1<<7) != 0 {
        if len(s) < 4 {
            return 0, 0
        }
        n = 4
        size = binary.BigEndian.Uint32(s)
        size &^= 1 << 31
    }
    return size, n
}

func readString(s []byte, size uint32) string {
    if size > uint32(len(s)) {
        return ""
    }
    return string(s[:size])
}

func encodeSize(b []byte, size uint32) int {
    if size > 127 {
        size |= 1 << 31
        binary.BigEndian.PutUint32(b, size)
        return 4
    }
    b[0] = byte(size)
    return 1
}

// bufWriter encapsulates bufio.Writer but also closes the underlying stream when
// Closed.
type bufWriter struct {
    closer io.Closer
    *bufio.Writer
}

func (w *bufWriter) Close() error {
    if err := w.Writer.Flush(); err != nil {
        w.closer.Close()
        return err
    }
    return w.closer.Close()
}

func newWriter(c *FCGIClient, recType uint8, reqId uint16) *bufWriter {
    s := &streamWriter{c: c, recType: recType, reqId: reqId}
    w := bufio.NewWriterSize(s, maxWrite)
    return &bufWriter{s, w}
}

// streamWriter abstracts out the separation of a stream into discrete records.
// It only writes maxWrite bytes at a time.
type streamWriter struct {
    c       *FCGIClient
    recType uint8
    reqId   uint16
}

func (w *streamWriter) Write(p []byte) (int, error) {
    nn := 0
    for len(p) > 0 {
        n := len(p)
        if n > maxWrite {
            n = maxWrite
        }
        if err := w.c.writeRecord(w.recType, w.reqId, p[:n]); err != nil {
            return nn, err
        }
        nn += n
        p = p[n:]
    }
    return nn, nil
}

func (w *streamWriter) Close() error {
    // send empty record to close the stream
    return w.c.writeRecord(w.recType, w.reqId, nil)
}

func (client *FCGIClient) Request(env map[string]string, reqStr string) (retout []byte, reterr []byte, err error) {

    var reqId uint16 = 1
    defer client.rwc.Close()

    err = client.writeBeginRequest(reqId, uint16(FCGI_RESPONDER), 0)
    if err != nil {
        return
    }
    err = client.writePairs(FCGI_PARAMS, reqId, env)
    if err != nil {
        return
    }
    if len(reqStr) > 0 {
        err = client.writeRecord(FCGI_STDIN, reqId, []byte(reqStr))
        if err != nil {
            return
        }
    }

    rec := &record{}
    var err1 error

    // recive untill EOF or FCGI_END_REQUEST
    for {
        err1 = rec.read(client.rwc)
        if err1 != nil {
            if err1 != io.EOF {
                err = err1
            }
            break
        }
        switch {
        case rec.h.Type == FCGI_STDOUT:
            retout = append(retout, rec.content()...)
        case rec.h.Type == FCGI_STDERR:
            reterr = append(reterr, rec.content()...)
        case rec.h.Type == FCGI_END_REQUEST:
            fallthrough
        default:
            break
        }
    }

    return
}

func main() {
    fcgi, _ := NewClient("unix", "/var/run/php5-fpm.sock")
    resOut, resErr, err := fcgi.Request(map[string]string{
        "SCRIPT_NAME":     "/status",
        "SCRIPT_FILENAME": "status",
        "REQUEST_METHOD":  "GET",
    }, "")

    log.Printf("Output: %v\n", string(resOut))
    log.Printf("Sock Err: %v\n", string(resErr))
    log.Printf("Err: %v\n", err)
}

And run it:

go run main.go

Maybe we can find something about it. I will submit a PRs to make the plugin return error information so Telegraf can log the in its log.

[sofixa]

I get the following when running your script

root@aws:/home/ubuntu# go run main.go
2016/01/16 18:25:53 Output: Status: 404 Not Found
X-Powered-By: PHP/5.5.9-1ubuntu4.14
Content-type: text/html

File not found.

2016/01/16 18:25:53 Sock Err: Primary script unknown

2016/01/16 18:25:53 Err: <nil>

As for the http method, i am not worried about unauthorized access, that's easy; my problem is that my PHP code immediately redirects the request(since i run a WordPress blog with internalization, which intercepts all requests and if they are not prefixed with a language name, sends them to the default one, and then demands the info) and returns a 404 every time, and even if i somehow manage to insert an exception for it to ignore /php-fpm-status, i'd have to do it each time i upgrade the plugins.. So not very practical...
That's why the socket would be perfect.
Sorry for the delay in my reply, i was really occupied at work :/

[sofixa]

When i put the actual path to my PHP-FPM status page i get it properly with your script:

root@aws:/home/ubuntu# go run main.go
2016/01/16 18:35:59 Output: X-Powered-By: PHP/5.5.9-1ubuntu4.14
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache, no-store, must-revalidate, max-age=0
Content-Type: text/plain

pool:                 www
process manager:      dynamic
start time:           11/Jan/2016:20:15:51 +0000
start since:          426008
accepted conn:        14369
listen queue:         0
max listen queue:     0
listen queue len:     0
idle processes:       2
active processes:     1
total processes:      3
max active processes: 3
max children reached: 0
slow requests:        0

2016/01/16 18:35:59 Sock Err:
2016/01/16 18:35:59 Err: <nil>

Which is odd, because i can't even wget into it:

root@aws:/home/ubuntu# wget 127.0.0.1:8080/fpmss
--2016-01-16 18:40:11--  http://127.0.0.1:8080/fpmss
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://127.0.0.1:8080/en/fpmss [following]
--2016-01-16 18:40:11--  http://127.0.0.1:8080/en/fpmss
Reusing existing connection to 127.0.0.1:8080.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://127.0.0.1/en/fpmss [following]
--2016-01-16 18:40:11--  http://127.0.0.1/en/fpmss
Connecting to 127.0.0.1:80... failed: Connection refused.

[v9n]

@sofixa You cannot wget it because it use fastcgi protocol, not http protocol.

So now I know what is the problem. Problem is that you don't use default path /status. That's why we get this: 2016/01/16 18:25:53 Sock Err: Primary script unknown

So I'm going to create a PR (#538) to allow you supply the path to socket like this:

urls = ["/var/run/php5-fpm.sock:fpmss"]

Are you ok with this solution?

[sofixa]

Yeap, sounds perfect.

[sofixa]

And i probably should have mentioned i am not using /status, i am using it for nginx's status page, sorry...

[v9n]

@sofixa yeah, this is because the plugin doesn't report error properly, it just fail silently. I fixed all kind of this issue in #538 .

Also, you can use only:

urls = ["/var/run/php5-fpm.sock:fpmss"]

You don't have to specify the host field.

It may take awhile for #538 to be merge and release. So you may want to compile it yourself. It's rebase on current master.

[sofixa]

Okay, thanks a lot, mate.
Cheers!