Collecting ideas for stats server

[henningpeters]

I would like to implement those features that are covered by real-work use cases and not otherwise covered. I.e., I would explicitly exclude all statistics that S3 collects anyway via its logging feature.

My use case are:

• check what and whether it is running (stable)?
• is it fast (enough)?
• how can I tune the configuration?
• can I find bugs?

Here is an overview about all things that came to my mind that could get logged (not everything has a clear use case, though).

1. operational information

• connected endpoint
• configuration options
• versioning
• security settings
• compile flags
• version info
• uptime

2. statistics on each S3 request type

• S3 ListBucket
• S3 GetObject
  etc.

by number of requests, sent bytes, received bytes, number of internal errors/error codes, number of retries/failures

3. aggregate statistics on S3 requests

• total number of requests
• total sent bytes
• total received bytes
• average latency (incl. cache)

4. cache statistics

• size (bytes)
• number of entries
• number of hits
• number of misses
• hit/miss ratio

5. FUSE statistics

• number of requests by type (mkdir, forget, listxattr, etc.)

6. real-time information

• info on last X FUSE requests
• info on last X S3 requests
• info on X most frequently accessed cache items