Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor diags and write into internal database #2081

Merged
merged 5 commits into from
Mar 27, 2015
Merged

Conversation

otoolep
Copy link
Contributor

@otoolep otoolep commented Mar 26, 2015

No description provided.

@otoolep
Copy link
Contributor Author

otoolep commented Mar 26, 2015

This is a significant refactor that splits out the common diags code into its own file, allowing it to be used by other types of nodes (brokers). The actual code is mostly the same, but diags types (as well as the server) can now convert diags data to InfluxQL rows. As rows, the latest diags data can be returned to the user as a query set, as well as easily written back into the database for monitoring.

This allows historical monitoring of all diagnostics. Of particular interest will be GC performance, memory usage, number of shards and shard groups on the cluster, index points, amongst other important data

@otoolep otoolep force-pushed the new_diags_file branch 6 times, most recently from 763fa56 to fa990e2 Compare March 26, 2015 06:59
@otoolep
Copy link
Contributor Author

otoolep commented Mar 26, 2015

For example, here is a memory query from the internal database:

$ curl -G http://localhost:8086/query?pretty=true --data-urlencode "db=internal" --data-urlencode "q=SELECT * FROM server_memory LIMIT 1"
{
    "results": [
        {
            "series": [
                {
                    "name": "server_memory",
                    "columns": [
                        "time",
                        "alloc",
                        "frees",
                        "heapAlloc",
                        "heapIdle",
                        "heapInUse",
                        "heapObjects",
                        "heapReleased",
                        "heapSys",
                        "lookups",
                        "mallocs",
                        "numGG",
                        "pauseTotalNs",
                        "sys",
                        "totalAlloc"
                    ],
                    "values": [
                        [
                            "2015-03-26T08:08:54.214372053Z",
                            431184,
                            2878,
                            431184,
                            925696,
                            712704,
                            3161,
                            0,
                            1.6384e+06,
                            61,
                            6039,
                            7,
                            2.115413e+06,
                            3.999992e+06,
                            743328
                        ]
                    ]
                }
            ]
        }
    ]
}

@otoolep otoolep force-pushed the new_diags_file branch 2 times, most recently from e375075 to c156eaf Compare March 26, 2015 08:18
This will make it easier to use these common diagnostics function in
other components, like the Broker, as well as minimizing growth of the
server type, in terms of SLOC.

This new diags code can be converted to InfluxQL rows, for easy display
and re-writes back to the database for self-monitoring.
This allows historical monitoring of all diagnostics. Of particular
interest will be GC performance, memory usage, number of shards and
shard groups on the cluster, index points, amongst other important data.
Also, some fixes to casts. This may need further work.
@toddboom
Copy link
Contributor

👍

so awesome.

toddboom added a commit that referenced this pull request Mar 27, 2015
Refactor diags and write into internal database
@toddboom toddboom merged commit b4fb4b0 into master Mar 27, 2015
@toddboom toddboom deleted the new_diags_file branch March 27, 2015 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants