Skip to content

[doc] List the httpjson and filelog handlers as perflog handlers only #2878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 25, 2023
Merged

[doc] List the httpjson and filelog handlers as perflog handlers only #2878

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7dd9738
Remove httpjson from standard handlers
ekouts May 8, 2023
e822f33
Remove filelog from standard handlers
ekouts May 16, 2023
e4323ad
Merge branch 'master' of https://github.com/reframe-hpc/reframe into …
ekouts May 16, 2023
b4b859a
Remove Graylog from normal handlers
vkarak May 24, 2023
0755b73
Merge branch 'master' into httpjson_fix
vkarak May 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 6 additions & 28 deletions docs/config_reference.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Even if a configuration object contains a list of other objects, this is not ref
For example, by ``systems.partitions.name`` we designate the ``name`` property of any partition object inside the ``partitions`` property of any system object inside the top level ``systems`` object.
If we were to use indices, that would be rewritten as ``systems[i].partitions[j].name`` where ``i`` indexes the systems and ``j`` indexes the partitions of the i-th system.
For cases, where the objects in a list are not homogeneous, e.g., the logging handlers, we surround the object type with ``..``.
For example, the ``logging.handlers..filelog..name`` syntax designates the ``name`` attribute of the ``filelog`` logging handler.
For example, the ``logging.handlers_perflog..filelog..name`` syntax designates the ``name`` attribute of the ``filelog`` logging handler.

.. |schemas/config.json| replace:: ``reframe/schemas/config.json``
.. _schemas/config.json: https://github.com/reframe-hpc/reframe/blob/master/reframe/schemas/config.json
Expand Down Expand Up @@ -1138,12 +1138,10 @@ The additional properties for the ``file`` handler are the following:
The ``filelog`` log handler
---------------------------

This handler is meant primarily for performance logging and logs the performance of a test in one or more files.
This handler is meant for performance logging only and logs the performance of a test in one or more files.
The additional properties for the ``filelog`` handler are the following:


.. py:attribute:: logging.handlers..filelog..basedir

.. py:attribute:: logging.handlers_perflog..filelog..basedir

:required: No
Expand All @@ -1152,13 +1150,11 @@ The additional properties for the ``filelog`` handler are the following:
The base directory of performance data log files.


.. py:attribute:: logging.handlers..filelog..prefix

.. py:attribute:: logging.handlers_perflog..filelog..prefix

:required: Yes

This is a directory prefix (usually dynamic), appended to the :attr:`~config.logging.handlers..filelog..basedir`, where the performance logs of a test will be stored.
This is a directory prefix (usually dynamic), appended to the :attr:`~config.logging.handlers_perflog..filelog..basedir`, where the performance logs of a test will be stored.
This attribute accepts any of the check-specific `formatting placeholders <#config.logging.handlers_perflog.format>`__.
This allows to create dynamic paths based on the current system, partition and/or programming environment a test executes with.
For example, a value of ``%(check_system)s/%(check_partition)s`` would generate the following structure of performance log files:
Expand All @@ -1177,8 +1173,6 @@ The additional properties for the ``filelog`` handler are the following:
...


.. py:attribute:: logging.handlers..filelog..append

.. py:attribute:: logging.handlers_perflog..filelog..append

:required: No
Expand All @@ -1197,20 +1191,16 @@ The additional properties for the ``filelog`` handler are the following:
The ``graylog`` log handler
---------------------------

This handler sends log records to a `Graylog <https://www.graylog.org/>`__ server.
This handler is meant for performance logging only and sends log records to a `Graylog <https://www.graylog.org/>`__ server.
The additional properties for the ``graylog`` handler are the following:

.. py:attribute:: logging.handlers..graylog..address

.. py:attribute:: logging.handlers_perflog..graylog..address

:required: Yes

The address of the Graylog server defined as ``host:port``.


.. py:attribute:: logging.handlers..graylog..extras

.. py:attribute:: logging.handlers_perflog..graylog..extras

:required: No
Expand Down Expand Up @@ -1317,17 +1307,13 @@ The ``httpjson`` log handler
This handler sends log records in JSON format to a server using HTTP POST requests.
The additional properties for the ``httpjson`` handler are the following:

.. py:attribute:: logging.handlers..httpjson..url

.. py:attribute:: logging.handlers_perflog..httpjson..url

:required: Yes

The URL to be used in the HTTP(S) request server.


.. py:attribute:: logging.handlers..httpjson..extra_headers

.. py:attribute:: logging.handlers_perflog..httpjson..extra_headers

:required: No
Expand All @@ -1339,8 +1325,6 @@ The additional properties for the ``httpjson`` handler are the following:
.. versionadded:: 4.2


.. py:attribute:: logging.handlers..httpjson..extras

.. py:attribute:: logging.handlers_perflog..httpjson..extras

:required: No
Expand All @@ -1349,8 +1333,6 @@ The additional properties for the ``httpjson`` handler are the following:
A set of optional key/value pairs to be passed with each log record to the server.
These may depend on the server configuration.

.. py:attribute:: logging.handlers..httpjson..ignore_keys

.. py:attribute:: logging.handlers_perflog..httpjson..ignore_keys

:required: No
Expand Down Expand Up @@ -1380,8 +1362,6 @@ An example configuration of this handler for performance logging is shown here:

This handler transmits the whole log record, meaning that all the information will be available and indexable at the remote end.

.. py:attribute:: logging.handlers..httpjson..debug

.. py:attribute:: logging.handlers_perflog..httpjson..debug

:required: No
Expand All @@ -1393,8 +1373,6 @@ This handler transmits the whole log record, meaning that all the information wi
.. versionadded:: 4.1


.. py:attribute:: logging.handlers..httpjson..json_formatter

.. py:attribute:: logging.handlers_perflog..httpjson..json_formatter

A callable for converting the log record into JSON.
Expand All @@ -1406,8 +1384,8 @@ This handler transmits the whole log record, meaning that all the information wi
:arg record: The prepared log record.
The log record is a simple Python object with all the attributes listed in :attr:`~config.logging.handlers.format`, as well as all the default Python `log record <https://docs.python.org/3.8/library/logging.html#logrecord-attributes>`__ attributes.
In addition to those, there is also the special :attr:`__rfm_check__` attribute that contains a reference to the actual test for which the performance is being logged.
:arg extras: Any extra attributes specified in :attr:`~config.logging.handlers..httpjson..extras`.
:arg ignore_keys: The set of keys specified in :attr:`~config.logging.handlers..httpjson..ignore_keys`.
:arg extras: Any extra attributes specified in :attr:`~config.logging.handlers_perflog..httpjson..extras`.
:arg ignore_keys: The set of keys specified in :attr:`~config.logging.handlers_perflog..httpjson..ignore_keys`.
ReFrame always adds the default Python log record attributes in this set.
:returns: A string representation of the JSON record to be sent to the server or :obj:`None` if the record should not be sent to the server.

Expand Down
6 changes: 3 additions & 3 deletions docs/manpage.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1321,7 +1321,7 @@ Whenever an environment variable is associated with a configuration option, its

================================== ==================
Associated command line option N/A
Associated configuration parameter :attr:`~config.logging.handlers..graylog..address`
Associated configuration parameter :attr:`~config.logging.handlers_perflog..graylog..address`
================================== ==================


Expand All @@ -1338,7 +1338,7 @@ Whenever an environment variable is associated with a configuration option, its

================================== ==================
Associated command line option N/A
Associated configuration parameter :attr:`~config.logging.handlers..httpjson..url`
Associated configuration parameter :attr:`~config.logging.handlers_perflog..httpjson..url`
================================== ==================


Expand Down Expand Up @@ -1445,7 +1445,7 @@ Whenever an environment variable is associated with a configuration option, its

================================== ==================
Associated command line option :option:`--perflogdir`
Associated configuration parameter :attr:`~config.logging.handlers..filelog..basedir`
Associated configuration parameter :attr:`~config.logging.handlers_perflog..filelog..basedir`
================================== ==================


Expand Down
27 changes: 12 additions & 15 deletions reframe/schemas/config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -412,11 +412,9 @@
"items": {
"anyOf": [
{"$ref": "#/defs/file_handler"},
{"$ref": "#/defs/filelog_handler"},
{"$ref": "#/defs/graylog_handler"},
{"$ref": "#/defs/stream_handler"},
{"$ref": "#/defs/syslog_handler"},
{"$ref": "#/defs/httpjson_handler"}
{"$ref": "#/defs/syslog_handler"}
]
}
},
Expand All @@ -425,11 +423,9 @@
"items": {
"anyOf": [
{"$ref": "#/defs/file_handler"},
{"$ref": "#/defs/filelog_handler"},
{"$ref": "#/defs/graylog_handler"},
{"$ref": "#/defs/stream_handler"},
{"$ref": "#/defs/syslog_handler"},
{"$ref": "#/defs/httpjson_handler"}
{"$ref": "#/defs/syslog_handler"}
]
},
"minItems": 1
Expand All @@ -442,7 +438,8 @@
{"$ref": "#/defs/filelog_handler"},
{"$ref": "#/defs/graylog_handler"},
{"$ref": "#/defs/stream_handler"},
{"$ref": "#/defs/syslog_handler"}
{"$ref": "#/defs/syslog_handler"},
{"$ref": "#/defs/httpjson_handler"}
]
}
},
Expand Down Expand Up @@ -566,17 +563,17 @@
"logging/handlers*/file_append": false,
"logging/handlers*/file_name": "",
"logging/handlers*/file_timestamp": false,
"logging/handlers*/filelog_append": true,
"logging/handlers*/filelog_basedir": "./perflogs",
"logging/handlers*/graylog_extras": {},
"logging/handlers*/httpjson_extras": {},
"logging/handlers*/httpjson_ignore_keys": [],
"logging/handlers*/httpjson_json_formatter": null,
"logging/handlers*/httpjson_extra_headers": {},
"logging/handlers*/httpjson_debug": false,
"logging/handlers*/stream_name": "stdout",
"logging/handlers*/syslog_socktype": "udp",
"logging/handlers*/syslog_facility": "user",
"logging/handlers_perflog/filelog_append": true,
"logging/handlers_perflog/filelog_basedir": "./perflogs",
"logging/handlers_perflog/graylog_extras": {},
"logging/handlers_perflog/httpjson_extras": {},
"logging/handlers_perflog/httpjson_ignore_keys": [],
"logging/handlers_perflog/httpjson_json_formatter": null,
"logging/handlers_perflog/httpjson_extra_headers": {},
"logging/handlers_perflog/httpjson_debug": false,
"modes/options": [],
"modes/target_systems": ["*"],
"systems/descr": "",
Expand Down