Skip to content

Add log field header fallbacks #13018

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bneradt wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add log field header fallbacks #13018

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions doc/admin-guide/logging/formatting.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -314,8 +314,27 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'

The above would insert the User Agent string from the client request headers
into your log entry (or a blank string if no such header was present, or it did
not contain a value).
into your log entry (or ``-`` if no such header was present).

Header fields can also be chained with a fallback operator, `?:`, when you want
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline markup for the fallback operator uses single backticks (?:), which in reStructuredText is interpreted text and can render unexpectedly / trigger Sphinx warnings. Use a literal (double-backtick) inline code block (?:) for consistency with the surrounding literals (e.g., -).

Suggested change
Header fields can also be chained with a fallback operator, `?:`, when you want
Header fields can also be chained with a fallback operator, ``?:``, when you want

Copilot uses AI. Check for mistakes.
the log to use the first header that exists. For example::

Format = '%<{x-primary-id}cqh?:{x-secondary-id}cqh>'

|TS| evaluates the candidates from left to right and logs the first header that
exists. If none of the headers exist, |TS| logs ``-``. A header that exists but
has an empty value is considered present, so |TS| logs the empty value instead
of falling back. So in the example above, the value of x-primary-id of the
client request is logged if it exists, otherwise the value of x-secondary-id is
logged if it exists, otherwise ``-`` is logged if neither of the headers is
present.

Note that this fallback chain feature is only supported for header field
expressions.

Slices apply to each candidate in the fallback chain individually::

Format = '%<{x-primary-id}cqh[0:8]?:{x-secondary-id}cqh[0:16]>'

===== ====================== ==================================================
Field Source Description
Expand Down
6 changes: 4 additions & 2 deletions include/proxy/logging/LogAccess.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,7 @@ class LogAccess
int marshal_milestone_diff(TSMilestonesType ms1, TSMilestonesType ms2, char *buf);
int marshal_milestones_csv(char *buf);

bool has_http_header_field(LogField::Container container, const char *field) const;
void set_http_header_field(LogField::Container container, char *field, char *buf, int len);

// Plugin
Expand Down Expand Up @@ -405,8 +406,9 @@ class LogAccess
char *m_cache_lookup_url_canon_str = nullptr;
int m_cache_lookup_url_canon_len = 0;

void validate_unmapped_url();
void validate_unmapped_url_path();
HTTPHdr *header_for_container(LogField::Container container) const;
void validate_unmapped_url();
void validate_unmapped_url_path();

void validate_lookup_url();
};
Expand Down
49 changes: 31 additions & 18 deletions include/proxy/logging/LogField.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
#include <string>
#include <variant>
#include <tuple>
#include <vector>

#include "tscore/ink_inet.h"
#include "tscore/ink_platform.h"
Expand Down Expand Up @@ -129,6 +130,12 @@ class LogField
N_AGGREGATES,
};

struct HeaderField {
std::string name;
Container container = NO_CONTAINER;
LogSlice slice;
};

LogField(const char *name, const char *symbol, Type type, MarshalFunc marshal, VarUnmarshalFuncSliceOnly unmarshal,
SetFunc _setFunc = nullptr);

Expand All @@ -138,6 +145,7 @@ class LogField
LogField(const char *name, const char *symbol, Type type, CustomMarshalFunc custom_marshal, CustomUnmarshalFunc custom_unmarshal);

LogField(const char *field, Container container);
LogField(const char *symbol, std::vector<HeaderField> header_fields);
LogField(const LogField &rhs);
~LogField();

Expand Down Expand Up @@ -193,27 +201,32 @@ class LogField
static Container valid_container_name(char *name);
static Aggregate valid_aggregate_name(char *name);
static bool fieldlist_contains_aggregates(const char *fieldlist);
static bool isHeaderContainer(Container container);
static bool isContainerUpdateFieldSupported(Container container);

private:
char *m_name;
char *m_symbol;
Type m_type;
Container m_container;
MarshalFunc m_marshal_func; // place data into buffer
VarUnmarshalFunc m_unmarshal_func; // create a string of the data
Aggregate m_agg_op;
int64_t m_agg_cnt;
int64_t m_agg_val;
TSMilestonesType m_milestone1; ///< Used for MS and MSDMS as the first (or only) milestone.
TSMilestonesType m_milestone2; ///< Second milestone for MSDMS
bool m_time_field;
Ptr<LogFieldAliasMap> m_alias_map; // map sINT <--> string
SetFunc m_set_func;
TSMilestonesType milestone_from_m_name();
int milestones_from_m_name(TSMilestonesType *m1, TSMilestonesType *m2);
CustomMarshalFunc m_custom_marshal_func = nullptr;
CustomUnmarshalFunc m_custom_unmarshal_func = nullptr;
char *m_name;
char *m_symbol;
Type m_type;
Container m_container;
MarshalFunc m_marshal_func; // place data into buffer
VarUnmarshalFunc m_unmarshal_func; // create a string of the data
Aggregate m_agg_op;
int64_t m_agg_cnt;
int64_t m_agg_val;
TSMilestonesType m_milestone1; ///< Used for MS and MSDMS as the first (or only) milestone.
TSMilestonesType m_milestone2; ///< Second milestone for MSDMS
bool m_time_field;
Ptr<LogFieldAliasMap> m_alias_map; // map sINT <--> string
SetFunc m_set_func;
TSMilestonesType milestone_from_m_name();
int milestones_from_m_name(TSMilestonesType *m1, TSMilestonesType *m2);
CustomMarshalFunc m_custom_marshal_func = nullptr;
CustomUnmarshalFunc m_custom_unmarshal_func = nullptr;
std::vector<HeaderField> m_header_fields;
bool is_header_field_fallback() const;
int select_header_field(LogAccess *lad) const;
unsigned marshal_header_field(LogAccess *lad, const HeaderField &field, char *buf) const;

public:
LINK(LogField, link);
Expand Down
104 changes: 50 additions & 54 deletions src/proxy/logging/LogAccess.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,54 @@ LogAccess::marshal_custom_field(char *buf, LogField::CustomMarshalFunc plugin_ma
return LogAccess::padded_length(len);
}

HTTPHdr *
LogAccess::header_for_container(LogField::Container container) const
{
switch (container) {
case LogField::CQH:
case LogField::ECQH:
return m_client_request;

case LogField::PSH:
case LogField::EPSH:
return m_proxy_response;

case LogField::PQH:
case LogField::EPQH:
return m_proxy_request;

case LogField::SSH:
case LogField::ESSH:
return m_server_response;

case LogField::CSSH:
case LogField::ECSSH:
return m_cache_response;

default:
return nullptr;
}
}

bool
LogAccess::has_http_header_field(LogField::Container container, const char *field) const
{
if (HTTPHdr const *header = header_for_container(container); header != nullptr) {
if (header->field_find(std::string_view{field}) != nullptr) {
return true;
}
}

if (container == LogField::SSH && strcmp(field, "Transfer-Encoding") == 0) {
const std::string &stored_te = m_http_sm->t_state.hdr_info.server_response_transfer_encoding;
if (!stored_te.empty()) {
return true;
}
}

return false;
}

inline int
LogAccess::unmarshal_with_map(int64_t code, char *dest, int len, const Ptr<LogFieldAliasMap> &map, const char *msg)
{
Expand Down Expand Up @@ -3282,33 +3330,7 @@ LogAccess::marshal_http_header_field(LogField::Container container, char *field,
int padded_len = INK_MIN_ALIGN;
int actual_len = 0;
bool valid_field = false;
HTTPHdr *header;

switch (container) {
case LogField::CQH:
header = m_client_request;
break;

case LogField::PSH:
header = m_proxy_response;
break;

case LogField::PQH:
header = m_proxy_request;
break;

case LogField::SSH:
header = m_server_response;
break;

case LogField::CSSH:
header = m_cache_response;
break;

default:
header = nullptr;
break;
}
HTTPHdr *header = header_for_container(container);

if (header) {
MIMEField *fld = header->field_find(std::string_view{field});
Expand Down Expand Up @@ -3413,33 +3435,7 @@ LogAccess::marshal_http_header_field_escapify(LogField::Container container, cha
int padded_len = INK_MIN_ALIGN;
int actual_len = 0, new_len = 0;
bool valid_field = false;
HTTPHdr *header;

switch (container) {
case LogField::ECQH:
header = m_client_request;
break;

case LogField::EPSH:
header = m_proxy_response;
break;

case LogField::EPQH:
header = m_proxy_request;
break;

case LogField::ESSH:
header = m_server_response;
break;

case LogField::ECSSH:
header = m_cache_response;
break;

default:
header = nullptr;
break;
}
HTTPHdr *header = header_for_container(container);

if (header) {
MIMEField *fld = header->field_find(std::string_view{field});
Expand Down
Loading
Loading