Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Conditional access logging #594

Closed
joachimBurket opened this issue Nov 11, 2021 · 3 comments · Fixed by #1044
Closed

Conditional access logging #594

joachimBurket opened this issue Nov 11, 2021 · 3 comments · Fixed by #1044
Assignees
@hongzhidao
Labels
z-enhancement ⬆️ Product Enhancement z-roadmap
Milestone
1.32

Comments

@joachimBurket
Copy link

Hi there,

It would be great to be able to filter access logs by its content. I'm running my apps in a Kubernetes cluster, and have some healthchecks configured on '/readyz' and '/livez'.
These checks are generating a lot of access logs, which make hard to see the important ones.
@VBart VBart added the z-enhancement ⬆️ Product Enhancement label Nov 13, 2021
@VBart
Copy link
Contributor

VBart commented Nov 24, 2021

Any ideas on how such configuration may look like? Or maybe we just need to allow configuring access log in the router objects somehow?

@akalineskou
Copy link

in nginx we do it like this

# ignore health checks from logs
    map $request_uri $loggable {
        /health 0;

        default 1;
    }

    access_log /proc/self/fd/1 main if=$loggable;

This feature is a must since it does flood the access logs

@hongzhidao hongzhidao self-assigned this Oct 12, 2023
@lcrilly lcrilly mentioned this issue Oct 20, 2023
Closed
@lcrilly lcrilly changed the title Feature: Access Logs filtering Conditional access logging Oct 20, 2023
@lcrilly
Copy link
Contributor

lcrilly commented Oct 20, 2023 •
edited
Loading

Current thoughts for this are to provide an if object within the access_log configuration that provides very simple matching with variables or more complex matching with JavaScript expressions.

  1. Only log requests that sent a session cookie (a non-empty string is considered true)
{
  "access_log": {
    "if": "$cookie_session",
    "path": "…"
}
  1. Do not log health check requests
{
  "access_log": {
    "if": "`${uri == '/health' ? false : true}`",
    "path": "…"
}

@tippexs tippexs added this to the 1.32 milestone Dec 6, 2023
@andrey-zelenkov andrey-zelenkov mentioned this issue Jan 10, 2024
Merged
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 15, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
47b1bce 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

For example:

    {
      	"access_log": {
	    "if": "`${new Date().getHours() < 22}`",
	    "path": "/path/access.log"
      	}
    }

Or

    {
      	"access_log": {
	    "if": "!`${new Date().getHours() >= 22}`",
	    "path": "/path/access.log"
      	}
    }

If the current hour is less than 22, the logs will be recorded.

Closes: <nginx#594>
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 17, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
1c83c41 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

For example:

    {
      	"access_log": {
	    "if": "`${new Date().getHours() < 22}`",
	    "path": "/path/access.log"
      	}
    }

Or

    {
      	"access_log": {
	    "if": "!`${new Date().getHours() >= 22}`",
	    "path": "/path/access.log"
      	}
    }

If the current hour is less than 22, the logs will be recorded.

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 18, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
b89292a 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1:

    {
        "access_log": {
	    "if": "$cookie_session",
       	    "path": "..."
        }
    }

Example 2:

    {
      	"access_log": {
	    "if": "`${new Date().getHours() < 22}`",
	    "path": "..."
      	}
    }

Example 3:

    {
      	"access_log": {
	    "if": "!`${new Date().getHours() >= 22}`",
	    "path": "..."
      	}
    }

If the current hour is less than 22, the logs will be recorded.

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 18, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
a181440 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
	    "if": "$cookie_session",
       	    "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
  	"access_log": {
    	    "if": "`${uri == '/health' ? false : true}`",
	    "path": "..."
    }

Example 3: Only log requests when the time is before 22:00.

    {
      	"access_log": {
	    "if": "`${new Date().getHours() < 22}`",
	    "path": "..."
      	}
    }

or

    {
      	"access_log": {
	    "if": "!`${new Date().getHours() >= 22}`",
	    "path": "..."
      	}
    }

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 23, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
60d8bf5 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
        "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
    "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
        "path": "..."
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
        "if": "`${new Date().getHours() < 22}`",
        "path": "..."
        }
    }

or

    {
        "access_log": {
        "if": "!`${new Date().getHours() >= 22}`",
        "path": "..."
        }
    }

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 23, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
4356157 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
        "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
    "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
        "path": "..."
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
        "if": "`${new Date().getHours() < 22}`",
        "path": "..."
        }
    }

or

    {
        "access_log": {
        "if": "!`${new Date().getHours() >= 22}`",
        "path": "..."
        }
    }

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 23, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
f109638 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
        "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
    "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
        "path": "..."
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
        "if": "`${new Date().getHours() < 22}`",
        "path": "..."
        }
    }

or

    {
        "access_log": {
        "if": "!`${new Date().getHours() >= 22}`",
        "path": "..."
        }
    }

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 24, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
3aa34b8 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
            "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
        "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
            "path": "..."
        }
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
            "if": "`${new Date().getHours() < 22}`",
            "path": "..."
        }
    }

or

    {
        "access_log": {
            "if": "!`${new Date().getHours() >= 22}`",
            "path": "..."
        }
    }

Closes: nginx#594
hongzhidao added a commit to hongzhidao/unit that referenced this issue Jan 29, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
a228940 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
            "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
        "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
            "path": "..."
        }
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
            "if": "`${new Date().getHours() < 22}`",
            "path": "..."
        }
    }

or

    {
        "access_log": {
            "if": "!`${new Date().getHours() >= 22}`",
            "path": "..."
        }
    }

Closes: nginx#594
andrey-zelenkov pushed a commit that referenced this issue Feb 27, 2024
@hongzhidao @andrey-zelenkov
HTTP: enhanced access log with conditional filtering.
0a1d500 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
            "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
        "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
            "path": "..."
        }
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
            "if": "`${new Date().getHours() < 22}`",
            "path": "..."
        }
    }

or

    {
        "access_log": {
            "if": "!`${new Date().getHours() >= 22}`",
            "path": "..."
        }
    }

Closes: #594
@callahad callahad moved this to 🚀 Released in NGINX Unit Planning May 20, 2024
pkillarjun pushed a commit to pkillarjun/unit that referenced this issue May 29, 2024
@hongzhidao
HTTP: enhanced access log with conditional filtering.
daf1442 
This feature allows users to specify conditions to control if access log
should be recorded. The "if" option supports a string and JavaScript code.
If its value is empty, 0, false, null, or undefined, the logs will not be
recorded. And the '!' as a prefix inverses the condition.

Example 1: Only log requests that sent a session cookie.

    {
        "access_log": {
            "if": "$cookie_session",
            "path": "..."
        }
    }

Example 2: Do not log health check requests.

    {
        "access_log": {
            "if": "`${uri == '/health' ? false : true}`",
            "path": "..."
        }
    }

Example 3: Only log requests when the time is before 22:00.

    {
        "access_log": {
            "if": "`${new Date().getHours() < 22}`",
            "path": "..."
        }
    }

or

    {
        "access_log": {
            "if": "!`${new Date().getHours() >= 22}`",
            "path": "..."
        }
    }

Closes: nginx#594
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hongzhidao hongzhidao

Labels
z-enhancement ⬆️ Product Enhancement z-roadmap
Projects
NGINX Unit Planning
Status: 🚀 Released
Milestone
1.32
Development

Successfully merging a pull request may close this issue.

HTTP: enhanced access log with conditional filtering. hongzhidao/unit
6 participants
@VBart @akalineskou @tippexs @hongzhidao @lcrilly @joachimBurket