[pull] master from sqlmapproject:master

.gitignore

@@ -17,3 +17,4 @@ thirdparty/.DS_Store
 CLAUDE.md
 .coverage
 .codegraph/
+.claude/

doc/CHANGELOG.md

@@ -1,3 +1,93 @@
+# Version 2.0 (WIP)
+
+* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.10...master)
+
+## Injection techniques
+
+* Added the switch `--nosql`. It tests for NoSQL injection. It also dumps the data that it finds.
+* Added the switch `--xpath`. It tests for XPath injection.
+* Added the switch `--ldap`. It tests for LDAP injection.
+* Added the switch `--ssti`. It tests for server-side template injection. It also covers Struts2 and OGNL.
+* Added the switch `--graphql`. It tests for GraphQL injection.
+* Added the switch `--hql`. It tests for HQL and JPQL (Hibernate ORM) injection.
+* Added the switch `--xxe`. It tests for XML External Entity injection. It uses in-band, error-based, and out-of-band channels.
+* Added the switch `--jwt`. It examines JSON Web Tokens for weak keys and for injection in the claims.
+
+## Speed
+
+* Added the switch `--timeless`. It reads each blind bit from the HTTP/2 response order. It does not use a delay. sqlmap calibrates the target first, and it uses the usual time-based technique if the target is not applicable.
+* Added set-membership (Huffman) retrieval for blind dumps. It needs fewer requests for each character. Use `--no-huffman` to stop it.
+* Added keyset (seek) pagination for blind table dumps. Use `--no-keyset` to stop it.
+* Added parallel retrieval of values in blind mode. Each thread retrieves a different value.
+* Made Keep-Alive the default. Use the switch `--no-keep-alive` to stop it.
+* Added the reuse of HTTP/2 connections.
+* Added the switch `--lengths`. It compares the pages only by the content length.
+* Made the HashDB operations faster.
+* Made the tamper script `luanginxmore` much faster.
+
+## Targets and results
+
+* Added the option `--openapi`. It makes the list of targets from an OpenAPI or Swagger document. The options `--openapi-base` and `--openapi-tags` limit that list.
+* Added the switch `--mine-params`. It finds hidden GET parameters.
+* Added the switch `--proof`. It proves the exploitation of each injection point that it finds.
+* Added the option `--report-json`. It writes the results of the run to a JSON file.
+* Added the switch `--procs`. It retrieves the stored procedures and their source code.
+* Added the option `--exclude`. It gives the databases that sqlmap must not enumerate.
+* Added JSONL as a dump format.
+* Added the use of sqlmap as a library.
+* Added experimental support for gRPC-Web (text) requests.
+* Improved the crawler. It now finds the endpoints in JavaScript files.
+
+## Back-end DBMS
+
+* Added support for SAP HANA.
+* Added support for Snowflake.
+* Added support for Google Cloud Spanner.
+* Added support for DuckDB and Trino as forks.
+* Added the switch `--esperanto`. It enumerates a back-end DBMS that sqlmap cannot identify.
+* Added error-based payloads for CUBRID, InterSystems Cache, Virtuoso, H2, Firebird, and Vertica.
+* Added time-based payloads for CUBRID.
+* Added out-of-band DNS channels for H2 and ClickHouse.
+* Added PostgreSQL command execution through a PL extension.
+* Added the tamper scripts `blindbinary`, `dollarquote`, `infoschema2innodb`, `oraclequote`, and `sign`.
+
+## Fewer dependencies
+
+* Added an HTTP/2 client. It uses only the standard library.
+* Added WebSocket support. It uses only the standard library.
+* Added the decoding of Brotli and Zstandard responses. It uses only the standard library.
+* Added Kerberos and Negotiate authentication. It uses only the standard library.
+* Added NTLM authentication. This removed a deprecated third-party library.
+* Rewrote the Keep-Alive handler. This removed the third-party package `keepalive`.
+* Removed the third-party packages `multipart`, `odict`, and `prettyprint`.
+* Replaced SocksiPy with PySocks.
+
+## Security
+
+* Removed all use of `pickle`. sqlmap now uses JSON for the session data and for the other serialized data.
+* The REST API now requires authentication credentials.
+* Put the `eval` behavior behind the environment variable `SQLMAP_UNSAFE_EVAL`.
+* Put the option `--alert` behind the environment variable `SQLMAP_UNSAFE_ALERT`.
+* Hardened the Brotli and the Zstandard decoders against hostile input.
+
+## Correctness
+
+* Made the boolean inference more reliable when the network has much jitter.
+* Added automatic recovery when the page charset and the data charset do not agree.
+* sqlmap now finds binary fields automatically in blind mode.
+* Added support for the response code 429 (rate limit).
+* Corrected the retrieval of UTF8MB4 characters from MySQL.
+* Corrected the Set-Cookie behavior in redirections. Added support for domain cookies.
+* sqlmap now keeps the value of an injected Host header.
+* Removed the time outliers from the time statistics.
+* Improved the detection of the SQL dialect of the target.
+
+## Quality
+
+* Added a unit test suite. It has more than 90 modules, and it runs on Python 2 and Python 3.
+* Added pyflakes and more self-test stages to the CI/CD pipeline.
+* Added `doc/ARCHITECTURE.md`.
+
 # Version 1.10 (2026-01-01)
 
 * [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.9...1.10)

extra/dbwire/__init__.py

@@ -48,3 +48,44 @@ class InternalError(DatabaseError):
 
 class NotSupportedError(DatabaseError):
     pass
+
+def http_origin(host, port):
+    """
+    'http://host:port', with a literal IPv6 address bracketed as RFC 3986 requires. Without the brackets
+    the colons in the address are parsed as the port separator and the URL is simply wrong.
+    """
+
+    host = host or "localhost"
+    if ":" in host and not host.startswith("["):
+        host = "[%s]" % host
+    return "http://%s:%d" % (host, int(port))
+
+
+def connection_lost(ex):
+    """
+    Turn a raw socket/OS failure into the DB-API hierarchy above.
+
+    Callers of a PEP 249 driver only ever catch Error and its subclasses, so a bare socket.error escaping
+    from a send/recv leaves them with an unhandled traceback instead of a handled connection failure.
+    """
+
+    return OperationalError("connection lost (%s)" % ex)
+
+def keepalive(sock):
+    """
+    Ask the kernel to probe an idle connection, so a peer that dies without a FIN is eventually detected.
+
+    Deliberately NOT a read timeout: a legitimate query can take minutes on a big table, and a fixed
+    deadline would kill it. Keepalive distinguishes a dead peer from a slow one, which is the actual
+    failure being guarded against. Best-effort - the options are not portable everywhere.
+    """
+
+    import socket as _socket
+
+    try:
+        sock.setsockopt(_socket.SOL_SOCKET, _socket.SO_KEEPALIVE, 1)
+        for name, value in (("TCP_KEEPIDLE", 60), ("TCP_KEEPINTVL", 10), ("TCP_KEEPCNT", 5)):
+            if hasattr(_socket, name):
+                sock.setsockopt(_socket.IPPROTO_TCP, getattr(_socket, name), value)
+    except Exception:
+        pass

extra/dbwire/clickhouse.py

@@ -19,10 +19,13 @@
 try:
     from urllib.request import Request, urlopen        # Python 3
     from urllib.error import HTTPError, URLError
+    from urllib.parse import quote
 except ImportError:
     from urllib2 import Request, urlopen, HTTPError, URLError  # Python 2
+    from urllib import quote
 
 from extra.dbwire import OperationalError
+from extra.dbwire import http_origin
 from extra.dbwire import ProgrammingError
 
 # TabSeparated backslash escapes -> the literal byte they denote
@@ -87,7 +90,8 @@ def close(self):
 
 class Connection(object):
     def __init__(self, host, port, user, password, database, timeout):
-        self._url = "http://%s:%d/?database=%s&default_format=TabSeparatedWithNames" % (host, port, database or "default")
+        # quote the database: a name with a reserved character would otherwise inject into the query string
+        self._url = "%s/?database=%s&default_format=TabSeparatedWithNames" % (http_origin(host, port), quote(database or "default", safe=""))
         self._headers = {}
         if user or password:
             token = base64.b64encode(("%s:%s" % (user or "", password or "")).encode("utf-8")).decode("ascii")

extra/dbwire/cubrid.py

@@ -23,6 +23,8 @@
 from extra.dbwire import IntegrityError
 from extra.dbwire import NotSupportedError
 from extra.dbwire import OperationalError
+from extra.dbwire import connection_lost
+from extra.dbwire import keepalive
 from extra.dbwire import ProgrammingError
 
 _MAGIC = b"CUBRK"
@@ -257,7 +259,10 @@ def _safe_close(self):
     def _recvn(self, n):
         buf = b""
         while len(buf) < n:
-            chunk = self._sock.recv(n - len(buf))
+            try:
+                chunk = self._sock.recv(n - len(buf))
+            except (socket.error, OSError) as ex:
+                raise connection_lost(ex)
             if not chunk:
                 raise InterfaceError("connection closed by server")
             buf += chunk
@@ -267,6 +272,7 @@ def _open(self):
         # broker handshake (may redirect to a dedicated CAS worker port), then cleartext OPEN_DATABASE login
         try:
             sock = socket.create_connection((self._host, self._port), timeout=self._timeout)
+            keepalive(sock)
             sock.settimeout(None)
             sock.sendall(_MAGIC + struct.pack(">BB", _CLIENT_JDBC, _CAS_VERSION) + b"\x00\x00\x00")
             self._sock = sock
@@ -276,6 +282,7 @@ def _open(self):
             if port > 0:  # redirected to a CAS worker: reconnect there, no second handshake
                 self._safe_close()
                 sock = socket.create_connection((self._host, port), timeout=self._timeout)
+                keepalive(sock)
                 sock.settimeout(None)
                 self._sock = sock
         except (socket.error, socket.timeout) as ex:
@@ -299,7 +306,10 @@ def _fixed(value, length):
 
     def _send(self, payload):
         # frame: [payload_len(4)][cas_info(4)][payload]
-        self._sock.sendall(struct.pack(">i", len(payload)) + self._cas_info + payload)
+        try:
+            self._sock.sendall(struct.pack(">i", len(payload)) + self._cas_info + payload)
+        except (socket.error, OSError) as ex:
+            raise connection_lost(ex)
 
     def _read_response(self):
         (data_length,) = struct.unpack(">i", self._recvn(4))
@@ -344,6 +354,17 @@ def _raise(errno, message):
     def _query(self, query):
         reader = self._call(_Writer(_FC_PREPARE).arg_nts(query).arg_byte(0).arg_byte(0))
         handle = reader.int()
+        try:
+            return self._execute(handle, reader)
+        finally:
+            # release the broker-side request handle even when execute/fetch raised: a connection that
+            # survives a few failed statements would otherwise hold every one of them until it closes
+            try:
+                self._call(_Writer(_FC_CLOSE_REQ_HANDLE).arg_int(handle))
+            except Exception:
+                pass
+
+    def _execute(self, handle, reader):
         reader.int()                 # result cache lifetime
         stmt_type = reader.byte()
         reader.int()                 # bind count
@@ -374,7 +395,6 @@ def _query(self, query):
             rows += self._fetch_remaining(handle, columns, len(rows), total)
         elif result_infos:
             rowcount = result_infos[0]
-        self._call(_Writer(_FC_CLOSE_REQ_HANDLE).arg_int(handle))
         return description, rows, rowcount
 
     def _fetch_remaining(self, handle, columns, fetched, total):

extra/dbwire/firebird.py

@@ -26,6 +26,8 @@
 from extra.dbwire import InterfaceError
 from extra.dbwire import NotSupportedError
 from extra.dbwire import OperationalError
+from extra.dbwire import connection_lost
+from extra.dbwire import keepalive
 
 # operation codes
 _op_connect = 1
@@ -286,12 +288,18 @@ def set_ciphers(self, rc, wc):
         self._rc, self._wc = rc, wc
 
     def send(self, data):
-        self._sock.sendall(self._wc.translate(data) if self._wc else data)
+        try:
+            self._sock.sendall(self._wc.translate(data) if self._wc else data)
+        except (socket.error, OSError) as ex:
+            raise connection_lost(ex)
 
     def _recv_raw(self, n):
         buf = b""
         while len(buf) < n:
-            chunk = self._sock.recv(n - len(buf))
+            try:
+                chunk = self._sock.recv(n - len(buf))
+            except (socket.error, OSError) as ex:
+                raise connection_lost(ex)
             if not chunk:
                 raise InterfaceError("connection closed by server")
             buf += chunk
@@ -494,23 +502,36 @@ def _run(self, query):
         self._send(_pack_int(_op_allocate_statement) + _pack_int(self._db_handle))
         stmt = self._response()[0]
 
-        desc_items = bytes(bytearray([_isc_info_sql_stmt_type])) + _INFO_SQL_SELECT_DESCRIBE_VARS
-        self._send(_pack_int(_op_prepare_statement) + _pack_int(self._trans_handle) + _pack_int(stmt) +
-                   _pack_int(3) + _pack_bytes(qbytes) + _pack_bytes(desc_items) + _pack_int(1024))
-        buf = self._response()[2]
-        stmt_type, columns = self._parse_describe(stmt, buf)
-
-        exec_msg = (_pack_int(_op_execute) + _pack_int(stmt) + _pack_int(self._trans_handle) +
-                    _pack_bytes(b"") + _pack_int(0) + _pack_int(0) + _pack_int(0))
-        self._send(exec_msg)
-        self._response()
+        try:
+            desc_items = bytes(bytearray([_isc_info_sql_stmt_type])) + _INFO_SQL_SELECT_DESCRIBE_VARS
+            self._send(_pack_int(_op_prepare_statement) + _pack_int(self._trans_handle) + _pack_int(stmt) +
+                       _pack_int(3) + _pack_bytes(qbytes) + _pack_bytes(desc_items) + _pack_int(1024))
+            buf = self._response()[2]
+            stmt_type, columns = self._parse_describe(stmt, buf)
+
+            exec_msg = (_pack_int(_op_execute) + _pack_int(stmt) + _pack_int(self._trans_handle) +
+                        _pack_bytes(b"") + _pack_int(0) + _pack_int(0) + _pack_int(0))
+            self._send(exec_msg)
+            self._response()
 
-        description, rows = None, []
-        if stmt_type == _isc_info_sql_stmt_select and columns:
-            description = [(c.name, c.sqltype, None, None, None, None, None) for c in columns]
-            rows = self._fetch(stmt, columns)
-        self._send(_pack_int(_op_free_statement) + _pack_int(stmt) + _pack_int(_DSQL_drop))
-        self._response()
+            description, rows = None, []
+            if stmt_type == _isc_info_sql_stmt_select and columns:
+                description = [(c.name, c.sqltype, None, None, None, None, None) for c in columns]
+                rows = self._fetch(stmt, columns)
+        finally:
+            # release the server-side handle even when prepare/execute/fetch raised: a connection that
+            # survives a few failed statements would otherwise hold every one of them until it detaches
+            try:
+                self._send(_pack_int(_op_free_statement) + _pack_int(stmt) + _pack_int(_DSQL_drop))
+                self._response()
+            except Exception:
+                pass
+
+        # dbwire statements are autonomous (see README), but Firebird has no auto-commit mode: everything
+        # runs inside the one transaction opened at attach. Without this, DML is lost when the caller
+        # closes the connection without an explicit commit(). commit-retaining makes the work durable and
+        # keeps the transaction handle valid, so the connection stays usable for the next statement.
+        self.commit()
         return description, rows
 
     def _parse_describe(self, stmt, buf):
@@ -737,6 +758,7 @@ def connect(host=None, port=3050, user=None, password=None, database=None, conne
 
     try:
         sock = socket.create_connection((host or "localhost", int(port or 3050)), timeout=connect_timeout)
+        keepalive(sock)
         sock.settimeout(None)
     except (socket.error, socket.timeout) as ex:
         raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex))

extra/dbwire/monetdb.py

@@ -22,14 +22,19 @@
 from extra.dbwire import InterfaceError
 from extra.dbwire import NotSupportedError
 from extra.dbwire import OperationalError
+from extra.dbwire import connection_lost
+from extra.dbwire import keepalive
 from extra.dbwire import ProgrammingError
 
 _MAX_BLOCK = 0xffff >> 1
 
 def _recvn(sock, n):
     buf = b""
     while len(buf) < n:
-        chunk = sock.recv(n - len(buf))
+        try:
+            chunk = sock.recv(n - len(buf))
+        except (socket.error, OSError) as ex:
+            raise connection_lost(ex)
         if not chunk:
             raise InterfaceError("connection closed by server")
         buf += chunk
@@ -52,7 +57,10 @@ def _putblock(sock, text):
         chunk = data[off:off + _MAX_BLOCK]
         off += _MAX_BLOCK
         last = off >= len(data)
-        sock.sendall(struct.pack("<H", (len(chunk) << 1) | (1 if last else 0)) + chunk)
+        try:
+            sock.sendall(struct.pack("<H", (len(chunk) << 1) | (1 if last else 0)) + chunk)
+        except (socket.error, OSError) as ex:
+            raise connection_lost(ex)
         if last:
             break
 
@@ -186,6 +194,7 @@ def connect(host=None, port=50000, user=None, password=None, database=None, conn
     host, port = host or "localhost", int(port or 50000)
     try:
         sock = socket.create_connection((host, port), timeout=connect_timeout)
+        keepalive(sock)
         sock.settimeout(None)
     except (socket.error, socket.timeout) as ex:
         raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex))
@@ -202,6 +211,7 @@ def connect(host=None, port=50000, user=None, password=None, database=None, conn
                     sock.close()
                     host, port, database = m.group(1), int(m.group(2)), m.group(3) or database
                     sock = socket.create_connection((host, port), timeout=connect_timeout)
+                    keepalive(sock)
                     sock.settimeout(None)
                 continue  # merovingian proxy redirect: keep reading the next challenge on this socket
             if block[0] == "!":