Release 37.0.0-deepbi1

Improvements

Configurable regex engine for ingestion service

The regex input format supports configurable regex engines using the runtime property:

druid.regex.engine=JAVA

Supported values:

Value	Description
JAVA	Uses Java's built-in java.util.regex.Pattern engine.
RE2J	Uses Google's RE2/J regex engine with linear-time matching guarantees.

Default value:

druid.regex.engine=JAVA

RE2/J engine

Setting:

druid.regex.engine=RE2J

enables the RE2/J regex engine for ingestion task regex input formats.

RE2/J mitigates CWE-1333 by enforcing linear-time regular expression evaluation, preventing ReDoS exploits caused by inefficient regex complexity.

Compatibility differences

RE2/J does not support all Java regex features.

Unsupported or partially supported features include:

• backreferences
• lookbehind assertions
• some advanced backtracking behavior

Patterns using unsupported constructs will fail during regex compilation.

Flexible mapping of OIDC/JWT roles to Druid roles

Configuration

New property introduced in runtime.properties:

# Dot-separated path to the claim containing user roles
druid.auth.pac4j.oidc.roleClaimPath=resource_access.client.roles

When set, all values found at roleClaimPath are mapped 1:1 to Druid roles.

• Path format: a.b.c for nested JSON fields
• Claim may contain a single value or be multivalued
• If not set, behavior remains unchanged (maps sub to a single Druid role)

Example JWT payload

{
  "iss": "https://idp.example.com/",
  "sub": "user123",
  "...": "...",
  "resource_access": {
    "client": {
      "roles": ["role1", "role2"]
    }
  },
  "...": "..."
}

With the configuration:

druid.auth.pac4j.oidc.roleClaimPath=resource_access.client.roles

The authenticated user will be assigned Druid roles role1 and role2, if those roles exist in the Druid security configuration.
By default, the pac4j OIDC authentication only maps the sub claim to a single Druid role, which is extremely inconvenient.
This PR adds support for extracting roles from any nested claim path in the JWT.

Fixes

Fix AWS S3 MRAP Deep Storage support

Include software.amazon.awssdk:http-auth-aws-crt dependency to resolve SigV4A signing issue and fix ARN identifier validation.
This allows to use both regional S3 Access Point ARN and S3 MRAP ARN in druid.storage.bucket.

Fix a race condition in off-heap cached lookups

Lookup usage is now safer during concurrent query execution and lookup refresh.

Previously, a query could keep using a lookup cache entry while a lookup refresh had already replaced and deleted that entry from mmap-backed storage. This could make in-flight lookup queries reference disposed off-heap data.

The fix adds lookup cache retirement: old cache entries are kept alive while queries still reference them and are deleted only after those references are released.

Retained lookup extractors are used across lookup query paths, including registered lookup extraction functions, lookup dimension specs, lookup joins, and lookup segment iteration.

Configuration

Property	Description	Default
druid.lookup.namespace.maxRetiredCacheEntries	The maximum number of retired cache versions that may be kept while still referenced by running queries. If the limit is reached, future cache refreshes may be skipped until retired entries are released or expire.	1
druid.lookup.namespace.retiredCacheEntryTimeoutMillis	The amount of time after retirement before a retained cache version may be disposed, even if its retained references have not been closed yet. This prevents abandoned retained references from blocking future cache refreshes indefinitely.	900,000

Fix ClassCastException in MSQ when HLLSketch is used

Address an issue where running Multi-Stage Queries on real-time segments that include columns of HLLSketch type resulted in a ClassCastException (apache#16607)

Upgraded netty3 to 3.10.6-patch2

Include patched netty-3.10.6.Final release fixing the following vulnerabilities (http request smuggling, caused by obfuscating TE header):

1. CVE-2019-16869
2. CVE-2019-20445
3. CVE-2019-20444
4. CVE-2021-43797
5. CVE-2024-29025

Derby Update

Updated Apache Derby to version 10.14.3.0, which includes a fix for an LDAP injection vulnerability.