[Bug] Enhanced server (Linux): SIGSEGV in memcpy when an HTTP request to a resource SetHttpHandler endpoint is aborted mid-flight

[IchigoWalker]

Summary

The Enhanced dedicated server (b94, Linux) dies with SIGSEGV (SEGV_MAPERR, memcpy reading an unmapped source) when an HTTP client aborts an in-flight request to a resource-registered SetHttpHandler endpoint. In both minidumps we captured, the length of the crashing memcpy exactly equals the byte size of the aborting client's HTTP request (448 bytes), suggesting a use-after-free of the request/connection buffer. A second fatal mode ("Assertion 'm_ref' failed | Ref invalid") appears under the same load. No players required. Removing the aggressive poller stopped the crashes entirely.

Issue type

Server

Repro rate

Often

Server build version

FXServer-early-access b94 linux (artifact uuid 019f8a75-b0f4-7dfc-8ac6-4dce4292da4a)

OS

No response

CPU

No response

GPU

No response

RAM

No response

Storage type

None

Connection type

None

ISP and bandwidth

No response

DxDiag

No response

Network graph

No response

Platform

Linux

OS version / distribution

Host: Ubuntu 24.04.4 LTS; server runs in Docker

CPU

AMD Ryzen 9 9950X 16-Core Processor 182Gi 1024

RAM

182Gi

Using txAdmin?

No

Hosting provider

Other GSP

Machine type

Dedicated

/perf endpoint output

ace_count{name="Amount of access control entries in the server permission manager."} 0
active_auth_ticket_count{name="Amount of active auth tickets."} 39
challenge_count{name="Amount of /challenge requests."} 72
challenge_rate_limited_count{name="Amount of /challenge requests that were rate limited."} 0
client_registry_clients{name="Number of clients in the client registry"} 39
client_registry_extensions_size{name="Number of elements in all client registry extensions."} 0
con_var_count{name="Number of registered convars."} 165
console_command_count{name="Number of commands in the console_command_manager."} 60
handshake_count{name="Amount of /handshake requests."} 69
handshake_failed_count{name="Amount of /handshake requests that failed the handshake process."} 10
handshake_packet_count{name="Amount of handshake packets received by the server."} 16
handshake_packet_failed_count{name="Amount of handshake packets that failed the handshake process."} 0
handshake_rate_limited_count{name="Amount of /handshake requests that failed the handshake process."} 0
heartbeat_count{name="Amount of Heartbeats sent to the master server"} 16
heartbeat_error_count{name="Amount of failed Heartbeats sent to the master server"} 0
heartbeat_success_count{name="Amount of successful Heartbeats sent to the master server"} 16
http_connection_pool_count{name="Number of connections in the HTTP connection pool."} 0
http_dynamic_endpoint_rate_limited_count{name="The total number of requests to /dynamic.json that were rate limited."} 0
http_dynamic_endpoint_request_count{name="The total number of requests to the /dynamic.json endpoint."} 51
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/challenge"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/dynamic.json"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/handshake"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/info.json"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/onesync/"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/perf"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/players.json"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/profileData.json"} 0
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/resources/"} 326269
http_handler_execution_time{name="Nanoseconds execution time ~2 seconds.",handler="/voicechannels.json"} 0
http_handlers_global_execution_time{name="Nanoseconds execution time ~2 seconds."} 0
http_info_endpoint_rate_limited_count{name="The total number of requests to /info.json that were rate limited."} 0
http_info_endpoint_request_count{name="The total number of requests to the /info.json endpoint."} 112
http_players_endpoint_rate_limited_count{name="The total number of requests to /players.json that were rate limited."} 0
http_players_endpoint_request_count{name="The total number of requests to the /players.json endpoint."} 69
http_profile_data_endpoint_rate_limited_count{name="The total number of requests to /profileData.json that were rate limited."} 0
http_profile_data_endpoint_request_count{name="The total number of requests to the /profileData.json endpoint."} 0
http_root_endpoint_rate_limited_count{name="The total number of requests to the root endpoint that were rate limited."} 0
http_root_endpoint_request_count{name="The total number of requests to the root endpoint."} 0
io_latency{name="Amount of milliseconds since last IO tick"} 2318
js_memory_usage{name="Total memory usage of Javascript in bytes."} 236742648
kvp_database_file_size_bytes{name="Size of the KVP database file in bytes."} 12288
kvp_database_iterator_count{name="Number of active iterators on the KVP database."} 0
lua_memory_usage{name="Total memory usage of Lua states in bytes."} 0
lua_pool_free_count{name="Number of free objects in the Lua object pool."} 0
main_loop_queue_count{name="Number of items in the main loop queue."} 0
net_latency{name="Amount of milliseconds since last Net tick"} 1
network_loop_queue_count{name="Number of items in the UDP server network loop queue."} 0
onesync_array_handler_count{name="Number of array handlers"} 0
onesync_blip_count{name="Number of blips"} 0
onesync_blip_scripting_count{name="Number of blips created from scripting"} 0
onesync_client_data_count{name="Amount of client data entries"} 7
onesync_endpoint_invalid_auth_count{name="The number of times the /onesync/ endpoints was accessed with invalid authentication."} 0
onesync_endpoint_rate_limited_count{name="The number of times the /onesync/ endpoints was accessed but rate limited."} 0
onesync_endpoint_request_count{name="The number of times the /onesync/ endpoints was accessed."} 0
onesync_frame_index_metric{name="Current frame index of the server"} 337290
onesync_statebag_client_data_count{name="Amount of client state bags."} 7
onesync_statebag_data_count{name="Amount of entity state bags."} 380
onesync_sync_tree_count{name="Amount of sync trees."} 380
onesync_world_grid_client_data_count{name="Amount of world grid client data entries."} 7

DDoS protection

Cosmic Global Networks (enabled by default on their grid; unclear whether it applies to this Enhanced host yet)

ulimit -n value (Linux only)

1024

Docker Compose file (Docker only)

# Trimmed to the service relevant to the crash; sibling services (MariaDB, backups,
# migrations runner, app integrations) and app-specific env vars omitted.
services:
  server:
    image: <private registry>/fivem-server:enhanced   # FXServer-early-access b94 linux + resources baked in
    pull_policy: always
    restart: unless-stopped
    stdin_open: true
    tty: true
    environment:
      RUNTIME_CONFIG: "1"          # entrypoint regenerates server.cfg from env at start
      SERVER_PORT: "30120"
      MAX_PLAYERS: "2048"
      LICENSE_KEY: ${LICENSE_KEY}
      # ...gameplay/app-specific vars omitted...
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"
    volumes:
      # FXServer writes crash dumps to /tmp inside the container
      - /docker/fivem-data/.../crash-tmp:/tmp
    deploy:
      mode: global
    ports:
      # Direct host-mode publish, no proxy in front at crash time
      - target: 30120
        published: 30120
        protocol: tcp
        mode: host
      - target: 30120
        published: 30120
        protocol: udp
        mode: host
    networks:
      - internal-bridge

networks:
  internal-bridge:
    external: true

Steps to Reproduce

1. Linux Enhanced server (b94) running a JS (node runtime) resource that registers an HTTP route via SetHttpHandler (we use @citizenfx/http-wrapper 0.2.2 + Koa; responses are written from promise continuations after the native handler call returns).
2. Poll that endpoint over the public game port at 1 Hz with a client-side timeout short enough that some requests get aborted mid-flight. In our case: a Prometheus scrape job with scrape_interval=1s and scrape_timeout=900ms — whenever the server hiccups past 900 ms, the scraper aborts the in-flight request (visible as 499 / request_time=0.900 on a fronting proxy).
3. Wait. No players required. The server SIGSEGVs within minutes to hours; crash frequency tracks abort frequency (during restart storms with slow responses it crash-looped every 1-10 min; with fast responses, one crash per several hours).
   Likely synthetic repro (we have not verified this exact loop): while true; do curl -m 0.05 http://:30120// >/dev/null 2>&1; sleep 0.2; done

Expected Behavior

Aborted/timed-out HTTP client connections to resource HTTP handlers are torn down gracefully.

Actual Behavior

Two fatal modes, interleaved, player-independent:
Mode 1 — SIGSEGV / SEGV_MAPERR on the main thread (tid 1), inside musl memcpy ("movs byte es:[rdi], byte ds:[rsi]", frame 0 = libc.musl-x86_64.so.1+0x64bf0), reading an UNMAPPED SOURCE. In two independent minidumps rdx=rbx=r12=0x1c0 (448) — exactly the wire size of the aborting client's HTTP request (confirmed via tcpdump). Identical cfx-server frame offsets in both dumps.
Mode 2 — console prints "[fatal] [fivem] Assertion 'm_ref' failed | Ref invalid" then terminate, with an empty "Callstack |". Note: this literal string does not exist in the public master tree; the only public m_ref assertion is assert(!m_ref.empty()) in fx::FunctionRef::GetRef (ResourceCallbackComponent.h). A candidate double-move of the handler FunctionRef exists on the HTTP cancel path (ResourceHttpHandler.cpp:135-141 in public master).
Disabling the 1 Hz scrape job stopped the crashes; the sibling 15s-interval scrape of the native /perf endpoint has never caused problems.

Evidence

Dump #1 (crash 01:47 UTC):

Crash reason: SIGSEGV / SEGV_MAPERR
Crash address: 0x0000728fdc0f8030
Crashing instruction: movs byte es:[rdi], byte ds:[rsi]
Thread 0 (crashed) - tid: 1
0 libc.musl-x86_64.so.1 + 0x64bf0
rdx = 0x00000000000001c0 rbx = 0x00000000000001c0 r12 = 0x00000000000001c0
rsi = 0x0000728fdc0f8030 (unmapped source) rdi = 0x0000728fb78230a4
1 cfx-server + 0x4e2056
2 cfx-server + 0xdbbb19
3 cfx-server + 0xdb7c91
4 cfx-server + 0xeaa880
5 libnode26.so + 0x54193a0
6 libnode26.so + 0x4259ddd
7 cfx-server + 0xe3bfc1
8 cfx-server + 0xdd58ed
9 cfx-server + 0xe9e3c3
(frames found by stack scanning)

Dump #2 (crash 04:16 UTC — identical offsets):

Crash reason: SIGSEGV / SEGV_MAPERR
Crash address: 0x00007366d980c010
Crashing instruction: movs byte es:[rdi], byte ds:[rsi]
Thread 0 (crashed) - tid: 1
0 libc.musl-x86_64.so.1 + 0x64bf0
rdx = 0x00000000000001c0 rbx = 0x00000000000001c0 r12 = 0x00000000000001c0
rsi = 0x00007366d980c010 (unmapped source) rdi = 0x00007366b46a30a4
1 cfx-server + 0x4e2056
2 cfx-server + 0xdbbb19
3 cfx-server + 0xdb7c91
(frames found by stack scanning)

Note: 0x1c0 = 448 bytes = the exact wire size of the aborting scraper's HTTP request (confirmed via tcpdump on the host). Identical register pattern and frame offsets across both dumps.

Full Breakpad minidumps are available for every crash but cannot be attached publicly: with runtime config the entire server config is passed via +set command-line args, so each dump's MD_LINUX_CMD_LINE stream contains the license key and DB/object-storage/Redis credentials in cleartext.

Additional Context

No response

[Gogsi]

A fix for this bug has been implemented and is available in the latest patch b95. Please try again and let us know if the issue persists. Thanks for your help!