v0.0.4.57 — the response cache neither answers nor stores from the first request after the pause file appears

Fixed

• The response cache neither answers nor stores while the pause file exists, from the first request after it appears. The pause file (Q.web.cache.pauseFile) was looked at no more than twice a second, so for up to half a second after an installation or a maintenance window began the cache still answered with the site. put() never looked at it, so a page a worker rendered during the pause, for a visitor the application lets through or from a database being rebuilt, was stored and handed to everybody once the pause ended. paused() now stats the file for every lookup and every store, and put() stores nothing while it exists. tests/unit-cache-paused-neither-answers-nor-stores.php starts a server and checks both, from the first request after the file appears.

Updated

• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.57.

---

Full changelog

v0.0.4.56 — instances share one opcode cache and one APCu, and a list of files is stat()ed in one go

Added

• Q.webserver.instances: several servers on one port that share one opcode cache and one APCu segment. Started one by one (reusePort, a launcher starting each), every server set PHP up on its own, so each had a cache of compiled scripts and an APCu segment of its own: a script was compiled once per server, and an entry the application keeps in APCu was missed and paid for once per server. On Exponential with four servers, the administration dashboard -- about 2.3 s of database work when its query cache misses -- took 2.3 to 3.3 s on each of the four in turn after every write it depends on, and 3 of 8 requests landed on a server that had not paid yet, where PHP-FPM, one pool with one APCu, paid once. Started once with Q.webserver.instances set, the server now forks the others itself after PHP has set up its shared memory and before any of them binds the port, so all of them and all their workers use one opcode cache and one APCu, as the children of one FPM pool do. Each is forked twice and starts a session, so none is the parent of another; each writes its own pid file (server.pid, server.1.pid ...) and start record; QBIX_FORKED_INSTANCE names it, so a reload comes back as the same instance and forks nothing. reusePort is implied. See architecture.md; tests/unit-instances-share-memory.php covers it (pid files, no parent among them, both answering, one APCu counter counting every request once, a reload, and ending one leaving the other).
• Q_WebServer_CompatFileWrapper::nativeStats(): the size and time of a whole list of files with PHP's own stat(). Under the include wrapper every filemtime() and filesize() is answered through libcurl's file protocol, about 30 microseconds a file against 3 for PHP's own stat, because every swap of the wrapper leaves a resource behind for the life of the worker. An application that looks at thousands of files at once can now hand over the list: one swap for all of it, the wrapper back in place before it returns (also when a stat makes an error handler throw), nothing remembered. Exponential's help index compares ~9 000 pages every few seconds; under the wrapper that took 310 to 410 ms and now takes what PHP-FPM takes, about 100 ms with the rest of the look. See compatibility.md; tests/unit-compat-native-stats.php covers it.

Updated

• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.56.

---

Full changelog

v0.0.4.55 — the server warns when the application's cache file loads classes the transform never reaches

Added

• A class that Q.web.appCache.file loads besides its own is reported at start. That file is loaded in the server process before the source transform is in place, and every worker inherits what it compiled, so such a class never gets the transform: its header(), setcookie() and exit stay PHP's own, which do nothing under the CLI SAPI, and nothing says so. With Exponential this went unseen for two weeks: the kernel's cache class started loading its file helpers, and every download on the server went out as text/html without its Content-Type, Content-Disposition or 206. The server now compares the declared classes and included files before and after loading the file. Each file or class beyond the listed one that the transform would change is logged as a warning naming the file and the classes, with what goes wrong and the fix, and is shown on the control panel's Cache tab; one the transform would leave unchanged is only listed there, as information. Engine files are not counted. Q.web.appCache.allow (class names or file paths) takes a file or class off the list; Q.web.appCache.strict: true refuses to start when a loaded class needs the transform. See cache.md; tests/unit-app-cache-preload.php covers it (a clean file, a helper calling header(), a helper the transform would not change, a second class in the file itself, a function library, both allow forms and strict mode).

Updated

• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.55.

---

Full changelog

v0.0.4.54 — Velocity installs and runs on PHP 8.0 (Red Hat Enterprise Linux)

Added

• PHP 8.0 is supported: composer.json requires PHP 8.0 or later instead of 8.1, so Composer installs the engine on Red Hat Enterprise Linux 9, whose default PHP is 8.0 (AppStream), and on RHEL 8 with the php:8.0 stream. Nothing new is required -- no package, extension or setting: no shipped file uses syntax newer than 8.0, every newer function is used behind a check, and the places where 8.0 behaves differently are handled by the engine (below). The whole unit suite runs on PHP 8.0 to 8.5; on 8.0 only the cases that need 8.1 syntax (enums, first-class callables) are skipped, with the reason printed. rhel.md is the new page for it: why 8.0 works, what differs there, installing on RHEL 8 and 9 (AppStream or Remi), the extensions and their EL packages, systemd, SELinux and the firewall, and how to verify an installation.
• The EL 9 package installs on the distribution's default PHP 8.0. It depends on php-cli >= 8.0, and no module stream has to be enabled first. Every release now installs it in a clean AlmaLinux 9 container on AppStream's PHP 8.0, checks the lite extension set and serves a page as the service user, besides the takeover from qbix-webserver (which runs on the php:8.2 stream the former package needed).
• envvars reaches the server's environment, as Debian's /etc/apache2/envvars does. At start-up the server reads the envvars file of every tree in its stack (base first, the top overlay winning) and exports what it sets before anything is forked, so every worker sees it with getenv() (and in $_ENV and $_SERVER) — whichever shell started or restarted the server, and whatever that shell had set. Until now only the *_RUN_* settings were read from it. The file is still read, never run: a value only a shell could work out ($X, `cmd`, a backslash, a bare value with a space) is not exported, and a variable the server was started with is kept. Names that change how programs are loaded or found, who the process is, or that collide with request headers are refused with a line on standard error: PATH, LD_*, DYLD_*, MALLOC_*, HTTP_*, PHPRC, PHP_INI_SCAN_DIR, GLIBC_TUNABLES, HOME, USER and the others listed in layout.md. --layout lists the refused lines under envvarsRefused. The start record a restart reuses leaves out what came from the file, so a changed value is taken at the next restart. tests/unit-envvars-export.php covers it, end to end through a pool worker.

Updated

• On PHP 8.0 a worker puts the statics of a class first loaded during a request back to their declared values. PHP 8.0 keeps a user class's static values in the same table as its declared defaults (unless the opcode cache made the class immutable), so ReflectionProperty::getDefaultValue() answers with the value the property holds now. The snapshot took that for the default: a class first loaded in a request was "reset" to that request's values, and every later request in the worker began with them. With Exponential the siteaccess stayed the first request's, so after one request for the public site the administration interface answered as the public site. On 8.0 the declaration is now read from the class's source and evaluated in its scope and namespace (Q_WebServer_Snapshot::declaredDefault()); the warm-up guard uses the same reader. A worker holding a static whose default cannot be told -- a class from eval() -- is replaced after that request instead of carrying it. tests/unit-snapshot-declared-defaults.php covers every form a declaration takes, on every version.
• On PHP 8.0 private and protected statics are put back between requests. Reflection reaches them there only after setAccessible(true), which the snapshot now calls on 8.0 (and only there: 8.5 deprecates it). They used to be skipped without a word, so they leaked from one request to the next.
• The dashboard's log tail shows new lines on PHP 8.0. It read a log's size through PHP's stat cache, which on 8.0 still held the size of the previous poll.
• The extension manifest's PHP minimum is 8.0, so qbixctl ext:check and the start-up check run on an 8.0 interpreter instead of stopping with "PHP 8.0 is older than the minimum".
• The documentation says PHP 8.0 to 8.5 wherever it named a version: README, requirements, packages, running, workers (the zygote runs on 8.0 as well), images (AVIF needs 8.1), the API discovery and benchmark pages.
• A changed script is compiled once for all workers when the opcode cache checks timestamps on every include. The include wrapper dropped the cache's compile of any file whose mtime had moved since that worker last served it, also when the cache (opcache.validate_timestamps=1, opcache.revalidate_freq=0) already recompiles a changed file by itself. Each worker then threw away the compile another worker had just stored and compiled the file again, and every discarded copy stayed in shared memory as waste. With a large generated data file — the 17 and 20 MB help indexes Exponential rebuilds whenever a help page changes — this cost every worker a full compile (about a second) per change, and in a test with eight workers and a 17 MB file changed three times, the waste reached 252 MB instead of 126 MB; once the cache is that full of waste nothing more can be stored, and every include compiles again (about a second each here) until a restart. In that mode the wrapper now leaves the compile to the cache; with validate_timestamps=0 or revalidate_freq above 0 it still drops it, as before. tests/unit-compat-shared-compile-kept.php covers it.
• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.54.

---

Full changelog

v0.0.4.53 — SVG files are served sandboxed

Added

• SVG files go out with Content-Security-Policy: sandbox; default-src 'none'; style-src 'unsafe-inline' and X-Content-Type-Options: nosniff. An SVG opened directly (its URL in the address bar, an <object>, an <iframe>) is a document of the origin it came from, and a browser runs its script. Every .svg and .svgz the server sends as a static file now runs nothing and loads nothing that way; shown with <img> or from a style sheet it renders as before. Over HTTP/1.1 and HTTP/2, from the in-memory copy, and for an X-Accel-Redirect answer; a header Q.webserver.headers already names is not sent twice. A second line of defence for applications that store uploaded SVG images (Exponential sanitizes them on upload). Q.webserver.svgSandbox: false turns it off. See security.md; tests/unit-svg-sandbox-headers.php covers it.

Updated

• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.53.

---

Full changelog

v0.0.4.52 — the dashboard's live status follows the browser's session

Fixed

• The dashboard no longer sits on a red "Connecting" after its session has ended. The page wrote the session token it was loaded with into its WebSocket URL, and the server checked the session only at the handshake. A dashboard left open stayed green on a session that had ended (seven days after sign-in, a sign-out, a password change) until the next restart closed the socket; from then on every reconnect presented the dead token, was refused with a 403 the page cannot see, and was retried every two seconds without end, even after signing in again in another tab. The page now leaves the cookie session's token out of the URL, so each reconnect presents the browser's current Q_panel_token cookie, and the session token no longer appears in the access log lines of /Q/ws.
• A session still waiting for its second factor opens nothing on the admin surface. Through the cookie it opened /Q/dashboard and its WebSocket, /Q/stats, /Q/metrics and /Q/phpinfo with the password alone. Q_WebServer::adminCredential() now applies the rule of Q_WebServer_Panel_Auth::validateToken() to every credential.

Updated

• An open dashboard socket is closed when its session ends. The two-second heartbeat checks the session that admitted each socket; the page is told ({"type":"auth","ok":false}) and the socket closed with code 4401. Sockets opened with Q.dashboard.token or from this machine stay open.
• GET /Q/ws without Upgrade says whether this browser would be let in, as JSON: 200 {"ok":true}, or 403 with the sign-in page that returns to the dashboard. Answered over HTTP/1.1 and HTTP/2 alike (over HTTP/2 the path used to fall through to the application's 404).
• The header status says what is going on and recovers by itself: Live (green), Connecting or Reconnecting (amber), Signed out with a sign-in link, or Offline (red). Retried with a backoff of 1 s doubling to 10 s and never given up; at once when the tab is shown again, comes back from the back-forward cache or the network returns; a connection without a heartbeat for ten seconds is replaced. See docs/dashboard.md, "The live status in the header".
• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.52.

---

Full changelog

v0.0.4.51 — the warm-up's request ends before the pool forks

Updated

• The warm-up's request ends before the pool forks. The script named by Q.webserver.warmup runs in the parent, and for an application it is a request -- Exponential's renders the front page through its kernel -- but nothing ended it, and what it registered lives in the compatibility layer, which the statics snapshot leaves alone. Every worker inherited it: the shutdown functions the warm-up registered ran in each worker's first request beside the request's own, and with a fresh worker per request (forkPerRequest) in every request; the headers and response code it set went out with those responses (with persistent workers they were in the snapshot, so in every response); its error and exception handlers were installed in requests that never set them; a session it opened kept its file locked in every worker; its ini and environment changes stood until a worker's first request ended. The pool now ends that request after the warm-up, succeeded or failed, before the snapshot and the first fork: the shutdown functions are dropped without running (they belong to a render nobody received), headers, response code and the Platform's response state are cleared, handlers are unwound to the boot ones, a session is released unwritten, ini values and environment are put back, and the output buffers it left open are discarded. The startup log says what was dropped. See reset.md. tests/unit-warmup-request-does-not-reach-workers.php covers both worker modes; it fails on v0.0.4.50.
• A request's $_SESSION ends with the request, as under PHP-FPM. Superglobals are not part of the statics snapshot and the worker did not rebuild this one, so a later request in the same persistent worker that started no session still saw the previous visitor's session data in it.
• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.51.

---

Full changelog

v0.0.4.50 — a HEAD over HTTP/2 is its headers only, and a stop drains its connections

Updated

• A HEAD over HTTP/2 is answered with a single HEADERS frame carrying END_STREAM, and no DATA. A HEAD for a page PHP rendered was sent its whole body as DATA frames, which a strict client rejects: curl with a current nghttp2 reported Error in the HTTP2 framing layer (or stream 1 was not closed cleanly: PROTOCOL_ERROR) for every rendered page, while older clients dropped the stray bytes silently. The connection now knows the request was a HEAD and drops the body itself, for a rendered page, a cached one, a static file, a streamed answer and the server's own error pages alike. The headers are the GET's: Content-Length and Content-Encoding included, computed from the body exactly as for the GET (a large static file states its size, or nothing when the GET would compress it).
• A HEAD over HTTP/2 is answered from the response cache, from the GET's entry, as over HTTP/1.1. It was never asked, so a monitor sending HEAD rendered the page every time. The application's cache (Q.web.appCache) is asked for a HEAD as the GET it stands for, so its answer states the GET's length instead of an empty body's.
• A streamed answer to a HEAD ends at its head, over HTTP/1.1 and HTTP/2, and the worker is told to stop as for a client that left. It ran the script to its end only to drop every piece, holding a stream slot (maxStreams) all the while; for an event stream, until streamTimeout. A fork-per-request child no longer writes chunks after the head of a HEAD either.
• The server's own HTTP/1.1 answers to a HEAD carry no body: the 403 for a type not served, a 404, and the pool's 502, 503 and 504 pages.
• A stop drains gracefully. Each HTTP/2 connection is sent GOAWAY with NO_ERROR and its last stream; the streams up to it are answered in full, a stream opened after it is refused with REFUSED_STREAM so the client retries it elsewhere, and the connection closes as soon as it is done. An idle HTTP/1.1 keep-alive connection is closed at once. A stop used to wait out the whole drain timeout (five seconds) whenever a browser held a connection open for reuse, with the listener already closed, and then closed those connections without a word, failing every request multiplexed on them together. Measured on a test server with one request in flight: exit after 1.4 s instead of 5.2 s, and the connection ended by GOAWAY instead of cut.
• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.50.

---

Full changelog

v0.0.4.49 — streaming responses from persistent pool workers

Added

• Streaming responses from persistent pool workers. A response that asks to stream -- Content-Type: text/event-stream, X-Accel-Buffering: no, or Q_WebServer_State::setStreaming(true) -- is sent as it is written: its status and headers on the first flush()/ob_flush(), then what the script printed on every flush, chunked over HTTP/1.1 and as DATA frames over HTTP/2, with no Content-Length, no compression and nothing stored in the response cache. A pooled worker used to collect the whole response and send it when the script ended, so Server-Sent Events arrived all at once at the end (measured: 40 events written 500 ms apart, all delivered at 20.7 s, with a Content-Length); only a fork-per-request child without a pool could stream. See workers.md. tests/unit-pool-streaming.php covers both protocols.
• A client that leaves a stream ends it. The server notices at once over HTTP/1.1 and within a second over HTTP/2 (RST_STREAM or a closed connection) and tells the worker: with ignore_user_abort off the script ends at its next flush, as under PHP-FPM, and with it on connection_aborted() returns 1. The worker then serves the next request; it used to run the script to the end for nobody. flush(), connection_aborted(), connection_status() and ignore_user_abort() are now provided by the compatibility layer in pooled requests.
• Q.webserver.streamTimeout (default 300 s): how long a streaming response, or a request with Accept: text/event-stream before its first byte, may run instead of requestTimeout. At the limit the stream is ended cleanly, so an EventSource reconnects. A stream used to be killed at requestTimeout (30 s) with nothing sent at all.
• A cap on streams: Q.webserver.streamShare (default 0.25 of workers, at least one) or Q.webserver.maxStreams (an absolute number). Each open stream holds a worker; past the cap a request for an event stream is answered 503 with Retry-After: 5, and the rest of the pool stays free for pages.
• Q.webserver.retryWithin (default 10 s): see below.

Updated

• A request whose worker died is run again only when that is safe. Only a GET, HEAD or OPTIONS that was lost within retryWithin seconds of reaching its worker, that is not a request for an event stream, and of whose answer nothing was sent, is retried (once). A long GET used to be retried however long it had run, so it ran twice; the log now says why a request was not run again. A stream whose worker dies part-way is cut short (HTTP/1.1 closes without the last chunk, HTTP/2 resets the stream) rather than answered with a second status line.
• An HTTP/2 connection with a stream in progress, or a request still with its worker, is not closed as idle after Q.web.http2.limits.idleSeconds, however quiet the socket.
• An HTTP/2 request answered at once by the pool (a refusal) no longer leaves a half-made stream behind, which every later WINDOW_UPDATE for it tripped over with PHP warnings.
• Q_WebServer_State::isStreaming() reads the header names case-insensitively, so header('content-type: text/event-stream') opts in like Content-Type.
• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.49.

---

Full changelog

v0.0.4.48 — the full Docker images load event after sockets

Fixed

• The full Docker images load event, and PHP starts in them without a warning. docker-php-ext-enable names an extension's ini file after it and PHP reads conf.d in name order, so event loaded before sockets, whose symbols it uses, and failed with undefined symbol: socket_ce. Every PHP command in the image began with that warning, which broke the image test that reads the first line of vc-qshell --version, and every full image failed its build test from v0.0.4.42 on. event is now enabled as zz-docker-php-ext-event.ini, after every docker-php-ext-*.ini (loadAfter in packaging/docker/debian-build-deps.json), and an extension is recorded as installed in image-extensions.txt only when PHP loads it: one that does not is left out, with its ini removed, instead of being listed as installed while qbixctl ext:check found it missing.
• sbin/qbixserver.phar is rebuilt from these sources, stamped v0.0.4.48.

---

Full changelog