Conversation
Add a new GitHub Actions job (pg_extension_tests_native) that runs native e2e tests for the PG extension against a live KalamDB + pgrx PostgreSQL. The job downloads the built server binary, installs system deps and Rust tools, initializes pgrx, starts the server, runs cargo nextest for the kalam-pg-extension e2e tests, stops services, collects logs and uploads artifacts. Wire the new job into the release workflow dependencies. Also: update a few workflow curl invocations to use -fsS and remove redundant KALAMDB_ROOT_PASSWORD exports in some steps. Change JobsManager logging for fetched jobs from info to debug. Update docker/build/test-docker-image.sh by adding a container_get helper and switching health/version requests to use busybox wget via docker exec for more reliable in-container checks.
Replace inline server start/bootstrap in the release workflow with a dedicated scripts/test-typescript-sdk-release.sh runner and pass KALAMDB_SERVER_BIN into the job. Add the test runner, example server.toml and captured test output, and ignore a local ts-sdk-repro/server.toml in .gitignore. Improve scripts: make pgrx-version resolution robust in pg/scripts/pgrx-test-setup.sh; strengthen scripts/cluster.sh PID handling, free leftover HTTP/RPC listeners, add get_access_token and use the e2e-tests feature flag for cargo tests, and have detect_leader_url authenticate with the server. These changes centralize TS SDK release testing, reduce CI flakiness, and harden local cluster/test tooling.
Upgrade CI workflows (actions/cache -> v5, download-artifact -> v6, setup-node -> Node 22) and add a Rust cache step for the dart bridge. Improve Docker healthcheck and test scripts to use a robust GET (-O /dev/null / curl) to avoid h2c/HEAD issues. Optimize link crate: pre-serialize WASM ping payload, use zero-copy gzip helper (decompress_if_gzip) for WS messages, add inline annotations and helper functions for validation to avoid allocations, and rewrite timestamp formatting to use write! into pre-sized buffers (with added regression tests). Bump WASM-related dependencies and update Cargo.toml accordingly.
Refine the OCI image description string in pg/docker/Dockerfile, Dockerfile.release-pg, and Dockerfile.runtime to improve clarity (changed "for connecting PostgreSQL foreign tables to a remote KalamDB server" to "so you can connect PostgreSQL foreign tables to a remote KalamDB server"). Add pg/docker/README.md documenting the pg-kalam Docker image, quick start, and usage examples. Remove the now-obsolete pg/docker/image-description.txt file.
Introduce protocol negotiation and MessagePack support for WebSocket connections: add send_message/send_data_binary helpers, handle msgpack binary frames (with optional gzip), and respect per-connection serialization/compression options parsed from the upgrade query string. Add a header-auth fast path to authenticate JWTs during the HTTP upgrade and consolidate post-validation steps into a common complete_ws_auth/send_current_auth_success flow. Split handler and job code into new workspace crates (kalamdb-handlers, kalamdb-jobs) and wire them into the workspace/backend Cargo.toml. Update websocket handler to use protocol-aware messaging, support msgpack deserialization, and add tests for protocol parsing. Also update CI workflow to wait for the KalamDB container to become healthy before starting Postgres.
Add a suite of integration tests under link/tests/proxied that simulate various TCP/proxy impairments using TcpDisconnectProxy to validate client reconnect and resume behavior. New tests include: - blackhole_during_subscribe.rs: blackhole during subscribe handshake and recovery - event_counter_integrity.rs: verify connect/disconnect event counters across outages - gradual_degradation.rs: latency ramp causing reconnect and recovery - heavy_write_burst_recovery.rs: large write backlog delivered exactly once after outage - latency_during_snapshot.rs: latency during initial snapshot and subsequent live delivery - subscribe_during_reconnect.rs: add subscription while reconnecting and ensure delivery - unsubscribe_during_outage.rs: unsubscribe during outage and ensure it is not re-subscribed Also update link/tests/proxied.rs to register the new modules and adjust rapid_flap.rs to ignore an unused connect_count variable. These tests exercise resume/replay, duplicate avoidance, subscription lifecycle, and event-counting correctness under adverse network conditions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request updates CI and release workflows to use newer versions of GitHub Actions and Node.js, and introduces a new native PostgreSQL extension end-to-end test suite. The changes improve test coverage, modernize dependencies, and enhance reliability and maintainability of the build and release pipelines.
New test suite and improved test coverage:
pg_extension_tests_nativejob to the release workflow, which runs native end-to-end tests for the PostgreSQL extension against a live KalamDB server and pgrx PostgreSQL, increasing confidence in extension compatibility and correctness. [1] [2] [3]Test script improvements:
test-typescript-sdk-release.shscript, consolidating server startup, setup, and test execution for improved maintainability and reliability. [1] [2]