Pgpool-II: Example about "In Memory Query Cache" #1292

amotl · 2025-12-09T04:45:42Z

About

Explore Pgpool-II's In Memory Query Cache with CrateDB.

References

Query cache for CrateDB crate-clients-tools#276

coderabbitai · 2025-12-09T04:45:48Z

Walkthrough

Adds a new Pgpool-II integration for CrateDB: Docker Compose orchestration, Pgpool configuration, a test orchestration script, documentation, and a GitHub Actions workflow to run the test stack.

Changes

Cohort / File(s)	Change Summary
CI / Workflow `.github/workflows/application-pgpool.yml`	New GitHub Actions workflow "Pgpool-II" that runs on push/PR, schedule, and manual dispatch; runs a single matrix job that checks out the repo and runs `application/pgpool/test.sh`.
Compose / Orchestration `application/pgpool/compose.yml`	New docker-compose configuration defining `cratedb`, `pgpool`, healthchecks, utility/task containers (`provision-functions`, `cratedb-query-data`, `pgpool-query-data`, `crash`, `start-dependencies`), volumes, env vars, and startup dependency ordering.
Pgpool Configuration `application/pgpool/pgpool.conf`	New Pgpool-II configuration file including listen/port and in-memory query cache settings (memqcache backend, host/port, sizing, expiration, block size, table list comments).
Test orchestration `application/pgpool/test.sh`	New shell script to bring up the compose stack, run provisioning (`provision-functions`), and execute data queries via the `pgpool-query-data` task.
Documentation `application/pgpool/README.md`	New README describing the Pgpool-II + CrateDB setup, components included, operations examples, status notes, and external links.

Sequence Diagram(s)

sequenceDiagram
    participant CI as GitHub Actions
    participant Runner as Test host
    participant Compose as Docker Compose
    participant Pgpool as Pgpool-II
    participant Crate as CrateDB

    CI->>Runner: checkout + run application/pgpool/test.sh
    Runner->>Compose: docker compose up --detach (cratedb, pgpool)
    Compose->>Crate: start CrateDB (healthcheck)
    Compose->>Pgpool: start Pgpool-II (healthcheck)
    Compose->>Compose: run provision-functions after Crate & Pgpool healthy
    Compose->>Pgpool: run pgpool-query-data (client query via Pgpool)
    Pgpool->>Crate: forward queries
    Crate-->>Pgpool: query responses
    Pgpool-->>Runner: return query results (task exit)
    Runner-->>CI: test.sh exits (status)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Pay attention to:
- application/pgpool/compose.yml healthchecks, service dependency ordering, and volume mounts
- application/pgpool/pgpool.conf memqcache parameters and host/port correctness
- application/pgpool/test.sh commands and use of docker compose flags

Poem

🐰 With twitching nose I hopped to play,

I stitched the pools where queries stay.
Crate hums low, Pgpool sings along,
In compose and config, we belong.
A tiny rabbit cheers: cache on! 🥕

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding a Pgpool-II example demonstrating the In Memory Query Cache feature with CrateDB.
Description check	✅ Passed	The description is directly related to the changeset, explaining the purpose of the example (exploring Pgpool-II's In Memory Query Cache with CrateDB) and providing relevant references.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch pgpool

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

amotl · 2025-12-09T05:26:57Z

application/pgpool/compose.yml

+  # BACKEND Error: "Unknown function: pg_catalog.has_function_privilege('crate', 'pgpool_regclass(cstring)', 'execute')"
+  provision-functions:
+    image: docker.io/crate/crate:nightly
+    entrypoint: /bin/bash -c
+    command:
+      - |
+        crash --hosts "http://cratedb:4200" <<SQL
+        CREATE OR REPLACE FUNCTION pg_catalog.has_function_privilege(string, string, string)
+        RETURNS BOOLEAN
+        LANGUAGE JAVASCRIPT
+        AS 'function has_function_privilege(foo, bar, baz) { return true; }';
+        SQL
+    deploy:
+      replicas: 0
+    depends_on:
+      cratedb:
+        condition: service_healthy
+


This detail about missing pg_catalog.has_function_privilege() needs to be reported to crate/crate.

Missing function pg_catalog.has_function_privilege crate#18814

amotl · 2025-12-09T06:12:43Z

application/pgpool/notes.md

+```
+LOG:  get_query_result: do_query failed
+LOG:  get_query_result: no result returned
+ERROR   : Unknown function: pg_catalog.pg_is_in_recovery()
+```


Substitute and/or implement pg_catalog.pg_is_in_recovery(), i.e. also report to crate/crate?

Substituted as UDF with 327350a, and reported to the database issue tracker.

Missing function pg_catalog.pg_is_in_recovery crate#18815

amotl · 2025-12-09T06:13:18Z

application/pgpool/notes.md

+```
+WARNING:  is_temp_table: unexpected PostgreSQL version: CrateDB 6.2.0-SNAPSHOT (built 9e707bd/NA, Linux 6.12.54-linuxkit amd64, OpenJDK 64-Bit Server VM 25.0.1+8-LTS)
+```


Is is_temp_table also a missing function, or is that message just some other spurious side effect? Is it actionable or ignorable?

It's a function in Pgpool-II: Because the PostgreSQL implementation is based on the server version, the code needs to do a version inquiry first. That's where this warning is coming from, because CrateDB's version string doesn't parse like PostgreSQL's.

https://github.com/pgpool/pgpool2/blob/55b6b9a9d4d82d4635dfd588c4869f3b252d94aa/src/utils/pool_select_walker.c#L760-L774

Is it advisable to submit a patch to make Pgpool also accept non-PostgreSQL servers (e.g. CrateDB) here with a reasonable default implementation?

I think it's unlikely that Pgpool-II will accept a patch for CrateDB, and it's only a WARNING, so let's ignore it?

https://www.pgpool.net/docs/latest/en/html/runtime-in-memory-query-cache.html

amotl marked this pull request as ready for review December 9, 2025 05:20

amotl requested review from matriv and seut December 9, 2025 05:23

amotl commented Dec 9, 2025

View reviewed changes

This comment was marked as spam.

Sign in to view

amotl commented Dec 9, 2025

View reviewed changes

seut removed the request for review from matriv December 9, 2025 13:17

amotl force-pushed the pgpool branch from 6610e68 to 327350a Compare December 9, 2025 19:29

This comment was marked as resolved.

Sign in to view

This was referenced Dec 9, 2025

Missing function pg_catalog.has_function_privilege crate/crate#18814

Open

Missing function pg_catalog.pg_is_in_recovery crate/crate#18815

Closed

amotl force-pushed the pgpool branch from 327350a to 6995528 Compare December 9, 2025 20:35

This comment was marked as spam.

Sign in to view

Pgpool-II: Example about In Memory Query Cache

efb6e0a

https://www.pgpool.net/docs/latest/en/html/runtime-in-memory-query-cache.html

amotl force-pushed the pgpool branch from 6995528 to efb6e0a Compare December 9, 2025 20:45

amotl changed the title ~~Pgpool-II: Example about In Memory Query Cache~~ Pgpool-II: Example about "In Memory Query Cache" Dec 9, 2025

This comment was marked as spam.

Sign in to view

amotl mentioned this pull request Dec 10, 2025

Support pg_catalog.pg_is_in_recovery() crate/crate#12085

Open

Pgpool-II: Example about "In Memory Query Cache" #1292

Are you sure you want to change the base?

Pgpool-II: Example about "In Memory Query Cache" #1292

Uh oh!

Conversation

amotl commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

About

References

Uh oh!

coderabbitai bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

amotl Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amotl Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

This comment was marked as spam.

Uh oh!

amotl Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amotl Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amotl Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amotl Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amotl Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amotl Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

amotl commented Dec 9, 2025 •

edited

Loading

coderabbitai bot commented Dec 9, 2025 •

edited

Loading

amotl Dec 9, 2025 •

edited

Loading

amotl Dec 9, 2025 •

edited

Loading

amotl Dec 9, 2025 •

edited

Loading

amotl Dec 9, 2025 •

edited

Loading

amotl Dec 9, 2025 •

edited

Loading

amotl Dec 9, 2025 •

edited

Loading