More Than Just a Cache: Data Structure Databases

Companion repository for the conference talk "More Than Just a Cache: Data Structure Databases" by Andy Snell.

Redis is best known as a high performance, in-memory, key-value database used for distributed caching. However, data structure databases like Redis, Valkey, and Key DB can do so much more than just operate on string values! With over a dozen different data types like hashes, lists, sets, sorted sets, bloom filters, and streams, these databases provide a number of tools that can help solve common problems.

We’ll explore these basic data structures in Redis and Valkey, with real world examples of using them to solve problems like rate limiting, distributed resource locking, and efficiently checking membership in massive sets of data.

We'll also discuss some of the newer functionality designed for AI and LLM applications, like vector similarity searches and vector sets.

Latest Slides

Slide Deck Link: https://slides.wickedbyte.com/data-structure-databases

The talk walks through the dozen-plus native data structures in Redis-compatible systems (Strings, Hashes, Lists, Sets, Sorted Sets, Streams, Bitmaps, HyperLogLog, Geo, JSON, Time Series, Vector Sets) with PHP code for rate limiting, distributed locking, bitmap membership at scale, sliding-window queues, semantic caching for LLMs, and agent memory built on Redis 8.

This repo contains:

Path	Contents
src/	PHP examples shown on the slides — rate limiter, phone-bitmap DNC list, vector search, semantic cache
tests/	Integration tests for the PHP examples (run against a real Redis)

Requirements

• Docker (with the compose plugin / Compose v2)
• GNU Make
• Optional: a GitHub token in .env to lift Composer's API rate limit

Quick Start

make              # build the image and install vendor dependencies
make ci           # run the full PHP CI suite (lint, cs, phpstan, phpunit, rector)
make help         # list every documented Make target

The first invocation copies .env.dist to .env (without overwriting an existing file), builds the Docker image, and runs composer install.

All PHP runs inside the project's Docker container. There is no php, composer, or phpunit on the host PATH — every command goes through Make or docker compose run --rm php <cmd>.

Running the PHP Examples

The examples in src/ mirror the patterns demonstrated on the slides. They are runnable CLI commands registered on bin/app:

make bash                       # open a shell inside the container
./bin/app list                  # list available commands
./bin/app embed "some text"     # fetch an embedding vector
./bin/app rate-limit            # demo the rate-limiting pattern
./bin/app distributed-lock      # demo SET NX EX locking
./bin/app phone-bitmap          # demo the bitmap-membership pattern
./bin/app vector-search         # demo Vector Sets similarity search
./bin/app semantic-cache        # demo the semantic-cache hit/miss loop

Tests run against a real Redis container started by compose.yml:

make phpunit                       # unit + integration tests

Useful Make Targets

Run make help for the full, documented list. Highlights:

Target	Purpose
make	Build the image and install dependencies (default goal).
make bash	Open a bash shell inside the PHP container.
make psysh	Open the PsySH REPL with the project autoloaded.
make phpunit / make test	Run the PHPUnit suite (unit + integration).
make ci	Full local CI: lint, cs-check, phpstan, phpunit, rector, audit.
make pre-ci	Run the formatters/fixers, then make ci.
make cs-fix	Apply php-cs-fixer coding-style fixes.
make rector	Apply Rector refactorings (rector-dry-run to preview).
make clean	Remove vendor/ and .cache/.

For anything not exposed as a Make target:

docker compose run --rm php <your-command-here>

Local Configuration

• .env is created from .env.dist on first build and is gitignored.
• A GITHUB_TOKEN in .env is optional — it only raises the GitHub API rate limit Composer hits when resolving public packages.
• Xdebug is preinstalled but disabled by default (XDEBUG_MODE=off). Set XDEBUG_MODE=debug in .env and restart the container to enable step debugging.
• The Makefile sources .local/Makefile if it exists, so per-machine overrides can be added without modifying the tracked Makefile.

Attribution

Slide deck © Andy Snell. Licensed under CC BY 4.0. Example code © Andy Snell. Licensed under MIT

Built on WickedByte's PHP CLI project skeleton — the Docker-based dev environment, Makefile workflow, and CI tooling come from that template.