feat(segmentation-service): implement User Segmentation and Audience Service (Closes #363)

[Topmatrixmor2014]

Summary

This PR implements issue #363: User Segmentation and Audience Service Setup for the Quest Service platform.

A new, fully-independent NestJS microservice at microservices/segmentation-service/ creates and manages user cohorts for targeted campaigns, A/B testing, personalization, and analytics.

Closes #363

Acceptance Criteria Mapping

Acceptance Criterion (from #363)	Implementation
Segments defined by rules	Segment + Rule entities with cascading on delete. Rules support 14 operators and AND/OR combinators.
Members calculated dynamically	In-memory rule engine evaluates UserSignal against rules, seeded with cached user data from Redis. Refreshed by evaluateSegment, the cron scheduler, and the real-time ingest endpoint.
Real-time updates work	POST /api/events upserts a UserSignal in Redis, then synchronously evaluates it against all active (or explicitly-named) segments, recording SegmentEvents.
Overlaps identified	POST /api/segments/overlap intersects membership userIds across 2+ segments and reports overlappingUsers, totalUsers, overlapPercentage, and perSegmentSizes.
Size metrics available	GET /api/segments/:id/size returns cached + DB size with measuredAt timestamp. Cached in Redis namespace segmentation:size:<id> and on the Segment.cachedSize column. Dashboard endpoint summarises total membership.
Service runs independently	Standalone Dockerfile + docker-compose.yml (Postgres + Redis + service). No shared runtime with src/, makes its own TypeORM + ioredis connections.

New Microservice

30+ files added under microservices/segmentation-service/

• Configuration: package.json, tsconfig.json, tsconfig.build.json, nest-cli.json, .env.example, .eslintrc.cjs, .prettierrc, .gitignore
• Container: Dockerfile (multi-stage node:20-alpine build) + docker-compose.yml (postgres + redis + service)
• Docs: README.md (endpoint table, run instructions, rule schema, docker instructions)
• Tests: 17 unit tests across 3 suites (rule engine, segmentation service, app controller)

Entities (TypeORM / PostgreSQL)

• Segment – cohort definition (slug, name, type, status, metadata, cachedSize, evaluationIntervalSeconds, lastEvaluatedAt)
• Rule – ordered predicate belonging to a Segment (field, operator, jsonb value, category, AND/OR combinator)
• Membership – user-in-segment link with a source enum (manual > realtime > evaluation)
• SegmentEvent – audit row for membership transitions (added / removed / refreshed)
• AbExperiment – A/B test bound to an optional Segment
• AbAssignment – sticky user-to-variant mapping with a unique (experimentId, userId) index for idempotent inserts

Rule Engine (14 operators)

SegmentationRuleEngineService:

• equals, notEquals, in, notIn, contains, notContains, gt, gte, lt, lte, between, exists, notExists, regex
• Field resolution supports top-level signal fields, attributes.* dot-paths, and bare attribute names.
• AND/OR combinators are evaluated per-rule (each rule joins with the previous). First rule's combinator is ignored (no previous).

Segmentation Service

SegmentationService:

• Segment CRUD with cascade (create, update, delete, getSegment, listSegments)
• Rule attach / detach
• Evaluation: evaluateSegment(id, { userId | userIds | limit? }) + cron-driven evaluateAllDueSegments()
• Membership: checkMembership, addManualMembers, removeManualMember, listMembers, plus auto-management during evaluation
• Overlap analysis: overlap({ segmentIds })
• A/B: createExperiment, assignVariant(id, dto) with deterministic MD5-hashed buckets + sticky Redis cache
• Size: getSize(id) (cache → DB fallback) and a service-wide dashboard
• Bootstrap: seeds 3 default segments (high-value-players, us-mobile-users, at-risk-churners) unless SEGMENTATION_SEED_DEFAULTS=false

Real-Time Ingest

POST /api/events:

• Upserts a per-user signal in Redis (24h TTL).
• Traverses either the explicit segmentsToReEvaluate set or every active segment, runs the rule engine, and books-adds/removes memberships. Explicitly-requested segments never auto-remove members.

Redis Cache (RedisCacheService)

Namespace segmentation:*:

• segmentation:user:{userId} – user signal JSON
• segmentation:size:{segmentId} – cached size counter
• segmentation:assignment:{experimentKey}:{userId} – sticky variant

Scheduler (SegmentationScheduler)

Cron task that runs the due-segment evaluation queue every minute (gated by SEGMENTATION_SCHEDULER_ENABLED).

API Surface

Method	Path	Purpose
GET	/api/health	Liveness probe
GET	/api/segments	List every segment (with rules)
GET	/api/segments/dashboard	Service-wide summary
POST	/api/segments/overlap	Compute overlap for 2+ segments
POST	/api/segments	Create segment (rules optional inline)
GET	/api/segments/:id	Fetch a segment with rules
PATCH	/api/segments/:id	Update segment metadata / status
DELETE	/api/segments/:id	Remove a segment
POST	/api/segments/:id/rules	Append a rule
DELETE	/api/segments/:id/rules/:ruleId	Remove a rule
POST	/api/segments/:id/evaluate	Force re-evaluation (optional userId(s) or limit)
POST	/api/segments/:id/membership	Manually add members
DELETE	/api/segments/:id/membership/:userId	Manually remove a member
GET	/api/segments/:id/members	List members
GET	/api/segments/:id/size	Cached size + measured timestamp
POST	/api/segments/:id/check	Check if a user matches a segment
POST	/api/events	Send a behavioural/demographic signal
POST	/api/experiments	Create A/B experiment
POST	/api/experiments/:id/assign	Assign + cache variant for user

How to Run

# Local development
cp .env.example .env
npm install
npm run start:dev

# Full stack via docker
docker compose up --build

# Tests / type-check / lint
npm run type-check
npm run test
npm run lint:check

By default the service listens on PORT=3023 and exposes Swagger friendly JSON at /api.

Testing

• npm run type-check – 0 errors (tsc --noEmit)
• npm run lint:check – 0 errors, 1 non-blocking any warning in the unit-test mock
• npm run test – 17 / 17 passing across:
  • segmentation-rule-engine.service.spec.ts (covers every operator + combinator + dot-notation + inactive rules)
  • segmentation.service.spec.ts (segment CRUD, duplicate-slug rejection, overlap with shared / disjoint / empty segments, deterministic A/B variants, sticky-cache replay, size metric)
  • app.controller.spec.ts (health endpoint)

The spec mocks use a relations-aware InMemoryRepo and a stateful Redis cache mock so the service can be exercised without a database.

Diff Stats

28 files changed, 1583 insertions(+)

Checklist

• Service can compile cleanly with npm install && npm run type-check
• Service passes npm run test for the entire test suite (17 / 17)
• No TypeORM/SQL errors at boot when NODE_ENV !== production (auto-synchronize schema for dev)
• Service runs standalone via docker compose up --build
• PR targets MindFlowInteractive/quest-service:main from the fork Topmatrixmor2014/quest-service:feat/segmentation-service-363
• Rebased onto latest upstream/main so the diff is conflict-free
• Does not modify any pre-existing files outside microservices/segmentation-service/

Notes for Reviewers

• The rebase against upstream/main is clean; this PR only adds the new service directory.
• Three seeded default segments (high-value-players, us-mobile-users, at-risk-churners) ship when the DB is empty, controlled by SEGMENTATION_SEED_DEFAULTS.
• The membership source hierarchy means manual additions stick through evaluation sweeps, which is the expected behaviour for marketing-segment overrides.
• Variant assignment is deterministic across deploys thanks to the MD5 salt and explicit inclusion of experiment.key in the hash.

[drips-wave]

@Topmatrixmor2014 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits