refactor: drop legacy metrics database schema#442
Conversation
There was a problem hiding this comment.
Pull request overview
Removes the legacy “metrics” persistence layer by dropping the underlying Postgres objects and deleting the corresponding NestJS/TypeORM code and configuration that referenced them, completing step 9 of #275.
Changes:
- Added a TypeORM migration to drop legacy metrics functions, materialized views, table, and enum types.
- Removed legacy metrics module/services/controllers/DTOs/utils and TypeORM entities; unregistered them from the DB/app modules.
- Removed metrics-related pg-boss queues/scheduling/config (
METRICS_PER_HOUR) and updated related docs/manifests.
Reviewed changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| kustomize/overlays/local/backend-configmap-local.yaml | Removes METRICS_PER_HOUR from local overlay config. |
| docs/jobs.md | Updates “critical env vars” list to drop METRICS_PER_HOUR. |
| docs/infra.md | Removes metrics worker/env references from infra guidance. |
| docs/environment-variables.md | Removes METRICS_PER_HOUR from jobs env var docs. |
| apps/backend/src/metrics/utils/time-window-parser.ts | Deletes legacy metrics time-window parsing utility. |
| apps/backend/src/metrics/utils/time-window-parser.test.ts | Deletes tests for the removed metrics utility. |
| apps/backend/src/metrics/services/network-stats.service.ts | Deletes legacy metrics network stats service. |
| apps/backend/src/metrics/services/metrics-scheduler.service.ts | Deletes legacy metrics aggregation/refresh service. |
| apps/backend/src/metrics/services/failed-retrievals.service.ts | Deletes legacy failed retrievals metrics service. |
| apps/backend/src/metrics/services/failed-deals.service.ts | Deletes legacy failed deals metrics service. |
| apps/backend/src/metrics/services/daily-metrics.service.ts | Deletes legacy daily metrics query service. |
| apps/backend/src/metrics/metrics.module.ts | Deletes legacy metrics HTTP module wiring. |
| apps/backend/src/metrics/metrics-worker.module.ts | Deletes legacy metrics worker-only module wiring. |
| apps/backend/src/metrics/dto/provider-performance.dto.ts | Deletes legacy provider performance DTOs. |
| apps/backend/src/metrics/dto/network-stats.dto.ts | Deletes legacy network stats DTOs. |
| apps/backend/src/metrics/dto/failed-retrievals.dto.ts | Deletes legacy failed retrievals DTOs. |
| apps/backend/src/metrics/dto/failed-deals.dto.ts | Deletes legacy failed deals DTOs. |
| apps/backend/src/metrics/dto/daily-metrics.dto.ts | Deletes legacy daily metrics DTOs. |
| apps/backend/src/metrics/controllers/network-stats.controller.ts | Deletes legacy metrics network controller endpoints. |
| apps/backend/src/metrics/controllers/failed-retrievals.controller.ts | Deletes legacy metrics failed-retrievals endpoints. |
| apps/backend/src/metrics/controllers/failed-deals.controller.ts | Deletes legacy metrics failed-deals endpoints. |
| apps/backend/src/metrics/controllers/daily-metrics.controller.ts | Deletes legacy metrics daily endpoints. |
| apps/backend/src/jobs/repositories/job-schedule.repository.ts | Removes metrics queue mapping from pg-boss job state aggregation queries. |
| apps/backend/src/jobs/jobs.service.ts | Removes metrics queue workers/schedules and adds legacy schedule skipping. |
| apps/backend/src/jobs/jobs.service.spec.ts | Updates tests to reflect removal of metrics queues and constructor deps. |
| apps/backend/src/jobs/jobs.module.ts | Removes metrics module imports/wiring from JobsModule. |
| apps/backend/src/jobs/job-queues.ts | Removes metrics.run and metrics.cleanup queue constants. |
| apps/backend/src/database/types.ts | Removes legacy MetricType enum (no longer needed). |
| apps/backend/src/database/migrations/1762000000000-DropMetricsSchema.ts | Adds migration that drops legacy metrics DB objects. |
| apps/backend/src/database/entities/sp-performance-last-week.entity.ts | Deletes legacy materialized-view entity. |
| apps/backend/src/database/entities/sp-performance-all-time.entity.ts | Deletes legacy materialized-view entity. |
| apps/backend/src/database/entities/metrics-daily.entity.ts | Deletes legacy metrics_daily entity. |
| apps/backend/src/database/entities/job-schedule-state.entity.ts | Keeps legacy job types in the JobType union for existing DB rows. |
| apps/backend/src/database/database.module.ts | Removes deleted metrics entities from TypeORM registrations/exports. |
| apps/backend/src/config/app.config.ts | Removes METRICS_PER_HOUR validation/config loading. |
| apps/backend/src/app.module.ts | Removes MetricsModule from app wiring. |
| apps/backend/README.md | Removes METRICS_PER_HOUR from backend env var documentation. |
| apps/backend/.env.example | Removes METRICS_PER_HOUR from example env file. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
bf9511d to
83261fd
Compare
Drop metrics_daily table, sp_performance materialized views, their refresh functions, and related enums. Remove corresponding entity files and database module registrations.
83261fd to
b8d06a9
Compare
silent-cipher
left a comment
There was a problem hiding this comment.
Could you update the e2e migration tests to include the new migration DropMetricsSchema1762000000000?
d85dbb3 to
cb125b5
Compare
so i was looking into this and there is a subtle issue with including |
cb125b5 to
31b6167
Compare
492a785
into
FilOzone:refactor/cleanup-metrics-jobs
* refactor: remove metrics job * chore: address copilot comments * chore: remove metrics job references from docs and tests * chore: address copilot comments * chore: add e2e migration test * fix: correct migrations table name * refactor: drop legacy metrics database schema (#442) * refactor: remove legacy metrics API endpoints and module wiring * refactor: add migration to drop legacy metrics schema Drop metrics_daily table, sp_performance materialized views, their refresh functions, and related enums. Remove corresponding entity files and database module registrations. * test: add e2e test for DropMetricsSchema migration * fix: test e2e migrations end state * chore: address pr review * chore: address pr comments --------- Co-authored-by: durga chaitanya <107246959+Chaitu-Tatipamula@users.noreply.github.com>
Summary
Adds a database migration to drop the legacy metrics schema objects and removes the corresponding TypeORM entity files and module registrations. This completes step 9 of #275.
Depends on #441.
Changes
[NEW] Migration
1762000000000-DropMetricsSchemaDrops the following database objects in dependency order:
refresh_sp_performance_last_week()andrefresh_sp_performance_all_time()functionssp_performance_last_weekandsp_performance_all_timematerialized viewsmetrics_dailytable (withCASCADE)metrics_daily_metric_type_enumandmetrics_daily_service_type_enumtypesDeleted entity files
metrics-daily.entity.tssp-performance-all-time.entity.tssp-performance-last-week.entity.tsModified
database.module.ts— RemovedMetricsDaily,SpPerformanceAllTime,SpPerformanceLastWeekfrom entity registrations,forFeature, providers, and exportstypes.ts— RemovedMetricTypeenum (only consumer was the deleted entity)Verification
pnpm typecheck✅pnpm test✅ (249 tests)pnpm check:ci✅