feat: add Deleted counter to usage metrics#2268
Merged
Merged
Conversation
* feat: add Deleted counter to usage metrics
- Add Deleted property to UsageInfo and UsageHourInfo models
- Add EventsDeleted counter to AppDiagnostics
- Add IncrementDeletedAsync to UsageService with cache key, save, and
GetUsageAsync support
- Update EventController.DeleteModelsAsync to track deleted event counts
- Update ResetProjectDataWorkItemHandler to track deleted events on
project reset
- Update Svelte org/project usage pages to chart Deleted series
- Update Angular org/project manage controllers to chart Deleted series
- Update generated TypeScript API types
- Add comprehensive UsageService tests for deleted metrics
- Add EventController integration tests for delete usage tracking
Does not change plan limits, billing, or admission control.
Does not instrument retention cleanup, bot cleanup, or orphan cleanup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address PR review feedback - rename group vars, fix chart colors, improve test quality
- Rename 'group' to 'projectGroup' in CleanupDataJob and EventController
to avoid confusion with C# contextual keyword
- Move RemoveByPrefixAsync back to after deletion in RemoveStacksAsync
to match original behavior (cache clear after data removal)
- Define --chart-7 (dark rose) in app.css for 'Deleted' series color
on both light and dark themes
- Use chart-7 consistently on both org and project usage pages
- Rename all new tests to three-part Method_Scenario_Expected pattern
- Add // Arrange, // Act, // Assert section comments to all new tests
- Replace == with String.Equals where comparing string values
- Replace 'i' loop variable with 'eventIndex'
- Fix long→int for GetEventsLeftAsync return type in tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add Exceptionless session management to Svelte UI
Add user identity tracking and session lifecycle management matching
the existing Angular legacy pattern:
- Create exceptionless-session.ts utility with setUserIdentity(),
endSession(), and submitFeatureUsage() -- all async/await, simple
userId + userName parameters
- Set identity and start session via getMeQuery.onSuccess (fires for
all auth methods: email, OAuth, page reload with existing token)
- End session and clear identity on logout
- Add submitFeatureUsage('organization.ChangePlan') to billing dialog
matching legacy Angular telemetry
No $effect logic needed -- identity is set imperatively when user
data resolves from the API.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: update openapi baseline, per-group try/catch, and session deduplication
- Update openapi.json baseline to include 'deleted' (int64) in UsageHourInfo
and UsageInfo required arrays and properties — fixes CI baseline test
- Move try/catch inside the IncrementDeletedAsync loop in EventController so a
cache failure on one project group does not silently skip remaining groups
- Guard submitSessionStart with an activeUserId check so repeated getMeQuery
onSuccess calls (refetch, focus, reconnect) do not create duplicate sessions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address PR feedback - fire-and-forget telemetry in billing and logout
- submitFeatureUsage in change-plan-dialog is now fire-and-forget; a
telemetry failure can no longer surface as a billing change error or
prevent the dialog from closing after a successful plan update
- endSession in logout is now fire-and-forget; a session-end failure
can no longer block query cancellation, cache clear, or token removal
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address lint errors - sort modules, imports, and union types
- Sort exported functions alphabetically (endSession before setUserIdentity)
- Sort union type (null | string) per perfectionist/sort-union-types
- Sort imports (/auth before /shared) per perfectionist/sort-imports
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address PR feedback - LastEventDateUtc, endSession state leak, unhandled rejection
- Only update LastEventDateUtc when actual ingestion occurs (not delete-only flushes)
- Use try/finally in endSession to clear identity even if submitSessionEnd fails
- Add .catch() to setUserIdentity call to prevent unhandled promise rejection
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: align Deleted field type to int32 matching other usage counters
- Change UsageInfo.Deleted and UsageHourInfo.Deleted from long to int
- Update IncrementDeletedAsync signature to int (matches Total/Blocked/Discarded)
- Fix OpenAPI baseline from int64 to int32
- Fix Valibot schema from number() to int32()
- Add explicit casts at CleanupDataJob/ResetProjectData call sites
- Add retention deletion intent comment in EnforceStackRetentionDaysAsync
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a first-class Deleted counter to organization/project usage metrics so that explicit deletes (API, project reset, soft-delete cleanup) can be distinguished from "no ingestion". The change spans backend domain model, ES mapping, Redis-backed UsageService, callers in EventController/CleanupDataJob/ResetProjectDataWorkItemHandler, generated TS clients, both Svelte and legacy Angular usage charts, plus new tests. The PR also includes some unrelated Exceptionless session/telemetry plumbing on the frontend.
Changes:
- Add
DeletedtoUsageInfo/UsageHourInfo, ES mappings,UsageService(IncrementDeletedAsync, bucket cache key, flush/read paths), and an OTel counter; also gateLastEventDateUtcon real ingestion so deletes don't mark an org/project as active. - Wire delete tracking into
EventController.DeleteModelsAsync,ResetProjectDataWorkItemHandler, andCleanupDataJob(soft-delete paths only; retention enforcement opts out viatrackDeletedUsage: false). - Surface the new counter in Svelte and Angular usage charts and update generated TS schemas/types; add tests across
UsageServiceTests,EventControllerTests,CleanupDataJobTests, and a newResetProjectDataWorkItemHandlerTests.
Reviewed changes
Copilot reviewed 24 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Exceptionless.Core/Models/UsageInfo.cs | Adds Deleted to both usage records. |
| src/Exceptionless.Core/Repositories/Configuration/Indexes/OrganizationIndex.cs | ES number mapping for Deleted on org usage. |
| src/Exceptionless.Core/Repositories/Configuration/Indexes/ProjectIndex.cs | ES number mapping for Deleted on project usage. |
| src/Exceptionless.Core/Utility/AppDiagnostics.cs | New ex.events.deleted OTel counter (Counter<long>). |
| src/Exceptionless.Core/Services/UsageService.cs | IncrementDeletedAsync, Deleted bucket cache key, flush/read inclusion, gate LastEventDateUtc on ingestion. |
| src/Exceptionless.Core/Jobs/CleanupDataJob.cs | Track deletes in RemoveProjectsAsync; add trackDeletedUsage flag and per-(org,project) splitting in RemoveStacksAsync. |
| src/Exceptionless.Core/Jobs/WorkItemHandlers/ResetProjectDataWorkItemHandler.cs | Track deletes after RemoveAllByProjectIdAsync. |
| src/Exceptionless.Web/Controllers/EventController.cs | Call IncrementDeletedAsync per org/project group after deletes, with try/catch. |
| src/Exceptionless.Web/Controllers/OrganizationController.cs | Project Deleted into the real-time usage view model. |
| src/Exceptionless.Web/Controllers/ProjectController.cs | Project Deleted into the real-time usage view model. |
| src/Exceptionless.Web/ClientApp/src/lib/generated/api.ts | Add deleted: number to UsageInfo/UsageHourInfo. |
| src/Exceptionless.Web/ClientApp/src/lib/generated/schemas.ts | Add deleted: int32() to generated schemas. |
| src/Exceptionless.Web/ClientApp/src/routes/(app)/organization/[organizationId]/usage/+page.svelte | Add Deleted series; remove dead org alias; include in yDomain. |
| src/Exceptionless.Web/ClientApp/src/routes/(app)/project/[projectId]/usage/+page.svelte | Add Deleted series to the project usage chart. |
| src/Exceptionless.Web/ClientApp/src/app.css | New --chart-7 color for the Deleted series. |
| src/Exceptionless.Web/ClientApp/src/lib/features/auth/exceptionless-session.ts | New session/identity/feature-usage helpers (unrelated to Deleted feature). |
| src/Exceptionless.Web/ClientApp/src/lib/features/auth/api.svelte.ts | Call endSession() on logout (unrelated). |
| src/Exceptionless.Web/ClientApp/src/lib/features/users/api.svelte.ts | Call setUserIdentity in getMeQuery onSuccess (unrelated). |
| src/Exceptionless.Web/ClientApp/src/lib/features/billing/components/change-plan-dialog.svelte | Emit submitFeatureUsage('organization.ChangePlan') (unrelated). |
| src/Exceptionless.Web/ClientApp.angular/app/organization/manage/manage-controller.js | Add Deleted series to legacy Angular chart. |
| src/Exceptionless.Web/ClientApp.angular/app/project/manage/manage-controller.js | Add Deleted series and fix existing project-vs-org indexing. |
| tests/Exceptionless.Tests/Controllers/Data/openapi.json | Update openapi snapshot with new deleted properties. |
| tests/Exceptionless.Tests/Controllers/EventControllerTests.cs | Integration tests for single/multi-event delete tracking. |
| tests/Exceptionless.Tests/Jobs/CleanupDataJobTests.cs | Tests for soft-delete tracking, multi-project distribution, retention no-op. |
| tests/Exceptionless.Tests/Jobs/WorkItemHandlers/ResetProjectDataWorkItemHandlerTests.cs | New file: reset-project deleted-usage tests. |
| tests/Exceptionless.Tests/Services/UsageServiceTests.cs | New IncrementDeletedAsync tests; assert Deleted == 0 in existing flows. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
niemyjski
commented
May 30, 2026
niemyjski
commented
May 30, 2026
niemyjski
commented
May 30, 2026
niemyjski
commented
May 30, 2026
niemyjski
commented
May 30, 2026
- Remove .catch(() => {}) hacks from frontend session/telemetry calls;
await them directly as they will never throw
- Add {Message} to EventController deleted-usage warning log template
- Fix RemoveStacksAsync perf regression: use single batch delete when
trackDeletedUsage is false (retention path), only split per-project
when actually tracking
- Change EventsDeleted OTel counter from Counter<long> to Counter<int>
for consistency with all other counters (int is acceptable ceiling)
- Make onSuccess callback async in getMeQuery to fix await in non-async
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Rethrow OperationCanceledException in EventController deleted-usage catch block so request cancellation is never swallowed - Correct PR description: int not long, exact per-project counts (no proportional distribution), document LastEventDateUtc semantic change Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Until now there was no way to see how many events were explicitly deleted via the API, project reset, or soft-delete cleanup. This makes it impossible to distinguish "events never arrived" from "events were intentionally removed" when analyzing usage. This PR adds a first-class
Deletedcounter alongside the existingTotal,Blocked,Discarded, andTooBigcounters -- without touching billing, plan limits, or admission control.What changed
Backend
UsageInfo/UsageHourInfo-- addedint Deletedproperty to both records, matching the type of all other usage counters (Total,Blocked,Discarded,TooBig).AppDiagnostics-- addedEventsDeletedas aCounter<int>OTel metric (consistent with all other event counters).UsageService-- newIncrementDeletedAsync(organizationId, projectId, eventCount)that pipelines all Redis operations viaTask.WhenAll; cache key pattern:usage:{bucket}:{orgId}[:projectId]:deleted. Save/flush/read logic updated to include the Deleted bucket (mirrors the other counters exactly).LastEventDateUtcbehavior change --LastEventDateUtcon org and project is now only updated when a bucket contains actual ingestion events (Total/Blocked/Discarded/TooBig> 0). Previously any flush (including delete-only buckets) could bump this timestamp. Deletes no longer count as "activity" for inactivity tracking.EventController--DeleteModelsAsynccallsIncrementDeletedAsyncper org/project group after the hard delete completes, inside a try/catch that rethrowsOperationCanceledExceptionand warns on all other failures so a tracking error never blocks the delete response.ResetProjectDataWorkItemHandler-- callsIncrementDeletedAsyncwith the count returned by the delete query.CleanupDataJob--RemoveProjectsAsynctracks deleted events;RemoveStacksAsyncaccepts atrackDeletedUsageflag. Whenfalse(retention enforcement), a single bulkRemoveAllByStackIdsAsyncis used. Whentrue(soft-delete cleanup), the stacks are grouped by(OrgId, ProjectId)and deleted per group so per-project counts can be attributed.RemoveOrganizationAsyncintentionally does NOT track -- the org is being hard-deleted so the data has no consumer.Frontend (Svelte 5)
api.ts,schemas.ts) updated to includedeleted: number.Deletedseries added to charts, dead variable alias removed.Frontend (Angular legacy)
Deletedseries added at the correct index.Test coverage
UsageServiceTestsEventControllerTestsCleanupDataJobTestsResetProjectDataWorkItemHandlerTestsExisting tests updated to assert
Deleted == 0where appropriate.Key design decisions
RemoveStacksAsyncis called withtrackDeletedUsage=falsefor retention enforcement paths. Only explicit user-initiated deletes and soft-delete cleanup show up asDeletedusage.intnotlong. Matches all other usage counters. A single cleanup pass removing >2.1B events per project is not a realistic scenario; theeventCount <= 0guard inIncrementDeletedAsyncsafely drops any overflow.Deletedis a display-only counter. It does not affectGetEventsLeftAsync, plan enforcement, or Stripe metering.