ci(api): Create a list of valid api urls

[ryan953]

This script is wired into pre-commit, so that whenever a urls.py file changes in sentry we re-build the list in typescript.
For getsentry, because it's a separate repo, we can't have the same automation to keep the files in sync :(. So that list is broken out into it's own file and will need to manually be managed for now.

Pre commit

I triggered the pre-commit step on any urls.py file becaues we have our main src/sentry/api/urls.py file, but also a few places that contribute imports like src/sentry/workflow_engine/endpoints/urls.py and src/sentry/notifications/platform/api/endpoints/urls.py. I'd rather not have these extra urls.py file at all, and have all api routes be defined in one py file. But even if i go fix it now, someone is bound to add an import to import their route definitions again in the future.

Ok, so we have a pre-commit. It will generate/update the ts file.

CI tasks

Changing the TS file does not trigger a full eslint/jest run in the frontend CI. Also it does not add the "Scope: Frontend" label to the PR.
Only the TypeScript command is executed when the generated file changes. This will catch cases where someone trys to remove an api that is still in use (assuming all api call sites are statically typed, which they are not yet). Adding a new api endpoint in python will trigger TS, but shouldn't ever fail.

If someone manages to add a new endpoint and the ts file doesn't re-generate then it won't be the end of the world. In the short term we'll just re-gen the file.

Testing the Script

There's about ~800 routes that the script identified. I got an llm to generate all the regexp that splits url patterns into N strings. It appears to be handling cases where a url segment is named, like /(?P<issue_id>[^/]+)/ or has alternates like /(issue|group)/.
With these examples working i didn't look through all 800 urls to validate them, I also didn't look for anything that's missing. As we statically type all our frontend code using this list, if something is wrong we can revisit the script and fix it up.

There are however tests (llm generated) that are passing.

Running the script

The generated file has the command embedded inside it, and we can also see from the pre-commit-config: python3 -m tools.api_urls_to_typescript will do it.

It can also be run with a flag:

• python3 -m tools.api_urls_to_typescript list -> output to the terminal a list of urls, no formatting

Related to #97155

---

Note

Add Python tool to generate TS typings for API URLs from Django URL patterns, commit the generated file, add tests, and wire into pre-commit and CI filters.

• Tools / Codegen:
  • Add tools/api_urls_to_typescript.py to derive routes from Django urlpatterns and emit static/app/api/urlpatterns.generated.ts (with KnownApiUrls and MaybeApiPath).
  • Supports listing routes via python3 -m tools.api_urls_to_typescript list.
• Generated Artifacts:
  • Add static/app/api/urlpatterns.generated.ts containing the enumerated API paths.
• Tests:
  • Add tests/tools/test_api_urls_to_typescript.py covering regexp_to_routes (named groups, alternates, optional parts, complex patterns).
• Pre-commit:
  • Add gen-ts-api-urls hook to regenerate TS routes on urls.py changes.
• CI / Filters:
  • Update .github/file-filters.yml to broaden TS/JS globs and exclude *.generated.ts; add frontend workflow anchor and include in triggers.
  • Update frontend.yml job outputs to expose typecheckable_rules_changed.

^{Written by Cursor Bugbot for commit 82c5d6a. This will update automatically on new commits. Configure here.}

[github-actions]

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

[ryan953]

The urls in this list have $ as the variable prefix, which aligns with getApiUrl()

But our routes.tsx file uses :. I think it'd be better switch the whole system over to that prefix, so routes and the static typing align.

[scttcper]

would it make sense to update the makefile or package.json scripts to make it easier to run on demand

[TkDodo]

This is great work. I’ll close my PR in favour of this one

[TkDodo]

is this file prettier/eslint ignored or do we need annotations at the top?

[ryan953]

It shouldn't trigger any rule violations.

This file is skipped from triggering CI steps, but not omitted from the commands when they do run. I was planning to leave it that way, fewer exceptions seems easier to deal with.

[TkDodo]

okay, just wanted to avoid opening it in an IDE and auto-save would maybe reformat it for some reason

[TkDodo]

I think we should export KnownApiUrls from here only and do the rest in apiDefinition:

import type {KnownApiUrls} from './urlpatterns.generated';

export type ApiPath = KnownApiUrls;

we should also be able to remove the | (string & {}) to get real type-checking instead of type hints. I only did this so that we can work with it even though we don’t have all urls yet. Can be done in a follow-up though.

[TkDodo]

Just a thought: I was wondering if maybe we should have a check that catches if someone was manually altering that generated file? Not sure if that is really a concern, but we could either flag it in CI when the generated file changes, but urls.py does not, or we hash the file, re-generate it in CI and check the hashes to see if what we have checked-in is the same as what gets generated. If we don’t think people will touch that generated file we likely don’t need this though.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #100515      +/-   ##
===========================================
+ Coverage   81.15%    81.21%   +0.06%     
===========================================
  Files        8624      8642      +18     
  Lines      382465    383932    +1467     
  Branches    24036     24036              
===========================================
+ Hits       310394    311820    +1426     
- Misses      71739     71780      +41     
  Partials      332       332              

[ryan953]

renamed, it's only used within the file

[ryan953]

inlined into the one callsite.

[ryan953]

had to update all the tests so they're using real routes

i tried to pick stuff that's near the top of the list, but had the properties the test cares about.

[TkDodo]

the /projects one also exists, just the variables are named differently:

'/projects/$organizationIdOrSlug/$projectIdOrSlug/releases/$version/'

[scttcper]

nice

[getsentry-bot]

revert failed (conflict? already reverted?) -- check the logs