Skip to content

refactor: consume common and ai-gateway protos from BSR packages#2324

Merged
c-julin merged 3 commits intomasterfrom
refactor/consume-common-ai-gateway-from-bsr
Mar 18, 2026
Merged

refactor: consume common and ai-gateway protos from BSR packages#2324
c-julin merged 3 commits intomasterfrom
refactor/consume-common-ai-gateway-from-bsr

Conversation

@c-julin
Copy link
Contributor

@c-julin c-julin commented Mar 18, 2026

Summary

  • Remove buf.build/redpandadata/common and buf.build/redpandadata/ai-gateway as explicit inputs from buf.gen.backend.yaml and buf.gen.frontend.yaml
  • Backend: common types consumed as pre-built Go packages from the BSR SDK, matching the cloudv2 pattern
  • Frontend: common and ai-gateway types consumed from BSR npm packages (@buf/redpandadata_common.bufbuild_es, @buf/redpandadata_ai-gateway.bufbuild_es, @buf/redpandadata_ai-gateway.connectrpc_query-es)
  • Eliminates the policymaterializer codegen issue and removes ~26k lines of locally generated code that was either dead or duplicated from BSR

c-julin added 2 commits March 18, 2026 13:42
Remove buf.build/redpandadata/common and buf.build/redpandadata/ai-gateway
as explicit inputs from buf.gen.backend.yaml and buf.gen.frontend.yaml.

Backend: common types are consumed as pre-built Go packages from the BSR
SDK (buf.build/gen/go/redpandadata/common/protocolbuffers/go), matching
the pattern used by cloudv2. This eliminates the policymaterializer
codegen issue entirely.

Frontend: common and ai-gateway types are consumed from BSR npm packages
(@buf/redpandadata_common.bufbuild_es, @buf/redpandadata_ai-gateway.bufbuild_es,
@buf/redpandadata_ai-gateway.connectrpc_query-es). Only console's own
dataplane protos are generated locally.
- Add check-bsr-versions task to proto:generate that verifies Go and npm
  BSR SDK package versions match the commits pinned in buf.lock
- Remove unused buf.build/redpandadata/ai-gateway dep from buf.yaml
- Pin Go BSR common dep to match buf.lock commit
@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

The latest Buf updates on your PR. Results from workflow Buf CI / validate (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMar 18, 2026, 3:42 PM

@c-julin c-julin force-pushed the refactor/consume-common-ai-gateway-from-bsr branch 2 times, most recently from 21cdee3 to ce6ff7a Compare March 18, 2026 14:35
buf.yaml Outdated
- buf.build/grpc-ecosystem/grpc-gateway:v2.24.0
- buf.build/redpandadata/common:5d899910f71449dda17179c2e198100d
- buf.build/redpandadata/core:05aa34b3829a4d5a801d9487623a7c76
- buf.build/redpandadata/core
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the version here - is this intended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh good catch it wasn't it was only meant to be temporary

Unpin buf.build/redpandadata/core to latest (923579c57f1c) which
includes admin/v2 types needed by monitoring and shadowlink gateway
code. Also commits all regenerated .gw.go files that were missing.
@c-julin c-julin force-pushed the refactor/consume-common-ai-gateway-from-bsr branch from ce6ff7a to d203bb7 Compare March 18, 2026 15:42
@c-julin c-julin merged commit 8210fb8 into master Mar 18, 2026
24 checks passed
@c-julin c-julin deleted the refactor/consume-common-ai-gateway-from-bsr branch March 18, 2026 15:58
weeco added a commit that referenced this pull request Mar 19, 2026
Remove buf.build/redpandadata/common and buf.build/redpandadata/ai-gateway
module inputs from buf.gen.frontend.yaml. These types are now consumed
from pre-built BSR npm packages per PR #2324, not generated locally.
weeco added a commit that referenced this pull request Mar 19, 2026
* proto: add transcript service and regenerate protobuf/openapi definitions

Adds the transcript proto definition for the dataplane v1alpha3 API
and regenerates all protobuf (backend Go, frontend TS) and OpenAPI outputs.

* chore(proto): pin ai-gateway version in buf.gen.frontend.yaml and regenerate

Pin ai-gateway to commit 0d2811b8 to prevent unrelated proto changes
from leaking into PRs that only need to regenerate dataplane protos.

* fix: revert buf.gen.frontend.yaml to match master's BSR package approach

Remove buf.build/redpandadata/common and buf.build/redpandadata/ai-gateway
module inputs from buf.gen.frontend.yaml. These types are now consumed
from pre-built BSR npm packages per PR #2324, not generated locally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants