-
Notifications
You must be signed in to change notification settings - Fork 0
feat(cli): pluggable image storage with Vercel Blob impl (PR 1/3) #52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| name: Storage Integration | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: [main] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
| # Secrets aren't available on PRs from forks, so this workflow only runs | ||
| # on same-repo branches and pushes to main. The test itself requires the | ||
| # env vars to be set — no skipIf fallback. | ||
| if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }} | ||
| steps: | ||
| - uses: actions/checkout@v6.0.2 | ||
| - uses: actions/setup-node@v6.4.0 | ||
| with: | ||
| node-version: 22 | ||
| cache: yarn | ||
| - run: yarn install --frozen-lockfile | ||
| - run: yarn workspace @gitpulse/cli test:integration | ||
| env: | ||
| BLOB_READ_WRITE_TOKEN: ${{ secrets.BLOB_READ_WRITE_TOKEN }} | ||
| # Store ID is public (appears in every blob URL) — not a secret. | ||
| GITPULSE_TEST_STORE_ID: store_ryXhp9N7MsNEbMh1 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| import type { ImageStorage, StorageConfig } from './types.ts'; | ||
| import { VercelBlobStorage } from './vercel-blob.ts'; | ||
|
|
||
| export type { ImageStorage, StorageConfig, StorageProvider } from './types.ts'; | ||
| export { VercelBlobStorage } from './vercel-blob.ts'; | ||
|
|
||
| export function createStorage(config: StorageConfig): ImageStorage { | ||
| switch (config.provider) { | ||
| case 'vercel-blob': | ||
| return new VercelBlobStorage({ storeId: config.storeId }); | ||
| case 'r2': | ||
| case 's3': | ||
| case 'supabase': | ||
| throw new Error( | ||
| `Storage provider "${config.provider}" is not yet implemented`, | ||
| ); | ||
| default: { | ||
| const _exhaustive: never = config; | ||
| throw new Error(`Unknown storage provider: ${JSON.stringify(_exhaustive)}`); | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| // Pluggable storage backend for binary assets (today: AI-generated images, later: | ||
| // any other artifact gitpulse needs to persist). The interface is deliberately | ||
| // thin — upload, resolve to a public URL, list, batch-delete. All four methods | ||
| // have direct equivalents on Vercel Blob, Cloudflare R2, AWS S3, and Supabase | ||
| // Storage, so adding providers later is purely additive. | ||
| export interface ImageStorage { | ||
| upload(key: string, body: Buffer, contentType: string): Promise<void>; | ||
| urlFor(key: string): string; | ||
| list(prefix: string): Promise<string[]>; | ||
| delete(keys: string[]): Promise<void>; | ||
| } | ||
|
|
||
| // Discriminated union so future providers can land without breaking | ||
| // .gitpulse.json schema. Only 'vercel-blob' is implemented in PR 1; the others | ||
| // are reserved here so the zod schema in project-config.ts stays stable. | ||
| export type StorageConfig = | ||
| | { provider: 'vercel-blob'; storeId: string } | ||
| | { provider: 'r2'; accountId: string; bucket: string; publicBaseUrl: string } | ||
| | { provider: 's3'; region: string; bucket: string; publicBaseUrl?: string } | ||
| | { provider: 'supabase'; projectUrl: string; bucket: string }; | ||
|
|
||
| export type StorageProvider = StorageConfig['provider']; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| import { describe, it, expect, afterEach } from 'vitest'; | ||
| import { randomUUID } from 'node:crypto'; | ||
| import { VercelBlobStorage } from './vercel-blob.ts'; | ||
|
|
||
| // Runs a real round-trip against the Vercel Blob store identified by | ||
| // GITPULSE_TEST_STORE_ID. Both env vars are required — the test fails loudly | ||
| // rather than silently skipping if they're missing. Workflow gates ensure | ||
| // this only runs in environments that have access to the secret. | ||
| const STORE_ID = requireEnv('GITPULSE_TEST_STORE_ID'); | ||
| requireEnv('BLOB_READ_WRITE_TOKEN'); | ||
|
|
||
| describe('VercelBlobStorage integration', () => { | ||
| const createdKeys: string[] = []; | ||
|
|
||
| afterEach(async () => { | ||
| if (createdKeys.length === 0) return; | ||
| const storage = new VercelBlobStorage({ storeId: STORE_ID }); | ||
| await storage.delete(createdKeys.splice(0)); | ||
| }); | ||
|
|
||
| it('uploads, fetches, lists, and deletes a blob', async () => { | ||
| const storage = new VercelBlobStorage({ storeId: STORE_ID }); | ||
| const key = `__integration-test__/${Date.now()}-${randomUUID()}.txt`; | ||
| const bodyText = `hello from gitpulse integration test ${randomUUID()}\n`; | ||
| createdKeys.push(key); | ||
|
|
||
| await storage.upload(key, Buffer.from(bodyText), 'text/plain'); | ||
|
|
||
| const url = storage.urlFor(key); | ||
| const fetched = await fetch(url); | ||
| expect(fetched.status).toBe(200); | ||
| expect(await fetched.text()).toBe(bodyText); | ||
|
|
||
| const keys = await storage.list('__integration-test__/'); | ||
| expect(keys).toContain(key); | ||
|
|
||
| await storage.delete([key]); | ||
| createdKeys.length = 0; | ||
|
|
||
| // Vercel Blob is eventually consistent at the CDN (up to ~60s propagation | ||
| // per docs). Retry briefly so CI doesn't flake on the post-delete check. | ||
| const status = await waitFor404(url, 10_000); | ||
| expect(status).toBe(404); | ||
| }); | ||
| }); | ||
|
|
||
| async function waitFor404(url: string, timeoutMs: number): Promise<number> { | ||
| const deadline = Date.now() + timeoutMs; | ||
| let status = 0; | ||
| while (Date.now() < deadline) { | ||
| const res = await fetch(url, { cache: 'no-store' }); | ||
| status = res.status; | ||
| if (status === 404) return status; | ||
| await new Promise((r) => setTimeout(r, 250)); | ||
| } | ||
| return status; | ||
| } | ||
|
|
||
| function requireEnv(name: string): string { | ||
| const value = process.env[name]; | ||
| if (!value) { | ||
| throw new Error( | ||
| `${name} is required to run the storage integration test`, | ||
| ); | ||
| } | ||
| return value; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| import { describe, it, expect, vi, beforeEach } from 'vitest'; | ||
| import { put, list as blobList, del } from '@vercel/blob'; | ||
| import { VercelBlobStorage, storeIdToHost } from './vercel-blob.ts'; | ||
|
|
||
| vi.mock('@vercel/blob', () => ({ | ||
| put: vi.fn(async () => ({ url: 'mocked' })), | ||
| list: vi.fn(), | ||
| del: vi.fn(async () => undefined), | ||
| })); | ||
|
|
||
| const TOKEN = 'vercel_blob_rw_FAKE_TOKEN_FOR_TESTS'; | ||
| const STORE_ID = 'store_AbCdEf123'; | ||
|
|
||
| describe('storeIdToHost', () => { | ||
| it('strips store_ prefix and lowercases', () => { | ||
| expect(storeIdToHost('store_AbCdEf123')).toBe( | ||
| 'abcdef123.public.blob.vercel-storage.com', | ||
| ); | ||
| }); | ||
|
|
||
| it('handles already-lowercase ids', () => { | ||
| expect(storeIdToHost('store_abc')).toBe('abc.public.blob.vercel-storage.com'); | ||
| }); | ||
|
|
||
| it('handles ids without store_ prefix (defensive)', () => { | ||
| expect(storeIdToHost('XyZ')).toBe('xyz.public.blob.vercel-storage.com'); | ||
| }); | ||
|
|
||
| it('trims whitespace', () => { | ||
| expect(storeIdToHost(' store_abc ')).toBe( | ||
| 'abc.public.blob.vercel-storage.com', | ||
| ); | ||
| }); | ||
|
|
||
| it('throws on empty or whitespace-only input', () => { | ||
| expect(() => storeIdToHost('')).toThrow(/storeId is required/); | ||
| expect(() => storeIdToHost(' ')).toThrow(/storeId is required/); | ||
| }); | ||
|
|
||
| it('throws when the prefix is the entire value', () => { | ||
| expect(() => storeIdToHost('store_')).toThrow(/Invalid storeId/); | ||
| }); | ||
| }); | ||
|
|
||
| describe('VercelBlobStorage', () => { | ||
| beforeEach(() => { | ||
| vi.mocked(put).mockClear(); | ||
| vi.mocked(blobList).mockReset(); | ||
| vi.mocked(del).mockClear(); | ||
| }); | ||
|
|
||
| it('throws if BLOB_READ_WRITE_TOKEN is missing', () => { | ||
| expect(() => new VercelBlobStorage({ storeId: STORE_ID }, {})).toThrow( | ||
| /BLOB_READ_WRITE_TOKEN/, | ||
| ); | ||
| }); | ||
|
|
||
| it('upload calls put with public access, exact options, and the token', async () => { | ||
| const storage = new VercelBlobStorage( | ||
| { storeId: STORE_ID }, | ||
| { BLOB_READ_WRITE_TOKEN: TOKEN }, | ||
| ); | ||
| const body = Buffer.from('hello'); | ||
| await storage.upload('a/b/c.webp', body, 'image/webp'); | ||
| expect(put).toHaveBeenCalledExactlyOnceWith('a/b/c.webp', body, { | ||
| access: 'public', | ||
| contentType: 'image/webp', | ||
| allowOverwrite: true, | ||
| addRandomSuffix: false, | ||
| token: TOKEN, | ||
| }); | ||
| }); | ||
|
|
||
| it('urlFor produces the deterministic public URL', () => { | ||
| const storage = new VercelBlobStorage( | ||
| { storeId: STORE_ID }, | ||
| { BLOB_READ_WRITE_TOKEN: TOKEN }, | ||
| ); | ||
| expect(storage.urlFor('foo/bar.webp')).toBe( | ||
| 'https://abcdef123.public.blob.vercel-storage.com/foo/bar.webp', | ||
| ); | ||
| }); | ||
|
|
||
| it('urlFor encodes special characters in path segments but preserves slashes', () => { | ||
| const storage = new VercelBlobStorage( | ||
| { storeId: STORE_ID }, | ||
| { BLOB_READ_WRITE_TOKEN: TOKEN }, | ||
| ); | ||
| // Branch names in PR 2 keys can include spaces, ?, #, etc. Each segment | ||
| // gets percent-encoded; slashes stay as path separators. | ||
| expect(storage.urlFor('feature/foo bar/#hash?.webp')).toBe( | ||
| 'https://abcdef123.public.blob.vercel-storage.com/feature/foo%20bar/%23hash%3F.webp', | ||
| ); | ||
| }); | ||
|
|
||
| it('list returns pathnames and follows cursors across pages', async () => { | ||
| vi.mocked(blobList) | ||
| .mockResolvedValueOnce({ | ||
| blobs: [ | ||
| { pathname: 'pfx/one.txt' }, | ||
| { pathname: 'pfx/two.txt' }, | ||
| ], | ||
| cursor: 'next-page', | ||
| } as never) | ||
| .mockResolvedValueOnce({ | ||
| blobs: [{ pathname: 'pfx/three.txt' }], | ||
| cursor: undefined, | ||
| } as never); | ||
|
|
||
| const storage = new VercelBlobStorage( | ||
| { storeId: STORE_ID }, | ||
| { BLOB_READ_WRITE_TOKEN: TOKEN }, | ||
| ); | ||
| const keys = await storage.list('pfx/'); | ||
|
|
||
| expect(keys).toEqual(['pfx/one.txt', 'pfx/two.txt', 'pfx/three.txt']); | ||
| expect(blobList).toHaveBeenCalledTimes(2); | ||
| expect(blobList).toHaveBeenNthCalledWith(1, { | ||
| prefix: 'pfx/', | ||
| cursor: undefined, | ||
| token: TOKEN, | ||
| }); | ||
| expect(blobList).toHaveBeenNthCalledWith(2, { | ||
| prefix: 'pfx/', | ||
| cursor: 'next-page', | ||
| token: TOKEN, | ||
| }); | ||
| }); | ||
|
|
||
| it('delete maps keys through urlFor before calling del', async () => { | ||
| const storage = new VercelBlobStorage( | ||
| { storeId: STORE_ID }, | ||
| { BLOB_READ_WRITE_TOKEN: TOKEN }, | ||
| ); | ||
| await storage.delete(['a/b.txt', 'c/d.webp']); | ||
| expect(del).toHaveBeenCalledExactlyOnceWith( | ||
| [ | ||
| 'https://abcdef123.public.blob.vercel-storage.com/a/b.txt', | ||
| 'https://abcdef123.public.blob.vercel-storage.com/c/d.webp', | ||
| ], | ||
| { token: TOKEN }, | ||
| ); | ||
| }); | ||
|
|
||
| it('delete is a no-op when given an empty array', async () => { | ||
| const storage = new VercelBlobStorage( | ||
| { storeId: STORE_ID }, | ||
| { BLOB_READ_WRITE_TOKEN: TOKEN }, | ||
| ); | ||
| await storage.delete([]); | ||
| expect(del).not.toHaveBeenCalled(); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| import { put, list as blobList, del } from '@vercel/blob'; | ||
| import type { ImageStorage } from './types.ts'; | ||
|
|
||
| export interface VercelBlobStorageOptions { | ||
| storeId: string; | ||
| } | ||
|
|
||
| export class VercelBlobStorage implements ImageStorage { | ||
| private readonly token: string; | ||
| private readonly host: string; | ||
|
|
||
| constructor(opts: VercelBlobStorageOptions, env: NodeJS.ProcessEnv = process.env) { | ||
| const token = env.BLOB_READ_WRITE_TOKEN; | ||
| if (!token) { | ||
| throw new Error( | ||
| 'BLOB_READ_WRITE_TOKEN env var is required for vercel-blob storage', | ||
| ); | ||
| } | ||
| this.token = token; | ||
| this.host = storeIdToHost(opts.storeId); | ||
|
greptile-apps[bot] marked this conversation as resolved.
|
||
| } | ||
|
|
||
| async upload(key: string, body: Buffer, contentType: string): Promise<void> { | ||
| await put(key, body, { | ||
| access: 'public', | ||
| contentType, | ||
| allowOverwrite: true, | ||
| addRandomSuffix: false, | ||
| token: this.token, | ||
| }); | ||
| } | ||
|
|
||
| urlFor(key: string): string { | ||
| return `https://${this.host}/${encodePath(key)}`; | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| async list(prefix: string): Promise<string[]> { | ||
| const keys: string[] = []; | ||
| let cursor: string | undefined; | ||
| do { | ||
| const page = await blobList({ prefix, cursor, token: this.token }); | ||
| for (const blob of page.blobs) keys.push(blob.pathname); | ||
| cursor = page.cursor; | ||
| } while (cursor); | ||
| return keys; | ||
| } | ||
|
|
||
| async delete(keys: string[]): Promise<void> { | ||
| if (keys.length === 0) return; | ||
| const urls = keys.map((k) => this.urlFor(k)); | ||
| await del(urls, { token: this.token }); | ||
| } | ||
| } | ||
|
|
||
| // Vercel Blob public URLs follow the pattern | ||
| // https://<lowercase-storeId-without-prefix>.public.blob.vercel-storage.com/<key> | ||
| // confirmed by derisking against the real API. | ||
| export function storeIdToHost(storeId: string): string { | ||
| const trimmed = storeId.trim(); | ||
| if (!trimmed) throw new Error('storeId is required'); | ||
| const slug = trimmed.replace(/^store_/, '').toLowerCase(); | ||
| if (!slug) throw new Error(`Invalid storeId: ${storeId}`); | ||
| return `${slug}.public.blob.vercel-storage.com`; | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| // Encode a slash-delimited key as URL path segments, preserving the slashes | ||
| // as path separators. Keys can include branch names or other arbitrary | ||
| // strings in PR 2/3, so we must handle spaces, ?, #, and unicode safely. | ||
| function encodePath(key: string): string { | ||
| return key.split('/').map(encodeURIComponent).join('/'); | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Add a job timeout for this network integration workflow.
Line 13 defines a network-dependent job without
timeout-minutes; a stuck call can tie up runners unnecessarily.Proposed patch
test: runs-on: ubuntu-latest + timeout-minutes: 15📝 Committable suggestion
🤖 Prompt for AI Agents