|
| 1 | +# Eszip Store Implementation Plan |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +Implement an abstracted storage layer for eszip bundles (and other EdgeFunction assets like .wasm and .so files) with two backends: |
| 6 | +- **File-backed store** - For local development and testing |
| 7 | +- **S3-backed store** - For production deployments |
| 8 | + |
| 9 | +## Current State |
| 10 | + |
| 11 | +EdgeFunction assets are stored locally at `$baseDir/run/ingest/store/{name}/` with explicit TODOs at `pkg/apiserver/ingest/edgefunction.go:405,634,764` indicating intent to migrate to object store. The current HTTP server (`ListenAndServeEdgeFuncs`) reads directly from the filesystem. |
| 12 | + |
| 13 | +## Proposed Architecture |
| 14 | + |
| 15 | +``` |
| 16 | +pkg/apiserver/ingest/store/ |
| 17 | +├── store.go # Interface + factory |
| 18 | +├── file.go # File-backed implementation |
| 19 | +├── s3.go # S3-backed implementation |
| 20 | +└── store_test.go # Tests |
| 21 | +``` |
| 22 | + |
| 23 | +## Interface Design |
| 24 | + |
| 25 | +```go |
| 26 | +// pkg/apiserver/ingest/store/store.go |
| 27 | + |
| 28 | +package store |
| 29 | + |
| 30 | +import ( |
| 31 | + "context" |
| 32 | + "io" |
| 33 | +) |
| 34 | + |
| 35 | +// AssetType identifies the type of EdgeFunction asset |
| 36 | +type AssetType string |
| 37 | + |
| 38 | +const ( |
| 39 | + AssetTypeEszip AssetType = "eszip" // JavaScript bundle |
| 40 | + AssetTypeWasm AssetType = "wasm" // WebAssembly module |
| 41 | + AssetTypeGo AssetType = "go" // Go plugin (.so) |
| 42 | +) |
| 43 | + |
| 44 | +// Store provides storage operations for EdgeFunction assets |
| 45 | +type Store interface { |
| 46 | + // Put stores an asset. The reader is consumed and closed by the implementation. |
| 47 | + Put(ctx context.Context, ref string, assetType AssetType, r io.Reader) error |
| 48 | + |
| 49 | + // Get retrieves an asset. Caller must close the returned ReadCloser. |
| 50 | + // Returns os.ErrNotExist if not found. |
| 51 | + Get(ctx context.Context, ref string, assetType AssetType) (io.ReadCloser, error) |
| 52 | + |
| 53 | + // Delete removes an asset. |
| 54 | + Delete(ctx context.Context, ref string, assetType AssetType) error |
| 55 | + |
| 56 | + // Exists checks if an asset exists. |
| 57 | + Exists(ctx context.Context, ref string, assetType AssetType) (bool, error) |
| 58 | +} |
| 59 | +``` |
| 60 | + |
| 61 | +## File Store Implementation |
| 62 | + |
| 63 | +```go |
| 64 | +// pkg/apiserver/ingest/store/file.go |
| 65 | + |
| 66 | +type FileStore struct { |
| 67 | + baseDir string |
| 68 | +} |
| 69 | + |
| 70 | +func NewFileStore(baseDir string) (*FileStore, error) { |
| 71 | + // Creates baseDir if it doesn't exist |
| 72 | +} |
| 73 | + |
| 74 | +// Key layout: {baseDir}/{ref}/{assetType} |
| 75 | +// e.g., /data/store/my-func-rev-abc123/eszip |
| 76 | +``` |
| 77 | + |
| 78 | +**Key behaviors:** |
| 79 | +- Atomic writes using temp file + rename (existing symlink pattern) |
| 80 | +- Direct file reads with `os.Open` |
| 81 | +- Compatible with existing HTTP serving (can mount same directory) |
| 82 | + |
| 83 | +## S3 Store Implementation |
| 84 | + |
| 85 | +```go |
| 86 | +// pkg/apiserver/ingest/store/s3.go |
| 87 | + |
| 88 | +type S3Store struct { |
| 89 | + client *s3.Client |
| 90 | + bucket string |
| 91 | + prefix string // optional key prefix |
| 92 | +} |
| 93 | + |
| 94 | +type S3Config struct { |
| 95 | + Region string |
| 96 | + Bucket string |
| 97 | + Prefix string |
| 98 | + Endpoint string // for MinIO/localstack compatibility |
| 99 | +} |
| 100 | + |
| 101 | +func NewS3Store(ctx context.Context, cfg S3Config) (*S3Store, error) { |
| 102 | + // Uses AWS SDK v2 with default credential chain |
| 103 | +} |
| 104 | + |
| 105 | +// Key layout: {prefix}/{ref}/{assetType} |
| 106 | +// e.g., s3://my-bucket/edgefuncs/my-func-rev-abc123/eszip |
| 107 | +``` |
| 108 | + |
| 109 | +**Key behaviors:** |
| 110 | +- Uses `s3.PutObject` with streaming upload |
| 111 | +- Uses `s3.GetObject` returning the response body as ReadCloser |
| 112 | +- Supports custom endpoints for MinIO/LocalStack testing |
| 113 | + |
| 114 | +## Configuration |
| 115 | + |
| 116 | +Add to existing config or environment: |
| 117 | + |
| 118 | +```go |
| 119 | +// pkg/apiserver/ingest/config.go or similar |
| 120 | + |
| 121 | +type StoreConfig struct { |
| 122 | + // Type selects the store backend: "file" or "s3" |
| 123 | + Type string `json:"type" yaml:"type"` |
| 124 | + |
| 125 | + // File store options (when Type = "file") |
| 126 | + File struct { |
| 127 | + BaseDir string `json:"baseDir" yaml:"baseDir"` |
| 128 | + } `json:"file" yaml:"file"` |
| 129 | + |
| 130 | + // S3 store options (when Type = "s3") |
| 131 | + S3 struct { |
| 132 | + Region string `json:"region" yaml:"region"` |
| 133 | + Bucket string `json:"bucket" yaml:"bucket"` |
| 134 | + Prefix string `json:"prefix" yaml:"prefix"` |
| 135 | + Endpoint string `json:"endpoint" yaml:"endpoint"` // optional |
| 136 | + } `json:"s3" yaml:"s3"` |
| 137 | +} |
| 138 | +``` |
| 139 | + |
| 140 | +## Integration Points |
| 141 | + |
| 142 | +### 1. Replace direct filesystem calls in `edgefunction.go` |
| 143 | + |
| 144 | +Current (line ~764): |
| 145 | +```go |
| 146 | +err = os.Rename(stagingPath, filepath.Join(storeDir, "bin.eszip")) |
| 147 | +``` |
| 148 | + |
| 149 | +New: |
| 150 | +```go |
| 151 | +f, err := os.Open(stagingPath) |
| 152 | +if err != nil { return err } |
| 153 | +defer f.Close() |
| 154 | +err = w.store.Put(ctx, name, store.AssetTypeEszip, f) |
| 155 | +``` |
| 156 | + |
| 157 | +### 2. Update HTTP handler (`ServeHTTP`) |
| 158 | + |
| 159 | +Current: |
| 160 | +```go |
| 161 | +p := filepath.Join(storeDir(name), filename) |
| 162 | +http.ServeFile(wr, req, p) |
| 163 | +``` |
| 164 | + |
| 165 | +New: |
| 166 | +```go |
| 167 | +rc, err := w.store.Get(req.Context(), name, assetType) |
| 168 | +if err != nil { |
| 169 | + if os.IsNotExist(err) { |
| 170 | + http.NotFound(wr, req) |
| 171 | + return |
| 172 | + } |
| 173 | + http.Error(wr, err.Error(), http.StatusInternalServerError) |
| 174 | + return |
| 175 | +} |
| 176 | +defer rc.Close() |
| 177 | +io.Copy(wr, rc) |
| 178 | +``` |
| 179 | + |
| 180 | +### 3. Cleanup in workflows |
| 181 | + |
| 182 | +Current: |
| 183 | +```go |
| 184 | +os.RemoveAll(storeDir(name)) |
| 185 | +``` |
| 186 | + |
| 187 | +New: |
| 188 | +```go |
| 189 | +w.store.Delete(ctx, name, store.AssetTypeEszip) |
| 190 | +// etc for other asset types |
| 191 | +``` |
| 192 | + |
| 193 | +## Implementation Steps |
| 194 | + |
| 195 | +1. **Create store package with interface** (`pkg/apiserver/ingest/store/store.go`) |
| 196 | + |
| 197 | +2. **Implement FileStore** (`file.go`) |
| 198 | + - Constructor with directory creation |
| 199 | + - Put with atomic write (temp + rename) |
| 200 | + - Get returning os.File |
| 201 | + - Delete and Exists |
| 202 | + |
| 203 | +3. **Implement S3Store** (`s3.go`) |
| 204 | + - Use AWS SDK v2 (`github.com/aws/aws-sdk-go-v2`) |
| 205 | + - Constructor with config loading |
| 206 | + - Put with streaming PutObject |
| 207 | + - Get returning GetObject response body |
| 208 | + - Delete and Exists (HeadObject) |
| 209 | + |
| 210 | +4. **Add factory function** (`store.go`) |
| 211 | + ```go |
| 212 | + func New(cfg StoreConfig) (Store, error) |
| 213 | + ``` |
| 214 | + |
| 215 | +5. **Write tests** (`store_test.go`) |
| 216 | + - Unit tests with FileStore |
| 217 | + - Integration test pattern for S3 (LocalStack or skip) |
| 218 | + |
| 219 | +6. **Integrate into worker** (`edgefunction.go`) |
| 220 | + - Add store field to worker struct |
| 221 | + - Update StoreEszipActivity |
| 222 | + - Update StoreWasmActivity |
| 223 | + - Update StoreGoActivity |
| 224 | + - Update ServeHTTP handler |
| 225 | + |
| 226 | +7. **Wire up configuration** |
| 227 | + - Add StoreConfig to worker options |
| 228 | + - Default to FileStore for backwards compatibility |
| 229 | + |
| 230 | +## Testing Strategy |
| 231 | + |
| 232 | +- **FileStore**: Standard unit tests with temp directories |
| 233 | +- **S3Store**: |
| 234 | + - Unit tests with mock S3 client interface |
| 235 | + - Optional integration tests with LocalStack (via `endpoint` config) |
| 236 | +- **Integration**: Existing EdgeFunction workflow tests should continue to pass |
| 237 | + |
| 238 | +## Dependencies to Add |
| 239 | + |
| 240 | +``` |
| 241 | +github.com/aws/aws-sdk-go-v2 |
| 242 | +github.com/aws/aws-sdk-go-v2/config |
| 243 | +github.com/aws/aws-sdk-go-v2/service/s3 |
| 244 | +``` |
| 245 | + |
| 246 | +## Backwards Compatibility |
| 247 | + |
| 248 | +- Default store type = "file" with existing baseDir location |
| 249 | +- Existing deployments continue to work without config changes |
| 250 | +- HTTP serving interface unchanged (backplane compatibility) |
| 251 | + |
| 252 | +## Future Considerations (Out of Scope) |
| 253 | + |
| 254 | +- Signed URL generation for direct S3 downloads (bypass apiserver) |
| 255 | +- Cache layer for frequently accessed assets |
| 256 | +- Multi-region replication |
| 257 | +- Compression/deduplication |
0 commit comments