Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions packager/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,142 @@ For more on go-bindata: https://github.com/jteeuwen/go-bindata
```
ginkgo -r
```

---

## Selective Dependency Packaging

`buildpack-packager` supports building **cached** buildpacks that contain only a named
subset of dependencies. This is useful for operators who need a smaller artifact or a
pre-defined variant (e.g. a "minimal" build without optional agent libraries).

> **Note:** `--profile`, `--exclude`, and `--include` are only valid for **cached**
> buildpacks (`--cached` flag). Using them on an uncached build is a hard error.

### Packaging profiles

A buildpack author defines named profiles in `manifest.yml` under the
`packaging_profiles` key:

```yaml
packaging_profiles:
minimal:
description: "Core runtime only — no agents or profilers"
exclude:
- agent-dep
- profiler-dep

no-profiler:
description: "Full build minus the profiler library"
exclude:
- profiler-dep
```

Each profile has:

| Field | Type | Description |
|---------------|----------------|-------------|
| `description` | string | Human-readable summary shown by `buildpack-packager summary` |
| `exclude` | list of string | Dependency names to omit when this profile is active |

Profile names must match `^[a-z0-9_-]+$` (lowercase letters, digits, hyphens,
underscores). This keeps names safe for embedding in zip filenames.

All dependency names listed in a profile's `exclude` list must exist in the
manifest — a typo is a hard error at packaging time.

### CLI flags

| Flag | Argument | Description |
|------|----------|-------------|
| `--profile` | profile name | Activate a named profile from `manifest.yml` |
| `--exclude` | `dep1,dep2,...` | Additional dependencies to exclude (comma-separated) |
| `--include` | `dep1,dep2,...` | Restore dependencies that the active profile excluded |

#### Examples

```sh
# Build with the "minimal" profile (omits agent-dep and profiler-dep)
buildpack-packager --cached --stack cflinuxfs4 --profile minimal

# Build with the "minimal" profile but restore profiler-dep
buildpack-packager --cached --stack cflinuxfs4 --profile minimal --include profiler-dep

# No profile — just exclude a specific dependency
buildpack-packager --cached --stack cflinuxfs4 --exclude agent-dep

# Combine a profile with an extra exclusion
buildpack-packager --cached --stack cflinuxfs4 --profile no-profiler --exclude agent-dep
```

### Resolution order

1. Profile's `exclude` list is applied first.
2. `--exclude` names are **unioned** with the profile exclusions.
3. `--include` names are **removed** from the exclusion set (overrides the profile).

Excluded dependencies are neither downloaded nor written into the packaged
`manifest.yml`.

### Output filename

The zip filename encodes which variant was built:

| Options used | Filename pattern |
|---|---|
| No opts | `<lang>_buildpack-cached-<stack>-v<ver>.zip` |
| `--profile minimal` | `<lang>_buildpack-cached-minimal-<stack>-v<ver>.zip` |
| `--profile minimal --include profiler-dep` | `<lang>_buildpack-cached-minimal+custom-<stack>-v<ver>.zip` |
| `--profile minimal --exclude extra-dep` | `<lang>_buildpack-cached-minimal+custom-<stack>-v<ver>.zip` |
| `--exclude agent-dep` (no profile) | `<lang>_buildpack-cached-custom-<stack>-v<ver>.zip` |

The `+custom` suffix appears only when the result deviates from a pure profile:
either an extra `--exclude` was added, or `--include` actually overrode one of
the profile's exclusions.

### Error conditions

All validation errors are **hard errors** — the packager exits non-zero with a
descriptive message. There are no silent no-ops or warnings.

| Situation | Error message |
|---|---|
| `--profile` / `--exclude` / `--include` on uncached build | `--profile/--exclude/--include are only valid for cached buildpacks` |
| `--include` without `--profile` | `--include requires --profile` |
| Unknown profile name | `packaging profile "<name>" not found in manifest` |
| Invalid profile name characters | `profile name "<name>" is invalid: must match ^[a-z0-9_-]+$` |
| Unknown dep in `--exclude` | `dependency "<name>" not found in manifest` |
| Unknown dep in `--include` | `dependency "<name>" not found in manifest` |
| `--include` of dep not excluded by profile | `--include "<name>" has no effect: dependency is not excluded by the profile or --exclude` |
| Profile's `exclude` list references unknown dep | `profile "<name>" references unknown dependency "<dep>"` |

### Go API

`Package()` is unchanged and delegates to `PackageWithOptions` with zero options:

```go
// Legacy — unchanged behaviour
zipFile, err := packager.Package(bpDir, cacheDir, version, stack, cached)

// New — selective packaging
zipFile, err := packager.PackageWithOptions(bpDir, cacheDir, version, stack, true,
packager.PackageOptions{
Profile: "minimal",
Include: []string{"profiler-dep"},
})
```

`PackageOptions` fields:

```go
type PackageOptions struct {
// Profile is a packaging_profiles key from manifest.yml.
Profile string
// Exclude lists additional dependency names to skip.
Exclude []string
// Include restores dependency names excluded by Profile.
// Requires Profile to be set. Hard error if a name was not excluded.
Include []string
}
```

42 changes: 39 additions & 3 deletions packager/buildpack-packager/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,23 +40,42 @@ type buildCmd struct {
version string
cacheDir string
stack string
profile string
exclude string
include string
}

func (*buildCmd) Name() string { return "build" }
func (*buildCmd) Synopsis() string { return "Create a buildpack zipfile from the current directory" }
func (*buildCmd) Usage() string {
return `build -stack <stack>|-any-stack [-cached] [-version <version>] [-cachedir <path to cachedir>]:
return `build -stack <stack>|-any-stack [-cached] [-version <version>] [-cachedir <path>]
[-profile <profile>] [-exclude <dep1,dep2,...>] [-include <dep1,dep2,...>]:
When run in a directory that is structured as a buildpack, creates a zip file.

-profile Name of a packaging profile defined in manifest.yml's
packaging_profiles section. Profiles declare which dependencies
to exclude from the cached zip.

-exclude Comma-separated list of dependency names to exclude, in addition
to any exclusions implied by -profile. Names must exist in
manifest.yml. Example: -exclude datadog-javaagent,newrelic

-include Comma-separated list of dependency names to force-include,
overriding exclusions implied by -profile. Useful for starting
from a restrictive profile and adding back a single dep.
Example: -profile minimal -include jprofiler-profiler

`
}
func (b *buildCmd) SetFlags(f *flag.FlagSet) {
f.StringVar(&b.version, "version", "", "version to build as")
f.BoolVar(&b.cached, "cached", false, "include dependencies")
f.StringVar(&b.cacheDir, "cachedir", packager.CacheDir, "cache dir")

f.StringVar(&b.stack, "stack", "", "stack to package buildpack for")
f.BoolVar(&b.anyStack, "any-stack", false, "package buildpack for any stack")
f.StringVar(&b.profile, "profile", "", "packaging profile defined in manifest.yml")
f.StringVar(&b.exclude, "exclude", "", "comma-separated dependency names to exclude")
f.StringVar(&b.include, "include", "", "comma-separated dependency names to include, overriding profile exclusions")
}
func (b *buildCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
if b.stack == "" && !b.anyStack {
Expand All @@ -76,7 +95,24 @@ func (b *buildCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{})
b.version = strings.TrimSpace(string(v))
}

zipFile, err := packager.Package(".", b.cacheDir, b.version, b.stack, b.cached)
parseCSV := func(s string) []string {
var out []string
for _, name := range strings.Split(s, ",") {
name = strings.TrimSpace(name)
if name != "" {
out = append(out, name)
}
}
return out
}

opts := packager.PackageOptions{
Profile: b.profile,
Exclude: parseCSV(b.exclude),
Include: parseCSV(b.include),
}

zipFile, err := packager.PackageWithOptions(".", b.cacheDir, b.version, b.stack, b.cached, opts)
if err != nil {
log.Printf("error while creating zipfile: %v", err)
return subcommands.ExitFailure
Expand Down
1 change: 1 addition & 0 deletions packager/fixtures/with_profiles/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.0.0
1 change: 1 addition & 0 deletions packager/fixtures/with_profiles/agent.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
agent-content
1 change: 1 addition & 0 deletions packager/fixtures/with_profiles/bin/filename
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
awesome content
1 change: 1 addition & 0 deletions packager/fixtures/with_profiles/core.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
core-content
38 changes: 38 additions & 0 deletions packager/fixtures/with_profiles/manifest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
language: ruby
default_versions:
- name: core-dep
version: 1.0.0
dependencies:
- name: core-dep
version: 1.0.0
sha256: 30d444543fbf15edc568d29705afe7ecbc813c4152b09f403f48393b0023c8a7
uri: file://PLACEHOLDER_CORE
cf_stacks:
- cflinuxfs4
- name: agent-dep
version: 2.0.0
sha256: 4376301a8bdefc71d4c686a919f9cae5232707c00a5949994b1faba7aefc3796
uri: file://PLACEHOLDER_AGENT
cf_stacks:
- cflinuxfs4
- name: profiler-dep
version: 3.0.0
sha256: b6d1482cd7b189071c3c48e936f04f7d0a6f7a0a75df6ecb6cb4b55fd398baa2
uri: file://PLACEHOLDER_PROFILER
cf_stacks:
- cflinuxfs4
include_files:
- manifest.yml
- VERSION
- bin/filename
packaging_profiles:
minimal:
description: Core deps only. No agents or profilers.
exclude:
- agent-dep
- profiler-dep
no-profiler:
description: Core and agents only. No profilers.
exclude:
- profiler-dep
1 change: 1 addition & 0 deletions packager/fixtures/with_profiles/profiler.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
profiler-content
7 changes: 7 additions & 0 deletions packager/models.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ type Dependency struct {
type SubDependency struct{ Name string }
type Dependencies []Dependency

// PackagingProfile defines a named dependency exclusion set for use at packaging time.
type PackagingProfile struct {
Description string `yaml:"description"`
Exclude []string `yaml:"exclude"`
}

type Manifest struct {
Language string `yaml:"language"`
Stack string `yaml:"stack"`
Expand All @@ -25,6 +31,7 @@ type Manifest struct {
Name string `yaml:"name"`
Version string `yaml:"version"`
} `yaml:"default_versions"`
PackagingProfiles map[string]PackagingProfile `yaml:"packaging_profiles"`
}

type File struct {
Expand Down
40 changes: 40 additions & 0 deletions packager/models_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
httpmock "github.com/jarcoal/httpmock"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
yaml "gopkg.in/yaml.v2"
)

var _ = Describe("Packager", func() {
Expand All @@ -33,4 +34,43 @@ var _ = Describe("Packager", func() {
}))
})
})

Describe("PackagingProfile YAML unmarshalling", func() {
It("parses packaging_profiles from manifest YAML", func() {
raw := `
language: ruby
dependencies: []
packaging_profiles:
minimal:
description: "JDKs only"
exclude:
- agent-dep
- profiler-dep
standard:
description: "Core and OSS agents"
exclude:
- profiler-dep
`
var m packager.Manifest
Expect(yaml.Unmarshal([]byte(raw), &m)).To(Succeed())
Expect(m.PackagingProfiles).To(HaveLen(2))

minimal := m.PackagingProfiles["minimal"]
Expect(minimal.Description).To(Equal("JDKs only"))
Expect(minimal.Exclude).To(ConsistOf("agent-dep", "profiler-dep"))

standard := m.PackagingProfiles["standard"]
Expect(standard.Description).To(Equal("Core and OSS agents"))
Expect(standard.Exclude).To(ConsistOf("profiler-dep"))
})

It("has nil PackagingProfiles when the section is absent", func() {
raw := `language: ruby
dependencies: []
`
var m packager.Manifest
Expect(yaml.Unmarshal([]byte(raw), &m)).To(Succeed())
Expect(m.PackagingProfiles).To(BeNil())
})
})
})
Loading
Loading