Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add GitLab CI provenance #6375

Merged
merged 12 commits into from
May 12, 2023
Merged

feat: Add GitLab CI provenance #6375

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
030956b
libnpmpublish: Add GitLab CI provenance.
wlynch Apr 18, 2023
d22dcee
libnpmpublish: [gitlab] un-refactor generate provenance, add check for
wlynch Apr 20, 2023
0830de9
GitLab provenance: change version to v0alpha1, include links to GitLab
wlynch Apr 24, 2023
cabf14f
Remove fallthrough case in provenance.js.
wlynch Apr 24, 2023
19bbf76
Set GITHUB_ACTIONS environment variable to false in tests.
wlynch May 3, 2023
47c544d
Add additional pipeline data into environment.
wlynch May 3, 2023
2806562
libnpmpublish/gitlab: var -> let, revise error string.
wlynch May 4, 2023
3853fbc
Remove CI_REPOSITORY_URL, CI_BUILD_REF.
wlynch May 11, 2023
8b1257d
Remove GITLAB_USER_NAME, GITLAB_USER_EMAIL.
wlynch May 11, 2023
3bb9bfb
Remove CI_COMMIT_AUTHOR, CI_COMMIT_DESCRIPTION, CI_COMMIT_MESSAGE
wlynch May 12, 2023
cd8a88d
Remove commented lines from test.
wlynch May 12, 2023
fcf02f9
Use ci-info.name for provenance log statement.
wlynch May 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
239 changes: 190 additions & 49 deletions workspaces/libnpmpublish/lib/provenance.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,68 +1,209 @@
const { sigstore } = require('sigstore')
const ci = require('ci-info')
const { env } = process

const INTOTO_PAYLOAD_TYPE = 'application/vnd.in-toto+json'
const INTOTO_STATEMENT_TYPE = 'https://in-toto.io/Statement/v0.1'
const SLSA_PREDICATE_TYPE = 'https://slsa.dev/provenance/v0.2'

const BUILDER_ID = 'https://github.com/actions/runner'
const BUILD_TYPE_PREFIX = 'https://github.com/npm/cli/gha'
const BUILD_TYPE_VERSION = 'v2'
const GITHUB_BUILDER_ID = 'https://github.com/actions/runner'
const GITHUB_BUILD_TYPE_PREFIX = 'https://github.com/npm/cli/gha'
const GITHUB_BUILD_TYPE_VERSION = 'v2'

const GITLAB_BUILD_TYPE_PREFIX = 'https://github.com/npm/cli/gitlab'
const GITLAB_BUILD_TYPE_VERSION = 'v0alpha1'

const generateProvenance = async (subject, opts) => {
const { env } = process
/* istanbul ignore next - not covering missing env var case */
const [workflowPath] = (env.GITHUB_WORKFLOW_REF || '')
.replace(env.GITHUB_REPOSITORY + '/', '')
.split('@')
const payload = {
_type: INTOTO_STATEMENT_TYPE,
subject,
predicateType: SLSA_PREDICATE_TYPE,
predicate: {
buildType: `${BUILD_TYPE_PREFIX}/${BUILD_TYPE_VERSION}`,
builder: { id: BUILDER_ID },
invocation: {
configSource: {
uri: `git+${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}@${env.GITHUB_REF}`,
digest: {
sha1: env.GITHUB_SHA,
let payload
if (ci.GITHUB_ACTIONS) {
/* istanbul ignore next - not covering missing env var case */
const [workflowPath] = (env.GITHUB_WORKFLOW_REF || '')
.replace(env.GITHUB_REPOSITORY + '/', '')
.split('@')
payload = {
_type: INTOTO_STATEMENT_TYPE,
subject,
predicateType: SLSA_PREDICATE_TYPE,
predicate: {
buildType: `${GITHUB_BUILD_TYPE_PREFIX}/${GITHUB_BUILD_TYPE_VERSION}`,
builder: { id: GITHUB_BUILDER_ID },
invocation: {
configSource: {
uri: `git+${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}@${env.GITHUB_REF}`,
digest: {
sha1: env.GITHUB_SHA,
},
entryPoint: workflowPath,
},
parameters: {},
environment: {
GITHUB_EVENT_NAME: env.GITHUB_EVENT_NAME,
GITHUB_REF: env.GITHUB_REF,
GITHUB_REPOSITORY: env.GITHUB_REPOSITORY,
GITHUB_REPOSITORY_ID: env.GITHUB_REPOSITORY_ID,
GITHUB_REPOSITORY_OWNER_ID: env.GITHUB_REPOSITORY_OWNER_ID,
GITHUB_RUN_ATTEMPT: env.GITHUB_RUN_ATTEMPT,
GITHUB_RUN_ID: env.GITHUB_RUN_ID,
GITHUB_SHA: env.GITHUB_SHA,
GITHUB_WORKFLOW_REF: env.GITHUB_WORKFLOW_REF,
GITHUB_WORKFLOW_SHA: env.GITHUB_WORKFLOW_SHA,
},
entryPoint: workflowPath,
},
parameters: {},
environment: {
GITHUB_EVENT_NAME: env.GITHUB_EVENT_NAME,
GITHUB_REF: env.GITHUB_REF,
GITHUB_REPOSITORY: env.GITHUB_REPOSITORY,
GITHUB_REPOSITORY_ID: env.GITHUB_REPOSITORY_ID,
GITHUB_REPOSITORY_OWNER_ID: env.GITHUB_REPOSITORY_OWNER_ID,
GITHUB_RUN_ATTEMPT: env.GITHUB_RUN_ATTEMPT,
GITHUB_RUN_ID: env.GITHUB_RUN_ID,
GITHUB_SHA: env.GITHUB_SHA,
GITHUB_WORKFLOW_REF: env.GITHUB_WORKFLOW_REF,
GITHUB_WORKFLOW_SHA: env.GITHUB_WORKFLOW_SHA,
metadata: {
buildInvocationId: `${env.GITHUB_RUN_ID}-${env.GITHUB_RUN_ATTEMPT}`,
completeness: {
parameters: false,
environment: false,
materials: false,
},
reproducible: false,
},
materials: [
{
uri: `git+${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}@${env.GITHUB_REF}`,
digest: {
sha1: env.GITHUB_SHA,
},
},
],
},
metadata: {
buildInvocationId: `${env.GITHUB_RUN_ID}-${env.GITHUB_RUN_ATTEMPT}`,
completeness: {
parameters: false,
environment: false,
materials: false,
}
}
if (ci.GITLAB) {
wlynch marked this conversation as resolved.
Show resolved Hide resolved
payload = {
_type: INTOTO_STATEMENT_TYPE,
subject,
predicateType: SLSA_PREDICATE_TYPE,
predicate: {
buildType: `${GITLAB_BUILD_TYPE_PREFIX}/${GITLAB_BUILD_TYPE_VERSION}`,
builder: { id: `${env.CI_PROJECT_URL}/-/runners/${env.CI_RUNNER_ID}` },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the claims mapped here, do you have access to the CI_RUNNER_ENVIRONMENT here so we could use that instead of the id? Or is id the env? This would make it easier to write policies trust only the gitlab-hosted runner for example

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also should this be per project or just set to the server url runner to make it easier to have some kind of stable identifier for all gitlab.com hosted runners for example?

? env.CI_SERVER_URL/-/runners/${env. CI_RUNNER_ENVIRONMENT}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at https://github.com/in-toto/attestation/blob/v0.1.0/spec/predicates/provenance.md#fields, I don't think that the runner id is an appropriate value to use here.

The identity MUST reflect the trust base that consumers care about. How detailed to be is a judgement call. For example, GitHub Actions supports both GitHub-hosted runners and self-hosted runners. The GitHub-hosted runner might be a single identity because, it's all GitHub from the consumer's perspective. Meanwhile, each self-hosted runner might have its own identity because not all runners are trusted by all consumers.

The runner id identifies the specific machine, and we cycle through machines quite often. For gitlab-hosted runners, we probably want this to be a generic value that denotes the runner is hosted by GitLab. For self-hosted runners, we probably want some way to map this back to the instance, group, or project that is hosting the runner.

Copy link
Contributor Author

@wlynch wlynch Apr 24, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm generally in favor, but I think there's some desire to have this match the existing GitLab provenance document, which uses CI_RUNNER_ID (see https://gist.github.com/wlynch/c7fd8f53adc77d3c0ec82356e4d43cb5 for an example provenance doc I pulled that I used as the basis for this).

This is a common theme for why fields are the way they are for this provenance document.

I think my preference would to start with parity with the existing GitLab SLSA v0.2 spec, and I can sync with @Brcrwilliams @marshall007 and other GitLab folks to how we want this to change this moving forward.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs say builder.id defines the "trust base that consumers care about". I can't say too much about GitLab without more knowledge of that platform, but generally I think this should identify something like "GitLab hosted runner" or "GitLab custom runner". I don't think the runner ID will identify a significant difference in the "trust base".

/cc @MarkLodato @kpk47

However, wrt the format overall maybe this is best taken up with the GitLab Supply Chain Security working group of which I think @kpk47 is a member.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping parity with the existing GitLab SLSA v0.2 spec makes sense to me. Sounds like it would also be useful to start an effort to figure out how the 1.0 spec should look for GitLab and get this included in the slsa org.

Before you can actually test this against the npm registry, we'll need to update the server side checks to accept gitlab provenance statements. There's currently a simple allow list of issuers to allow us to gradually roll out support, as well as checks to make sure the extensions in the signing cert matches what's in the provenance statement.

@wlynch to aid with registry integration, could you include a complete .sigstore bundle files for GitLab? It's a bit annoying getting the sigstore bundle at the moment, but you could modify this line and write the file to disc during a pipeline run:

cli/workspaces/libnpmpublish/lib/publish.js

Line 185 in a558bbd

const serializedBundle = JSON.stringify(provenanceBundle)

Ideally we would also have the fulcio certificate updated to include all the new gitlab claims but can start without this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, wrt the format overall maybe this is best taken up with the GitLab Supply Chain Security working group

I already brought it up in the WG. 馃檪 There's general interest in working towards improving the existing SLSA v0.2 spec + moving to v1.0. We can follow up there.

@wlynch to aid with registry integration, could you include a complete .sigstore bundle files for GitLab?

I'll do this as soon as sigstore/fulcio#983 goes live!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One additional problem of using builder: { id: ${env.CI_PROJECT_URL}/-/runners/${env.CI_RUNNER_ID}} as builder.id is that every project has a different builder. This kinda defeats the purpose of SLSA which scales by having a small set of builders, which you can then use for policy enforcement during verification. The builder.id, as written in this PR, would need to be generated dynamically for verification, even though the builders are technically the same (for GitLab hosted, I mean). See a longer discussion in slsa-framework/slsa#655.

invocation: {
configSource: {
uri: `git+${env.CI_PROJECT_URL}@${env.CI_BUILD_REF}`,
wlynch marked this conversation as resolved.
Show resolved Hide resolved
digest: {
sha1: env.CI_COMMIT_SHA,
},
entryPoint: env.CI_JOB_NAME,
wlynch marked this conversation as resolved.
Show resolved Hide resolved
},
parameters: {
wlynch marked this conversation as resolved.
Show resolved Hide resolved
CI: env.CI,
CI_API_GRAPHQL_URL: env.CI_API_GRAPHQL_URL,
CI_API_V4_URL: env.CI_API_V4_URL,
CI_BUILD_BEFORE_SHA: env.CI_BUILD_BEFORE_SHA,
CI_BUILD_ID: env.CI_BUILD_ID,
CI_BUILD_NAME: env.CI_BUILD_NAME,
CI_BUILD_REF: env.CI_BUILD_REF,
CI_BUILD_REF_NAME: env.CI_BUILD_REF_NAME,
CI_BUILD_REF_SLUG: env.CI_BUILD_REF_SLUG,
CI_BUILD_STAGE: env.CI_BUILD_STAGE,
CI_COMMIT_AUTHOR: env.CI_COMMIT_AUTHOR,
CI_COMMIT_BEFORE_SHA: env.CI_COMMIT_BEFORE_SHA,
CI_COMMIT_BRANCH: env.CI_COMMIT_BRANCH,
CI_COMMIT_DESCRIPTION: env.CI_COMMIT_DESCRIPTION,
CI_COMMIT_MESSAGE: env.CI_COMMIT_MESSAGE,
wlynch marked this conversation as resolved.
Show resolved Hide resolved
CI_COMMIT_REF_NAME: env.CI_COMMIT_REF_NAME,
CI_COMMIT_REF_PROTECTED: env.CI_COMMIT_REF_PROTECTED,
CI_COMMIT_REF_SLUG: env.CI_COMMIT_REF_SLUG,
CI_COMMIT_SHA: env.CI_COMMIT_SHA,
CI_COMMIT_SHORT_SHA: env.CI_COMMIT_SHORT_SHA,
CI_COMMIT_TIMESTAMP: env.CI_COMMIT_TIMESTAMP,
CI_COMMIT_TITLE: env.CI_COMMIT_TITLE,
CI_CONFIG_PATH: env.CI_CONFIG_PATH,
CI_DEFAULT_BRANCH: env.CI_DEFAULT_BRANCH,
CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX:
env.CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX,
CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: env.CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX,
CI_DEPENDENCY_PROXY_SERVER: env.CI_DEPENDENCY_PROXY_SERVER,
CI_DEPENDENCY_PROXY_USER: env.CI_DEPENDENCY_PROXY_USER,
CI_JOB_ID: env.CI_JOB_ID,
CI_JOB_NAME: env.CI_JOB_NAME,
CI_JOB_NAME_SLUG: env.CI_JOB_NAME_SLUG,
CI_JOB_STAGE: env.CI_JOB_STAGE,
CI_JOB_STARTED_AT: env.CI_JOB_STARTED_AT,
CI_JOB_URL: env.CI_JOB_URL,
CI_NODE_TOTAL: env.CI_NODE_TOTAL,
CI_PAGES_DOMAIN: env.CI_PAGES_DOMAIN,
CI_PAGES_URL: env.CI_PAGES_URL,
CI_PIPELINE_CREATED_AT: env.CI_PIPELINE_CREATED_AT,
CI_PIPELINE_ID: env.CI_PIPELINE_ID,
CI_PIPELINE_IID: env.CI_PIPELINE_IID,
CI_PIPELINE_SOURCE: env.CI_PIPELINE_SOURCE,
CI_PIPELINE_URL: env.CI_PIPELINE_URL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wlynch could we add pipeline_ref and pipeline_sha that are now in the claims? Or are these in here but with different names?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see these are still in progress: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117923

Copy link
Contributor Author

@wlynch wlynch May 8, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pipeline_ref is populated in environent.pipeline.ref via CI_CONFIG_PATH.

pipeline_sha is WIP on the GitLab side and may not always be populated (i.e. if being fetched from remote sources) - I've raised with GitLab folks. The safer thing to key off of for provenance checking is pipeline id / job id. I don't think this is blocking for this PR.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wlynch pipeline_ref is actually a URI that links to the pipeline configuration. The name is easy to confuse with a git ref.

@marshall007 @aladh Maybe we should call it something else like pipeline_config_uri and pipeline_config_sha.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wlynch pipeline_ref is actually a URI that links to the pipeline configuration. The name is easy to confuse with a git ref.

@marshall007 @aladh Maybe we should call it something else like pipeline_config_uri and pipeline_config_sha.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what we want. 馃檪

What we're ultimately looking for is a hook to the Fulcio cert Build Config URI to match for verification.

No objections to renaming though.

CI_PROJECT_CLASSIFICATION_LABEL: env.CI_PROJECT_CLASSIFICATION_LABEL,
CI_PROJECT_DESCRIPTION: env.CI_PROJECT_DESCRIPTION,
CI_PROJECT_ID: env.CI_PROJECT_ID,
CI_PROJECT_NAME: env.CI_PROJECT_NAME,
CI_PROJECT_NAMESPACE: env.CI_PROJECT_NAMESPACE,
CI_PROJECT_NAMESPACE_ID: env.CI_PROJECT_NAMESPACE_ID,
CI_PROJECT_PATH: env.CI_PROJECT_PATH,
CI_PROJECT_PATH_SLUG: env.CI_PROJECT_PATH_SLUG,
CI_PROJECT_REPOSITORY_LANGUAGES: env.CI_PROJECT_REPOSITORY_LANGUAGES,
CI_PROJECT_ROOT_NAMESPACE: env.CI_PROJECT_ROOT_NAMESPACE,
CI_PROJECT_TITLE: env.CI_PROJECT_TITLE,
CI_PROJECT_URL: env.CI_PROJECT_URL,
CI_PROJECT_VISIBILITY: env.CI_PROJECT_VISIBILITY,
CI_REGISTRY: env.CI_REGISTRY,
CI_REGISTRY_IMAGE: env.CI_REGISTRY_IMAGE,
CI_REGISTRY_USER: env.CI_REGISTRY_USER,
CI_REPOSITORY_URL: env.CI_REPOSITORY_URL,
CI_RUNNER_DESCRIPTION: env.CI_RUNNER_DESCRIPTION,
CI_RUNNER_ID: env.CI_RUNNER_ID,
CI_RUNNER_TAGS: env.CI_RUNNER_TAGS,
CI_SERVER_HOST: env.CI_SERVER_HOST,
CI_SERVER_NAME: env.CI_SERVER_NAME,
CI_SERVER_PORT: env.CI_SERVER_PORT,
CI_SERVER_PROTOCOL: env.CI_SERVER_PROTOCOL,
CI_SERVER_REVISION: env.CI_SERVER_REVISION,
CI_SERVER_SHELL_SSH_HOST: env.CI_SERVER_SHELL_SSH_HOST,
CI_SERVER_SHELL_SSH_PORT: env.CI_SERVER_SHELL_SSH_PORT,
CI_SERVER_URL: env.CI_SERVER_URL,
CI_SERVER_VERSION: env.CI_SERVER_VERSION,
CI_SERVER_VERSION_MAJOR: env.CI_SERVER_VERSION_MAJOR,
CI_SERVER_VERSION_MINOR: env.CI_SERVER_VERSION_MINOR,
CI_SERVER_VERSION_PATCH: env.CI_SERVER_VERSION_PATCH,
CI_TEMPLATE_REGISTRY_HOST: env.CI_TEMPLATE_REGISTRY_HOST,
GITLAB_CI: env.GITLAB_CI,
GITLAB_FEATURES: env.GITLAB_FEATURES,
GITLAB_USER_EMAIL: env.GITLAB_USER_EMAIL,
GITLAB_USER_ID: env.GITLAB_USER_ID,
GITLAB_USER_LOGIN: env.GITLAB_USER_LOGIN,
GITLAB_USER_NAME: env.GITLAB_USER_NAME,
RUNNER_GENERATE_ARTIFACTS_METADATA: env.RUNNER_GENERATE_ARTIFACTS_METADATA,
},
environment: {
name: env.CI_RUNNER_DESCRIPTION,
architecture: env.CI_RUNNER_EXECUTABLE_ARCH,
server: env.CI_SERVER_URL,
project: env.CI_PROJECT_PATH,
job: {
id: env.CI_JOB_ID,
},
pipeline: {
id: env.CI_PIPELINE_ID,
ref: env.CI_CONFIG_PATH,
},
},
},
reproducible: false,
},
materials: [
{
uri: `git+${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}@${env.GITHUB_REF}`,
digest: {
sha1: env.GITHUB_SHA,
metadata: {
buildInvocationId: `${env.CI_JOB_URL}`,
wlynch marked this conversation as resolved.
Show resolved Hide resolved
completeness: {
parameters: true,
environment: true,
materials: false,
},
reproducible: false,
},
],
},
materials: [
{
uri: `git+${env.CI_PROJECT_URL}@${env.CI_BUILD_REF}`,
digest: {
sha1: env.CI_COMMIT_SHA,
},
},
],
},
}
}

return sigstore.attest(Buffer.from(JSON.stringify(payload)), INTOTO_PAYLOAD_TYPE, opts)
}

Expand Down
32 changes: 20 additions & 12 deletions workspaces/libnpmpublish/lib/publish.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,19 +144,27 @@ const buildMetadata = async (registry, manifest, tarballData, spec, opts) => {
digest: { sha512: integrity.sha512[0].hexDigest() },
}

// Ensure that we're running in GHA, currently the only supported build environment
if (ciInfo.name !== 'GitHub Actions') {
if (ciInfo.GITHUB_ACTIONS) {
// Ensure that the GHA OIDC token is available
if (!process.env.ACTIONS_ID_TOKEN_REQUEST_URL) {
throw Object.assign(
/* eslint-disable-next-line max-len */
new Error('Provenance generation in GitHub Actions requires "write" access to the "id-token" permission'),
{ code: 'EUSAGE' }
)
}
} else if (ciInfo.GITLAB) {
// Ensure that the Sigstore OIDC token is available
if (!process.env.SIGSTORE_ID_TOKEN) {
throw Object.assign(
/* eslint-disable-next-line max-len */
new Error('Provenance generation in GitLab CI requires "SIGSTORE_ID_TOKEN" with "sigstore" audience to be present in "id_tokens". For more info see:\nhttps://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html'),
{ code: 'EUSAGE' }
)
}
} else {
throw Object.assign(
new Error('Automatic provenance generation not supported outside of GitHub Actions'),
{ code: 'EUSAGE' }
)
}

// Ensure that the GHA OIDC token is available
if (!process.env.ACTIONS_ID_TOKEN_REQUEST_URL) {
throw Object.assign(
/* eslint-disable-next-line max-len */
new Error('Provenance generation in GitHub Actions requires "write" access to the "id-token" permission'),
new Error('Automatic provenance generation not supported for provider: ' + ciInfo.name),
{ code: 'EUSAGE' }
)
}
Expand Down