Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 0 additions & 54 deletions docs/howto/use_package_storage_v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,57 +41,3 @@ We identified a few issues in v1 design, we couldn't easily overcome or patch:
4. Enable validation for incoming packages (spec and signatures).
5. Support package signatures. It wasn't possible to calculate the signature for unarchived package directories.

## What should a package owner do to automatically publish their packages?

### Existing packages

Package revisions already deployed in the production Package Storage (present in the `production` branch of the [package-storage](https://github.com/elastic/package-storage))
are automatically synced with the new storage. In this case we disable the validation as some older packages will not be able
to pass current spec requirements.

Sync between v1 and v2 will be enabled until we officially deprecate the v1 storage (no more PRs or promotions).

### Next revisions

Before we deprecate the v1 storage, package owners will have to adjust their releasing pipelines to submit packages
to the new destination. Every package candidate should be submitted together with a corresponding signature, generated
using the [Elastic signing pipeline](https://internal-ci.elastic.co/job/elastic+unified-release+master+sign-artifacts-with-gpg/).

Here is the list of requirements and code modifications based on the `beats-ci`.

#### Requirements

1. CI job signing credentials (`sign-artifacts-with-gpg-job`) - use them to call the signing pipeline on
the `internal-ci` Jenkins instance. The pipeline will sign artifacts uploaded to the signing bucket and upload there their signatures.
2. Signing bucket credentials (`internal-ci-gcs-plugin`) - use them to upload zipped packages to be signed
and download matching signatures.
3. Package Storage GCP uploader credentials (`upload-package-to-package-storage`) - use them to upload a package candidate to the "queue" bucket.
The candidates will be picked by the publishing job and removed after processing.
4. Package Storage uploader secret (`secret/gce/elastic-bekitzur/service-account/package-storage-uploader`) - use it to kick off
the publishing job to process the uploaded candidate.

#### Code modifications

These code modifications refer to the Jenkinsfile/groovy files, which will orchestrate the Jenkins worker to sign the package
and publish it using the Package Storage publishing job.

Function [packageStoragePublish(...)](https://github.com/elastic/elastic-package/blob/f8f678d20b9b60d438188e8dfd2fb4e7519b5a69/.ci/package-storage-publish.groovy#L70)

##### Sign the package candidate

Function [signUnpublishedArtifactsWithElastic(...)](https://github.com/elastic/elastic-package/blob/f8f678d20b9b60d438188e8dfd2fb4e7519b5a69/.ci/package-storage-publish.groovy#L87-L122).

1. Check if the package has been already published (HTTP request to EPR).
2. Upload the package candidate to the signing bucket.
3. Call the Elastic signing pipeline to create matching signatures. The pipeline signs them using the Elastic private key.
4. Once the job succeeded, download package signatures.

##### Publish the package candidate

Function [uploadUnpublishedToPackageStorage(...)](https://github.com/elastic/elastic-package/blob/f8f678d20b9b60d438188e8dfd2fb4e7519b5a69/.ci/package-storage-publish.groovy#L124-L151).

1. Check if the package has been already published (HTTP request to EPR).
2. Upload the package candidate to the special "queue" bucket - `elastic-bekitzur-package-storage-internal`.
3. Call the [publishing job](https://internal-ci.elastic.co/job/package_storage/job/publishing-job-remote/). The publishing jobs verifies
correctness of the package format and corresponding signature. Next, the job extracts static resources, uploads the zipped package
to the public bucket, and schedules indexing in background.