Skip to content

Remove code related to Package Storage v1 and update README #1021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Dec 12, 2022
Merged

Remove code related to Package Storage v1 and update README #1021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
13e6cc9
Remove promte and publish subcommands
mrodm Oct 26, 2022
caee464
Update status command
mrodm Oct 26, 2022
de92be8
Add test for local pending changes
mrodm Oct 27, 2022
dd944ae
Remove promote and publish flags
mrodm Oct 27, 2022
7bb0f43
Update Readme
mrodm Oct 27, 2022
0b53208
Remove comment from CI job
mrodm Oct 27, 2022
8fd630f
Merge remote upstream/main into remove_code_package_storage_v1
mrodm Nov 14, 2022
c5f9f02
Add promote and publish commands with deprecated message
mrodm Nov 21, 2022
1b099da
Update readme and add DEPRECATED in promote and publish commands
mrodm Dec 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .ci/package-storage-publish.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ pipeline {
withGoEnv() {
dir("${BASE_DIR}") {
sh(label: 'Install elastic-package',script: "make install")
// sh(label: 'Install elastic-package', script: 'go build github.com/elastic/elastic-package')
dir("test/packages/package-storage/package_storage_candidate") {
sh(label: 'Build package', script: "elastic-package build -v --zip")
}
Expand Down
22 changes: 3 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ WARNING: This is a generated file. Do NOT edit it manually. To regenerate this f
# elastic-package

`elastic-package` is a command line tool, written in Go, used for developing Elastic packages. It can help you lint, format,
test, build, and promote your packages. Learn about each of these and other features in [_Commands_](#commands) below.
test and build your packages. Learn about each of these and other features in [_Commands_](#commands) below.

Currently, `elastic-package` only supports packages of type [Elastic Integrations](https://github.com/elastic/integrations).

Expand Down Expand Up @@ -200,7 +200,7 @@ User profiles are not overwritten on upgrade of elastic-stack, and can be freely

_Context: global_

Use this command to move packages between the snapshot, staging, and production stages of the package registry.
[DEPRECATED] Use this command to move packages between the snapshot, staging, and production stages of the package registry.

This command is intended primarily for use by administrators.

Expand All @@ -210,7 +210,7 @@ It allows for selecting packages for promotion and opens new pull requests to re

_Context: package_

Use this command to publish a new package revision.
[DEPRECATED] Use this command to publish a new package revision.

The command checks if the package hasn't been already published (whether it's present in snapshot/staging/production branch or open as pull request). If the package revision hasn't been published, it will open a new pull request.

Expand Down Expand Up @@ -298,22 +298,6 @@ Use this command to print the version of elastic-package that you have installed



### GitHub authorization

The `promote` and `publish` commands require access to the GitHub API to open pull requests or check authorized account data.
The tool uses the GitHub token to authorize user's call to API. The token can be stored in the `~/.elastic/github.token`
file or passed via the `GITHUB_TOKEN` environment variable.

Here are the instructions on how to create your own personal access token (PAT):
https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token

Make sure you have enabled the following scopes:
* `public_repo` — to open pull requests on GitHub repositories.
* `read:user` and `user:email` — to read your user profile information from GitHub in order to populate pull requests appropriately.

After creating or modifying your personal access token, authorize the token for
use of the Elastic organization: https://docs.github.com/en/github/authenticating-to-github/authenticating-with-saml-single-sign-on/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on

Comment on lines -301 to -316
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As elastic-package should not create any PRs, I think this section is not needed anymore. Is there any other usage for this token ? @elastic/ecosystem

## Development

Even though the project is "go-gettable", there is the `Makefile` present, which can be used to build, format or vendor
Expand Down
230 changes: 3 additions & 227 deletions cmd/promote.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,34 +5,17 @@
package cmd

import (
"fmt"
"strings"
"time"

"github.com/AlecAivazis/survey/v2"
"github.com/pkg/errors"
"github.com/spf13/cobra"

"github.com/elastic/elastic-package/internal/cobraext"
"github.com/elastic/elastic-package/internal/github"
"github.com/elastic/elastic-package/internal/promote"
"github.com/elastic/elastic-package/internal/storage"
)

const promoteLongDescription = `Use this command to move packages between the snapshot, staging, and production stages of the package registry.
const promoteLongDescription = `[DEPRECATED] Use this command to move packages between the snapshot, staging, and production stages of the package registry.

This command is intended primarily for use by administrators.

It allows for selecting packages for promotion and opens new pull requests to review changes. Please be aware that the tool checks out an in-memory Git repository and switches over branches (snapshot, staging and production), so it may take longer to promote a larger number of packages.`

const (
promoteDirectionSnapshotStaging = "snapshot-staging"
promoteDirectionStagingProduction = "staging-production"
promoteDirectionSnapshotProduction = "snapshot-production"
)

var promotionDirections = []string{promoteDirectionSnapshotStaging, promoteDirectionStagingProduction, promoteDirectionSnapshotProduction}

func setupPromoteCommand() *cobraext.Command {
cmd := &cobra.Command{
Use: "promote",
Expand All @@ -41,6 +24,7 @@ func setupPromoteCommand() *cobraext.Command {
RunE: promoteCommandAction,
SilenceUsage: true,
}

cmd.Flags().StringP(cobraext.DirectionFlagName, "d", "", cobraext.DirectionFlagDescription)
cmd.Flags().BoolP(cobraext.NewestOnlyFlagName, "n", false, cobraext.NewestOnlyFlagDescription)
cmd.Flags().StringSliceP(cobraext.PromotedPackagesFlagName, "p", nil, cobraext.PromotedPackagesFlagDescription)
Expand All @@ -50,214 +34,6 @@ func setupPromoteCommand() *cobraext.Command {

func promoteCommandAction(cmd *cobra.Command, _ []string) error {
cmd.Println("Promote packages")
cmd.Println("DEPRECATED: Packages stored in the Package Storage v2 won't require to be promoted. This command will be removed soon. README: https://github.com/elastic/elastic-package/blob/main/docs/howto/use_package_storage_v2.md")

// Setup GitHub
err := github.EnsureAuthConfigured()
if err != nil {
return errors.Wrap(err, "GitHub auth configuration failed")
}

githubClient, err := github.Client()
if err != nil {
return errors.Wrap(err, "creating GitHub client failed")
}

githubUser, err := github.User(githubClient)
if err != nil {
return errors.Wrap(err, "fetching GitHub user failed")
}
cmd.Printf("Current GitHub user: %s\n", githubUser)

// Prompt for promotion options
sourceStage, destinationStage, err := promptPromotion(cmd)
if err != nil {
return errors.Wrap(err, "prompt for promotion failed")
}

newestOnly, err := promptPromoteNewestOnly(cmd)
if err != nil {
return errors.Wrap(err, "prompt for promoting newest versions only failed")
}

cmd.Println("Cloning repository...")
repository, err := storage.CloneRepository(githubUser, sourceStage)
if err != nil {
return errors.Wrapf(err, "cloning source repository failed (branch: %s)", sourceStage)
}

cmd.Println("Creating list of packages...")
allPackages, err := storage.ListPackages(repository)
if err != nil {
return errors.Wrapf(err, "listing packages failed")
}

packagesToBeSelected := allPackages.FilterPackages(newestOnly)
if len(packagesToBeSelected) == 0 {
fmt.Println("No packages available for promotion.")
return nil
}

promotedPackages, err := promptPackages(cmd, packagesToBeSelected)
if err != nil {
return errors.Wrap(err, "prompt for package selection failed")
}

removedPackages := promote.DeterminePackagesToBeRemoved(allPackages, promotedPackages, newestOnly)

nonce := time.Now().UnixNano()
// Copy packages to destination
fmt.Printf("Promote packages from %s to %s...\n", sourceStage, destinationStage)
newDestinationBranch := fmt.Sprintf("promote-from-%s-to-%s-%d", sourceStage, destinationStage, nonce)
err = storage.CopyPackages(repository, sourceStage, destinationStage, promotedPackages, newDestinationBranch)
if err != nil {
return errors.Wrapf(err, "copying packages failed (source: %s, destination: %s)", sourceStage, destinationStage)
}

// Remove packages from source
newSourceBranch := fmt.Sprintf("delete-from-%s-%d", sourceStage, nonce)
err = storage.RemovePackages(repository, sourceStage, removedPackages, newSourceBranch)
if err != nil {
return errors.Wrapf(err, "removing packages failed (source: %s)", sourceStage)
}

// Push changes
err = storage.PushChanges(githubUser, repository, newSourceBranch, newDestinationBranch)
if err != nil {
return errors.Wrapf(err, "pushing changes failed")
}

// Calculate package signatures
signedPackages, err := storage.CalculatePackageSignatures(repository, newDestinationBranch, promotedPackages)
if err != nil {
return errors.Wrap(err, "signing packages failed")
}

// Open PRs
url, err := promote.OpenPullRequestWithPromotedPackages(githubClient, githubUser, newDestinationBranch, destinationStage, sourceStage, destinationStage, signedPackages)
if err != nil {
return errors.Wrapf(err, "opening PR with promoted packages failed (head: %s, base: %s)", newDestinationBranch, destinationStage)
}
cmd.Println("Pull request with promoted packages:", url)

url, err = promote.OpenPullRequestWithRemovedPackages(githubClient, githubUser, newSourceBranch, sourceStage, sourceStage, url, removedPackages)
if err != nil {
return errors.Wrapf(err, "opening PR with removed packages failed (head: %s, base: %s)", newDestinationBranch, destinationStage)
}
cmd.Println("Pull request with removed packages:", url)

cmd.Println("Done")
cmd.Println("DEPRECATED: Packages stored in the Package Storage v2 do not require to be promoted. README: https://github.com/elastic/elastic-package/blob/main/docs/howto/use_package_storage_v2.md")
return nil
}

func promptPromotion(cmd *cobra.Command) (string, string, error) {
direction, err := cmd.Flags().GetString(cobraext.DirectionFlagName)
if err != nil {
return "", "", errors.Wrapf(err, "can't read %s flag:", cobraext.DirectionFlagName)
}

if direction != "" {
if !isSupportedPromotionDirection(direction) {
return "", "", fmt.Errorf("unsupported promotion direction, use: %s",
strings.Join(promotionDirections, ", "))
}

s := strings.Split(direction, "-")
return s[0], s[1], nil
}

promotionPrompt := &survey.Select{
Message: "Which promotion would you like to run",
Options: promotionDirections,
Default: promoteDirectionSnapshotStaging,
}

err = survey.AskOne(promotionPrompt, &direction)
if err != nil {
return "", "", err
}

s := strings.Split(direction, "-")
return s[0], s[1], nil
}

func isSupportedPromotionDirection(direction string) bool {
for _, d := range promotionDirections {
if d == direction {
return true
}
}
return false
}

func promptPromoteNewestOnly(cmd *cobra.Command) (bool, error) {
newestOnly := false

newestOnlyFlag := cmd.Flags().Lookup(cobraext.NewestOnlyFlagName)
if newestOnlyFlag.Changed {
newestOnly, _ = cmd.Flags().GetBool(cobraext.NewestOnlyFlagName)
return newestOnly, nil
}

prompt := &survey.Confirm{
Message: "Would you like to promote newest versions only and remove older ones?",
Default: true,
}
err := survey.AskOne(prompt, &newestOnly)
if err != nil {
return false, err
}
return newestOnly, nil
}

func promptPackages(cmd *cobra.Command, packages storage.PackageVersions) (storage.PackageVersions, error) {
revisions, _ := cmd.Flags().GetStringSlice(cobraext.PromotedPackagesFlagName)
if len(revisions) > 0 {
parsed, err := storage.ParsePackageVersions(revisions)
if err != nil {
return nil, errors.Wrap(err, "can't parse package versions")
}
return selectPackageVersions(packages, parsed)
}

packagesPrompt := &survey.MultiSelect{
Message: "Which packages would you like to promote",
Options: packages.Strings(),
PageSize: 100,
}

var selectedOptions []string
err := survey.AskOne(packagesPrompt, &selectedOptions, survey.WithValidator(survey.Required))
if err != nil {
return nil, err
}

var selected storage.PackageVersions
for _, option := range selectedOptions {
for _, p := range packages {
if p.String() == option {
selected = append(selected, p)
}
}
}
return selected, nil
}

func selectPackageVersions(packages storage.PackageVersions, toBeSelected storage.PackageVersions) (storage.PackageVersions, error) {
var selected storage.PackageVersions
for _, r := range toBeSelected {
var found bool
for _, pv := range packages {
if pv.Equal(r) {
selected = append(selected, pv)
found = true
break
}
}

if !found {
return nil, fmt.Errorf("package revision is not present (%s) in the source stage, try to run the command without %s flag", r.String(), cobraext.NewestOnlyFlagName)
}
}
return selected, nil
}
42 changes: 2 additions & 40 deletions cmd/publish.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,12 @@
package cmd

import (
"github.com/pkg/errors"
"github.com/spf13/cobra"

"github.com/elastic/elastic-package/internal/cobraext"
"github.com/elastic/elastic-package/internal/github"
"github.com/elastic/elastic-package/internal/publish"
)

const publishLongDescription = `Use this command to publish a new package revision.
const publishLongDescription = `[DEPRECATED] Use this command to publish a new package revision.

The command checks if the package hasn't been already published (whether it's present in snapshot/staging/production branch or open as pull request). If the package revision hasn't been published, it will open a new pull request.`

Expand All @@ -37,42 +34,7 @@ func setupPublishCommand() *cobraext.Command {

func publishCommandAction(cmd *cobra.Command, args []string) error {
cmd.Println("Publish the package")
cmd.Println("DEPRECATED: Package candidates to the Package Storage v2 will be published using Jenkins jobs. This command will be removed soon. README: https://github.com/elastic/elastic-package/blob/main/docs/howto/use_package_storage_v2.md")
cmd.Println("DEPRECATED: Package candidates to the Package Storage v2 are published using Jenkins jobs. README: https://github.com/elastic/elastic-package/blob/main/docs/howto/use_package_storage_v2.md")

fork, err := cmd.Flags().GetBool(cobraext.ForkFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.ForkFlagName)
}

skipPullRequest, err := cmd.Flags().GetBool(cobraext.SkipPullRequestFlagName)
if err != nil {
return cobraext.FlagParsingError(err, cobraext.SkipPullRequestFlagName)
}

// Setup GitHub
err = github.EnsureAuthConfigured()
if err != nil {
return errors.Wrap(err, "GitHub auth configuration failed")
}

githubClient, err := github.Client()
if err != nil {
return errors.Wrap(err, "creating GitHub client failed")
}

// GitHub user
githubUser, err := github.User(githubClient)
if err != nil {
return errors.Wrap(err, "fetching GitHub user failed")
}
cmd.Printf("Current GitHub user: %s\n", githubUser)

// Publish the package
err = publish.Package(githubUser, githubClient, fork, skipPullRequest)
if err != nil {
return errors.Wrap(err, "can't publish the package")
}

cmd.Println("Done")
return nil
}
2 changes: 1 addition & 1 deletion cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ var commands = []*cobraext.Command{
setupFormatCommand(),
setupInstallCommand(),
setupLintCommand(),
setupProfilesCommand(),
setupPromoteCommand(),
setupProfilesCommand(),
setupPublishCommand(),
setupReportsCommand(),
setupServiceCommand(),
Expand Down
Loading