bosh storage cli repo consolidation

[stephanme]

rfc-0043-cc-blobstore-storage-cli proposes to consolidate the 5 existing bosh storage cli repos into one storage-cli repo.

This issue is about discussion different options and to lay out a migration plan.

Goals

• better reuse of code, e.g. cmd line processing
• foster consistency across the different IaaS implementations
• less effort with maintaining CI

Ideas, Discussion Points

• GitHub Actions instead of Concourse pipeline for CI
  • positive: simpler, nice Github integration, Github Packages and ghcr instead of extra infrastructure like S3 buckets
  • positive: CFF promotes Github Actions over Concourse because its cheaper
  • negative: more migration effort
• one storage CLI binary for all IaaS instead of multiple binaries per IaaS (like before)
  • positive: simpler packaging and build, bosh and capi need to package storage CLI(s) for all IaaS anyway
  • neutral: minor changes in CLI options required
  • negative: no option to rollback a storage cli version for one IaaS

Migration Steps

1. Move coding of existing 5 bosh-storage-cli repos into storage-cli repo as-is
   • one subfolder per IaaS, one CLI binary by IaaS
   • adapt go packages (e.g. github.com/cloudfoundry/bosh-azure-storage-cli -> github.com/cloudfoundry/storage/cli/azure)
   • setup/migrate CI and release process (TODO: Github Actions or Concourse)
   • adapt consumption in bosh and capi
2. (optional, TODO) Migrate from multiple CLIs to one CLI
   • adapt consumption in bosh and capi
3. Refactoring
   • e.g. reduce duplicated coding

Repo Structure

storage-cli
├── .github
│   └── ...
├── ali
│   └── ...
├── azure
│   ├── client
│   ├── config
│   ├── integration
│   └── main.go
├── dav
│   └── ...
├── gcs
│   └── ...
├── go.mod
├── go.sum
├── LICENSE
├── NOTICE
├── README.md
├── s3
│   ├── client
│   ├── config
│   ├── integration
│   └── main.go
├── tool.go
└── tools
    └── tools.go

[stephanme]

FI WG meeting:

• single storage CLI binary preferred (simpler distribution)
• github actions preferred
• docker images produced by todays Concourse pipeline is consumed by the pipeline only (no external consumers known) -> could be dropped when using github actions