Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Provide consolidated bundles with Pact Ruby/Rust Core #117

Closed
wants to merge 11 commits into from

Conversation

YOU54F
Copy link
Member

@YOU54F YOU54F commented Aug 4, 2023

Initial thoughts

Looking at the size of the pact-plugin-cli executable (11mb) I am inclined to suggest we seperate out and publish two sets of packages

  • one containing the pure ruby executables
  • one containing all our pact executables
├── pact
├── pactflow
├── pact-broker
├── pact-message
├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli
└── pact-plugin-cli

If that is produced in one place, its easy for HomeBrew / Scoop and our scripted installed to provide a ruby-standalone or full-fat cli experience.

Ideally we could use Rust going forward as a way to provide a directory to install all the cli tooling, the pact-plugin-cli provides a nice template for that.

Could a pact-cli command in Rust, abstract away the rubyisms behind, so a pact-cli broker command feels natural and the fact it is ruby or rust in the backend is a moot point.

For tools there are duplicated in rust/ruby, we could provide an abstraction through a single command (rather than having two) and use a switch --legacy or PACT_USE_LEGACY=1 env var to switch between either the rust or the ruby impl.

├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli

Brain dump

The pact-ruby-standalone project recently added the pact-plugin-cli to the bundle

This weighs in at 12Mb and hugely inflates our bundle.

This proposal suggests a couple of different packages

  • pact - ruby only pact packages
  • pact-rust - ruby + rust packages
  • pact-ffi - ruby + pact_ffi lib and pact-ffi gem
  • pact-cli - ruby + rust + pact_ffi lib and pact-ffi gem

for any of the combinations above, they can have

  • pact--slim

augmentations, which seek to trim out as much as possible from the ruby runtime.

Note:- the slim versions may not work and need throughly testing, cursory testing has been
done so far.

The overall aim is provide users with a single bundled distribution called pact-cli

which contains every single pact executable so the users can get all the pact goodness at their
fingertips.

This will have benefits for packaging with HomeBrew/Scoop/Choco and our own install script.

In the future it would be nice to extend pact-plugin-cli which can discover and install plugins

to discover all of the pact tooling (official and community provided) and provide a mechanism to install

to a shared location such as PACT_HOME_DIR which would default to $HOME/.pact

This pattern is already in use in the pact-plugin-cli

Could we further this, so users only need the standalone in a single location, and the ffi libs in a single location, and our various client libraries to point to the relevant home folder, and not require the packages to contain the host libraries themselves. This may not be practical for some libs which need to build from source, but if the source was already on their machine, via our installer, maybe that makes it less of an issue.

The tree of all tools would currently look like this

├── pact
├── pactflow
├── pact-broker
├── pact-message
├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli
└── pact-plugin-cli

Users would be able to interact with the FFI by

`${PATH_TO_RUBY}ruby -rpact/ffi -e "puts PactFfi.pactffi_version"`

Package sizes

Full Ruby Bundle - with pact-plugin-cli

As per todays release.

49M    pkg/pact

16M    pkg/pact-2.0.2-linux-arm64.tar.gz
16M    pkg/pact-2.0.2-linux-x86_64.tar.gz
14M    pkg/pact-2.0.2-osx-arm64.tar.gz
15M    pkg/pact-2.0.2-osx-x86_64.tar.gz
11M    pkg/pact-2.0.2-windows-x86.zip
16M    pkg/pact-2.0.2-windows-x86_64.zip

Full Ruby (only) Bundle

37M    pkg/pact

12M    pkg/pact-2.0.2-windows-x86_64.zip
12M    pkg/pact-2.0.2-windows-x86.zip
10M    pkg/pact-2.0.2-osx-x86_64.tar.gz
10M    pkg/pact-2.0.2-linux-x86_64.tar.gz
9.0M   pkg/pact-2.0.2-osx-arm64.tar.gz
9.0M   pkg/pact-2.0.2-linux-arm64.tar.gz

Trimmed Ruby (only) Bundle

TRIM_PACKAGE_FULL=true bundle exec rake package

28M    pkg/pact

10M    pact-2.0.2-windows-x86.zip
9.7M   pact-2.0.2-windows-x86_64.zip
7.0M   pact-2.0.2-linux-arm64.tar.gz
7.0M   pact-2.0.2-linux-x86_64.tar.gz
7.0M   pact-2.0.2-osx-arm64.tar.gz
7.0M   pact-2.0.2-osx-x86_64.tar.gz

Full Ruby Bundle + Pact Rust Tools

PACKAGE_PACT_RUST_TOOLS=true bundle exec rake package

98M    pkg/pact

41M    pkg/pact-2.0.2-linux-x86_64.tar.gz
39M    pkg/pact-2.0.2-linux-arm64.tar.gz
34M    pkg/pact-2.0.2-osx-x86_64.tar.gz
33M    pkg/pact-2.0.2-osx-arm64.tar.gz
33M    pkg/pact-2.0.2-windows-x86_64.zip
11M    pkg/pact-2.0.2-windows-x86.zip

du -sh pkg/pact/bin/* | sort -nr

18M    pkg/pact/bin/pact_verifier_cli
17M    pkg/pact/bin/pact_mock_server_cli
14M    pkg/pact/bin/pact-stub-server
12M    pkg/pact/bin/pact-plugin-cli

Ruby (only) + Pact-Ffi Gem (and pact_ffi libs) Bundle

PACKAGE_PACT_FFI=true bundle exec rake package

64M    pkg/pact

21M    pkg/pact-2.0.2-windows-x86_64.zip
21M    pkg/pact-2.0.2-windows-x86.zip
19M    pkg/pact-2.0.2-osx-x86_64.tar.gz
19M    pkg/pact-2.0.2-osx-arm64.tar.gz
19M    pkg/pact-2.0.2-linux-x86_64.tar.gz
19M    pkg/pact-2.0.2-linux-arm64.tar.gz

24M    pkg/pact/lib/vendor/ruby/3.2.0/gems/pact-ffi-0.0.2-arm64-darwin/ffi/macos-arm64/libpact_ffi.dylib

Full Ruby Bundle + Pact Rust Tools + Pact-Ffi Gem (and pact_ffi libs)

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true bundle exec rake package

123M    pkg/pact-cli
52M    pkg/pact-cli-2.0.2-linux-x86_64.tar.gz
51M    pkg/pact-cli-2.0.2-linux-arm64.tar.gz
43M    pkg/pact-cli-2.0.2-osx-x86_64.tar.gz
41M    pkg/pact-cli-2.0.2-osx-arm64.tar.gz
33M    pkg/pact-cli-2.0.2-windows-x86_64.zip
12M    pkg/pact-cli-2.0.2-windows-x86.zip
24M    pkg/pact/lib/vendor/ruby/3.2.0/gems/pact-ffi-0.0.2-arm64-darwin/ffi/macos-arm64/libpact_ffi.dylib

Building & Testing.

You can easily run through the testing by passing the same env vars.

The following scenario will

  • package the pact-ruby-standalone for osx:arm64
  • include the rust tools
  • include the pact ffi
  • unpack the built package that would be distributed to users
  • runs a smoke test against the package

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true bundle exec rake package:osx:arm64

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true PACKAGE_NAME=pact-cli ./script/unpack-and-test.sh

You can run the smoke tests anyway, your standalone directory lives.

  • Run the script /script/test.sh
  • Set your env vars
    • PATH_TO_BIN=pkg/pact-cli/bin/ default is $PACKAGE_NAME/bin/
      • ensure it has a trailing /
      • if windows, reverse the slashes \
    • PACKAGE_NAME default is pact

Example scenario

  • runs a smoke test against the package
    • includes tests for the rust tools
    • includes test for the pact ffi

PACKAGE_PACT_RUST_TOOLS=true PACKAGE_PACT_FFI=true PACKAGE_NAME=pact-cli PATH_TO_BIN=pkg/pact-cli/bin/ ./script/test.sh

YOU54F and others added 11 commits June 8, 2023 16:04
The pact-ruby-standalone project recently added the pact-plugin-cli to the bundle

This weighs in at 12Mb and hugely inflates our bundle.

This proposal suggests a couple of different packages

pact - ruby only pact packages
pact-rust - ruby + rust packages
pact-ffi - ruby + pact_ffi lib and pact-ffi gem
pact-cli - ruby + rust + pact_ffi lib and pact-ffi gem

for any of the combinations above, they can have

pact-slim-<xyz>

augmentations, which seek to trim out as much as possible from the ruby runtime.

Note:- the slim versions may not work and need throughly testing, cursory testing has been
  done so far.

The overall aim is provide users with a single bundled distribution called pact-cli

which contains every single pact executable so the users can get all the pact goodness at their
  fingertips.

This will have benefits for packaging with HomeBrew/Scoop/Choco and our own install script.

In the future it would be nice to extend pact-plugin-cli which can discover and install plugins

to discover all of the pact tooling (official and community provided) and provide a mechanism to install

to a shared location such as PACT_HOME_DIR which would default to $HOME/.pact

This pattern is already in use in the pact-plugin-cli

Could we further this, so users only need the standalone in a single location, and the ffi libs in a single location, and our various client libraries to point to the relevant home folder, and not require the packages to contain the host libraries themselves. This may not be practical for some libs which need to build from source, but if the source was already on their machine, via our installer, maybe that makes it less of an issue.

The tree of all tools would currently look like this

```lisp
├── pact
├── pactflow
├── pact-broker
├── pact-message
├── pact-mock-service
├── pact-stub-service
├── pact-provider-verifier
├── pact_mock_server_cli
├── pact-stub-server
├── pact_verifier_cli
└── pact-plugin-cli
```

Users would be able to interact with the FFI by

    ${PATH_TO_RUBY}ruby -rpact/ffi -e "puts PactFfi.pactffi_version"
@YOU54F
Copy link
Member Author

YOU54F commented Aug 4, 2023

example release here with the some of the proposed formats

https://github.com/YOU54F/pact-ruby-standalone/releases/tag/v2.3.0

@YOU54F YOU54F mentioned this pull request Aug 17, 2023
@YOU54F YOU54F closed this Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant