-
Notifications
You must be signed in to change notification settings - Fork 13
Update Rails Parallel Example (consistency sweep) #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,10 @@ | ||
| # Used by the 'Add to Buildkite' button in the readme | ||
| name: "Rails Parallel Example" | ||
| description: "An example pipeline that runs Rails tests in parallel across multiple agents using Knapsack." | ||
| emoji: ":rails:" | ||
| color: "#FDF5F5" | ||
| languages: | ||
| - "Ruby" | ||
| - "Shell" | ||
| steps: | ||
| - label: ":pipeline:" | ||
| command: "buildkite-agent pipeline upload" | ||
| - command: "buildkite-agent pipeline upload" | ||
| label: ":pipeline:" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # Buildkite Rails Parallel Pipeline Example | ||
|
|
||
| [](https://buildkite.com/buildkite/rails-parallel-example/builds/latest?branch=main) | ||
| [](https://buildkite.com/new) | ||
|
|
||
| This repository is an example [Buildkite](https://buildkite.com/) pipeline that demonstrates how to run 20 parallel test jobs for a [Rails](https://rubyonrails.org/) application using [rbenv](https://github.com/rbenv/rbenv) and [Knapsack](https://github.com/ArturT/knapsack). | ||
|
|
||
| 👉 **See this example in action:** [buildkite/rails-parallel-example](https://buildkite.com/buildkite/rails-parallel-example/builds/latest?branch=main) | ||
|
|
||
| [](https://buildkite.com/new) | ||
|
|
||
| <a href="https://buildkite.com/buildkite/rails-parallel-example/builds/latest?branch=main"> | ||
| <img width="1491" alt="Screenshot of Buildkite Rails Parallel example pipeline" src=".buildkite/screenshot.png" /> | ||
| </a> | ||
|
|
||
| <!-- docs:start --> | ||
|
|
||
| ## How it works | ||
|
|
||
| This repository doesn't include a Rails application, but it does provide the CI setup you'd typically use in a real project. | ||
|
|
||
| In the [pipeline configuration file](.buildkite/pipeline.yml) the `parallelism` property for the test step is set to 20. When a build, the step will appear 20 times in the pipeline, each with different environment variables exposed so you can divvy up your test suite accordingly. You can then run 20 agents (on the same machine, or distributed) to work on the 20 jobs in parallel. | ||
|
|
||
| This example: | ||
| - Uses rbenv to manage Ruby versions | ||
| - Runs tests in parallel using Buildkite’s `parallelism` feature | ||
| - Leverages [Knapsack](https://github.com/ArturT/knapsack) for test suite splitting | ||
| - Can be used on a single machine or distributed across multiple agents | ||
|
|
||
| Key files: | ||
| - [.buildkite/hooks/environment](.buildkite/hooks/environment): loads rbenv before steps run | ||
| - [.buildkite/pipeline.yml](.buildkite/pipeline.yml): sets up parallel steps | ||
| - [scripts/ci/setup.sh](scripts/ci/setup.sh): prepares the CI environment | ||
| - [scripts/ci/parallel_specs.sh](scripts/ci/parallel_specs.sh): runs specs via Knapsack | ||
|
|
||
| ### Parallel test execution | ||
|
|
||
| The test step in [`pipeline.yml`](.buildkite/pipeline.yml) is configured like this: | ||
|
|
||
| ```yml | ||
| steps: | ||
| - name: ":rspec:" | ||
| command: "scripts/ci/setup.sh && scripts/ci/parallel_specs.sh" | ||
| parallelism: 20 | ||
| ``` | ||
|
|
||
| Each parallel job receives unique environment variables (`BUILDKITE_PARALLEL_JOB`, `BUILDKITE_PARALLEL_JOB_COUNT`) to help split the workload. | ||
|
|
||
| See the [parallelizing builds guide](https://buildkite.com/docs/guides/parallelizing-builds) for more information to create parallelized and distributed builds with Buildkite. | ||
|
|
||
| > 💡 Want to use Docker? Try the [Rails Docker Parallel Example](https://github.com/buildkite/rails-docker-parallel-example) | ||
|
|
||
| ## Knapsack Pro | ||
|
|
||
| Consult the [Knapsack documentation](https://github.com/ArturT/knapsack) for configuring your database and dependent services to support running parallel steps on the one machine. | ||
|
|
||
| See [Knapsack Pro version of this example pipeline](https://github.com/KnapsackPro/buildkite-rails-parallel-example-with-knapsack_pro) for how to perform dynamic splits/allocation of test nodes. | ||
|
|
||
| <!-- docs:end --> | ||
|
|
||
| ## License | ||
|
|
||
| See [LICENSE.md](LICENSE.md) (MIT) | ||
This file was deleted.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.