Split RSpec test files by test examples#102
Conversation
ArturT
force-pushed
the
rspec-split-by-test-examples
branch
from
cc3963f to
753152f
Compare
…stead of default formatter progress
…ew encrypted RSPec test example paths?
…O_RSPEC_SPLIT_BY_TEST_EXAMPLES=true
…ACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true then detect test examples from json report
|
|
||
| # generate RSpec JSON report in separate process to not pollute RSpec state | ||
| cmd = 'bundle exec rake knapsack_pro:rspec_test_example_detector' | ||
| Kernel.system(cmd) |
There was a problem hiding this comment.
I wonder if we should have a backup in case this fails. We could react here differently depending on the return value. (For reference, according to the docs: true means zero exit status, false - non-zero status and nil means that the execution failed).
There was a problem hiding this comment.
Sorry, I just saw we are raising an error in the other class in such a case, so scratch that.^
| # if user didn't provide the format then use explicitly default progress formatter | ||
| # in order to avoid KnapsackPro::Formatters::RSpecQueueSummaryFormatter being the only default formatter | ||
| cli_args += ['--format', 'progress'] unless cli_args.include?('--format') | ||
| if !cli_args.include?('--format') && !cli_args.include?('-f') |
|
|
||
| # generate RSpec JSON report in separate process to not pollute RSpec state | ||
| cmd = 'bundle exec rake knapsack_pro:rspec_test_example_detector' | ||
| unless Kernel.system(cmd) |
There was a problem hiding this comment.
@shadre I added it anyway. detector.test_file_example_paths method can raise when json file missing.
| hash_report | ||
| .fetch('examples') | ||
| .map { |e| e.fetch('id') } | ||
| .map { |path| test_file_hash_for(path) } |
There was a problem hiding this comment.
What do you think about calling the block variable path_with_example_id just so that it's clear?
There was a problem hiding this comment.
I'm not sure what do you mean?
Something like that
.map(&:test_file_hash_for)
There was a problem hiding this comment.
I just meant substituting |path| with |path_with_example_id| :)
…#generate_json_report
ArturT
changed the title
2 participants
Split test files by test cases
Note: this is an experimental feature. It works for Regular Mode and Queue Mode. For large test suite with a few thousand test files, it may generate too many RSpec test example paths that may lead to too large JSON payload in request to Knapsack Pro API and this could trigger the API timeout.
Please give us feedback so we could improve the feature.
https://knapsackpro.com/contact
How it works: You can split slow test file by test cases. Thanks to that the test file can be split across parallel CI nodes because test cases from the test file will run on different CI nodes.
This is helpful when you have one or a few very slow test files that are a bottleneck for CI build speed and you don't want to manually create a few smaller test files from the slow test files. Instead, you can tell
knapsack_progem to split your test files by test cases across parallel CI nodes.RSpec split test files by test examples (by individual
its)In order to split RSpec test files by test examples across parallel CI nodes you need to set flag:
Thanks to that your CI build speed can be faster. We recommend using this feature with Queue Mode to ensure parallel CI nodes finish work at a similar time which gives you the shortest CI build time.
Doing tests split by test examples can generate a lot of logs by
knapsack_progem in Queue Mode. We recommend to set log level to:Story https://trello.com/c/qfUaxAg4 (gem maintainers only)