Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Maintainability Test Coverage Gem Version

RSpec Tracer is a specs dependency analyzer, flaky tests detector, tests accelerator, and coverage reporter tool. It maintains a list of files for each test, enabling itself to skip tests in the subsequent runs if none of the dependent files are changed.

It uses Ruby's built-in coverage library to keep track of the coverage for each test. For each test executed, the coverage diff provides the desired file list. RSpec Tracer takes care of reporting the correct code coverage when skipping tests by using the cached reports. Also, note that it will never skip:

  • Flaky examples
  • Failed examples
  • Pending examples

Knowing the examples and files dependency gives us a better insight into the codebase, and we have a clear idea of what to test for when making any changes. With this data, we can also analyze the coupling between different components and much more.

RSpec Tracer requires Ruby 2.5+ and rspec-core >= 3.6.0. To use with Rails 5+, make sure to use rspec-rails >= 4.0.0. If you are using SimpleCov, it is recommended to use simplecov >= 0.12.0. To use RSpec Tracer cache on CI, you need to have an S3 bucket and AWS CLI installed.

You should take some time and go through the document describing the intention and implementation details of managing dependency, managing flaky tests, skipping tests, and caching on CI.

Table of Contents


First Run

Next Run

You get the following three reports:

All Examples Report

These reports provide basic test information:

First Run

Next Run

Duplicate Examples Report

These reports provide duplicate tests information.

Flaky Examples Report

These reports provide flaky tests information. Assuming the following two tests failed in the first run.

Next Run

Another Run

Examples Dependency Report

These reports show a list of dependent files for each test.

Files Dependency Report

These reports provide information on the total number of tests that will run after changing this particular file.

Getting Started

  1. Add this line to your Gemfile and bundle install:

    gem 'rspec-tracer', '~> 0.9', group: :test, require: false

    And, add the followings to your .gitignore:

  2. Load and launch RSpec Tracer at the very top of spec_helper.rb (or rails_helper.rb, test/test_helper.rb). Note that RSpecTracer.start must be issued before loading any of the application code.

    # Load RSpec Tracer
    require 'rspec_tracer'

    If you are using SimpleCov, load RSpec Tracer right after the SimpleCov load and launch:

    require 'simplecov'
    # Load RSpec Tracer
    require 'rspec_tracer'

    If you use RSpec Tracer with SimpleCov, then SimpleCov would not report branch coverage results even when enabled.

  3. After running your tests, open rspec_tracer_report/index.html in the browser of your choice.

Working with JRuby

It is recommend to use JRuby Also, configure it with JRUBY_OPTS="--debug -X+O" or have the .jrubyrc file:


Configuring CI Caching

To enable RSpec Tracer to share cache between different builds on CI, update the Rakefile in your project to have the following:

spec = Gem::Specification.find_by_name('rspec-tracer')

load "#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile"

Before running tests, download the remote cache using the following rake task:

bundle exec rake rspec_tracer:remote_cache:download

After running tests, upload the local cache using the following rake task:

bundle exec rake rspec_tracer:remote_cache:upload

You must set the following two environment variables:

  • GIT_BRANCH is the git branch name you are running the CI build on.
  • RSPEC_TRACER_S3_URI is the S3 bucket path to store the cache files.
    export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache

Advanced Configuration

Configuration settings can be applied in three formats, which are completely equivalent:

  • The most common way is to configure it directly in your start block:

    RSpecTracer.start do
      config_option 'foo'
  • You can also set all configuration options directly:

    RSpecTracer.config_option 'foo'
  • If you do not want to start tracer immediately after launch or want to add additional configuration later on in a concise way, use:

    RSpecTracer.configure do
      config_option 'foo'

The available configuration options are:

  • root dir to set the project root. The default value is the current working directory.
  • add_filter filter to apply filters on the source files to exclude them from the dependent files list.
  • filters.clear to remove the default configured dependent files filters.
  • add_coverage_filter filter to apply filters on the source files to exclude them from the coverage report.
  • coverage_filters.clear to remove the default configured coverage files filters.
  • coverage_track_files glob to include files in the given glob pattern in the coverage report if these files are not already present.
RSpecTracer.start do
  # Configure project root
  root '/tmp/my_project'

  # Clear existing filters
  # Add dependent files filter
  add_filter %r{^/tasks/}

  # Clear existing coverage filters
  # Add coverage files filter
  add_coverage_filter %w[/features/ /spec/ /tests/]

  # Define glob to track files in the coverage report
  coverage_track_files '{app,lib}/**/*.rb'

You can configure the RSpec Tracer reports directories using the following environment variables:

  • RSPEC_TRACER_CACHE_DIR to update the default cache directory (rspec_tracer_cache).
    export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_cache
  • RSPEC_TRACER_COVERAGE_DIR to update the default coverage directory (rspec_tracer_coverage).
    export RSPEC_TRACER_COVERAGE_DIR=/tmp/rspec_tracer_coverage
  • RSPEC_TRACER_REPORT_DIR to update the default html reports directory (rspec_tracer_report).
    export RSPEC_TRACER_REPORT_DIR=/tmp/rspec_tracer_report

These settings are available through environment variables because the rake tasks to download and upload the cache files need to use the same directories.


By default, RSpec Tracer ignores all the files outside of the project root directory - otherwise you would end up with the source files in the gems you are using in the project. It also applies the following filters:

RSpecTracer.configure do
  add_filter '/vendor/bundle/'

  add_coverage_filter %w[

Defining Custom Filteres

You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source file's name relative to the project root), a block or by passing in your own Filter class.

  • String Filter: The string filter matches files that have the given string in their name. For example, the following string filter will remove all files that have "/helpers/" in their name.

    RSpecTracer.start do
      add_filter '/helpers/'
  • Regex Filter: The regex filter removes all files that have a successful name match against the given regex expression. This simple regex filter will remove all files that start with %r{^/helper/} in their name:

    RSpecTracer.start do
      add_filter %r{^/helpers/}
  • Block Filter: Block filters receive a Hash object and expect your block to return either true (if the file is to be removed from the result) or false (if the result should be kept). In the below example, the filter will remove all files that match "/helpers/" in their path.

    RSpecTracer.start do
      add_filter do |source_file|

    You can also use source_file[:name] to define the return value of the block filter for the given source file.

  • Array Filter: You can pass in an array containing any of the other filter types:

    RSpecTracer.start do
      add_filter ['/helpers/', %r{^/utils/}]

Environment Variables

To get better control on execution, you can use the following environment variables whenever required.

  • LOCAL_AWS (default: false): In case you want to test out the caching feature in the local development environment. You can install localstack and awscli-local and then invoke the rake tasks with LOCAL_AWS=true.

  • RSPEC_TRACER_FAIL_ON_DUPLICATES (default: true): By default, RSpec Tracer exits with one if there are duplicate examples.

  • RSPEC_TRACER_NO_SKIP (default: false): Use this environment variables to not skip any tests. Note that it will continue to maintain cache files and generate reports.

  • RSPEC_TRACER_UPLOAD_LOCAL_CACHE (default: false): By default, RSpec Tracer does not upload local cache files. You can set this environment variable to true to upload the local cache to S3.

  • RSPEC_TRACER_VERBOSE (default: false): To print the intermediate steps and time taken, use this environment variable.

  • TEST_SUITES: Set this environment variable when running parallel builds in the CI. It determines the total number of different test suites you are running.

    export TEST_SUITES=8
  • TEST_SUITE_ID: If you have a large set of tests to run, it is recommended to run them in separate groups. This way, RSpec Tracer is not overwhelmed with loading massive cached data in the memory. Also, it generates and uses cache for specific test suites and not merge them.

    TEST_SUITE_ID=1 bundle exec rspec spec/models
    TEST_SUITE_ID=2 bundle exec rspec spec/helpers

Duplicate Examples

To uniquely identify the examples is one of the requirements for the correctness of the RSpec Tracer. Sometimes, it would not be possible to do so depending upon how we have written the specs. The following attributes determine the uniqueness:

  • The example group
  • The example full description
  • The spec file location, i.e., file name and line number
  • All the shared examples and contexts

Consider the following Calculator module:

module Calculator

  def add(a, b) a + b; end
  def sub(a, b) a - b; end
  def mul(a, b) a * b; end

And the corresponding spec file spec/calculator_spec.rb:

RSpec.describe Calculator do
  describe '#add' do
      [1, 2, 3],
      [0, 0, 0],
      [5, 32, 37],
      [-1, -8, -9],
      [10, -10, 0]
    ].each { |a, b, r| it { expect(described_class.add(a, b)).to eq(r) } }

  describe '#sub' do
      [1, 2, -1],
      [10, 0, 10],
      [37, 5, 32],
      [-1, -8, 7],
      [10, 10, 0]
    ].each do |a, b, r|
      it 'performs subtraction' do
        expect(described_class.sub(a, b)).to eq(r)

  describe '#mul' do
      [1, 2, -2],
      [10, 0, 0],
      [5, 7, 35],
      [-1, -8, 8],
      [10, 10, 100]
    ].each do |a, b, r|
      it "multiplies #{a} and #{b} to #{r}" do
        expect(described_class.mul(a, b)).to eq(r)

Running the spec with bundle exec rspec spec/calculator_spec.rb generates the following output:

    multiplies 5 and 7 to 35
    multiplies 10 and 10 to 100
    multiplies 10 and 0 to 0
    multiplies 1 and 2 to -2 (FAILED - 1)
    multiplies -1 and -8 to 8
    example at ./spec/calculator_spec.rb:13
    example at ./spec/calculator_spec.rb:13
    example at ./spec/calculator_spec.rb:13
    example at ./spec/calculator_spec.rb:13
    example at ./spec/calculator_spec.rb:13
    performs subtraction
    performs subtraction
    performs subtraction
    performs subtraction
    performs subtraction

In this scenario, RSpec Tracer cannot determine the Calculator#add and Calculator#sub group examples.

RSpec tracer could not uniquely identify the following 10 examples:
  - Example ID: eabd51a899db4f64d5839afe96004f03 (5 examples)
      * Calculator#add (spec/calculator_spec.rb:13)
      * Calculator#add (spec/calculator_spec.rb:13)
      * Calculator#add (spec/calculator_spec.rb:13)
      * Calculator#add (spec/calculator_spec.rb:13)
      * Calculator#add (spec/calculator_spec.rb:13)
  - Example ID: 72171b502c5a42b9aa133f165cf09ec2 (5 examples)
      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)


Read the contribution guide.


The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Rspec Tracer project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the Code of Conduct.