Skip to content

Commit

Permalink
Merge pull request #395 from danmayer/coverband_5_readme_updates
Browse files Browse the repository at this point in the history
README modifications
  • Loading branch information
danmayer committed Aug 30, 2020
2 parents 8045394 + 1f5e55d commit 830e7db
Showing 1 changed file with 59 additions and 49 deletions.
108 changes: 59 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
<p align="center">
<a href="#key-features">Key Features</a> •
<a href="#installation">Installation</a> •
<a href="#coverage-report">Coverage Report</a> •
<a href="#coverband-web-ui">Coverband Web UI</a> •
<a href="#advanced-config">Advanced Config</a> •
<a href="#license">License</a> •
<a href="/changes.md">Change Log / Roadmap</a> •
Expand All @@ -28,7 +28,7 @@ The primary goal of Coverband is giving deep insight into your production runtim
- Low performance overhead
- Simple setup and configuration
- Out of the box support for all standard code execution paths (web, cron, background jobs, rake tasks, etc)
- Splits load time (Rails eager load) and Run time metrics
- Splits load time (Rails eager load) and runtime metrics
- Easy to understand actionable insights from the report
- Development mode, offers deep insight of code usage details (number of LOC execution during single request, etc) during development.
- Mountable web interface to easily share reports
Expand All @@ -41,7 +41,7 @@ Coverband stores coverage data in Redis. The Redis endpoint is looked for in thi

1. `ENV['COVERBAND_REDIS_URL']`
2. `ENV['REDIS_URL']`
3. `localhost`
3. `localhost:6379`

The redis store can also be explicitly defined within the coverband.rb. See [advanced config](#advanced-config).

Expand All @@ -53,8 +53,6 @@ Add this line to your application's `Gemfile`, remember to `bundle install` afte
gem 'coverband'
```

## Upgrading to Latest

### No custom code or middleware required

With older versions of coverband, projects would report to redis using rack or sidekiq middleware. After coverband 4.0, this should no longer be required and could cause performance issues. Reporting to redis is now automatically done within a background thread with no custom code needed.
Expand All @@ -67,7 +65,7 @@ The Railtie integration means you shouldn't need to do anything else other than

## Sinatra

For the best coverage you want this loaded as early as possible. I have been putting it directly in my `config.ru` but you could use an initializer, though you may end up missing some boot up coverage. To start collection require Coverband as early as possible.
For the best coverage you want this loaded as early as possible. We recommend requiring cover band directly in the `config.ru`. Requiring coverband within an initializer could also work, but you may end up missing some boot up coverage. To start collection require Coverband as early as possible.

```ruby
require 'coverband'
Expand All @@ -77,7 +75,33 @@ use Coverband::BackgroundMiddleware
run ActionController::Dispatcher.new
```

# Coverage Report
## Coverband Web UI

![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_web_ui.png)

> The web index as available on the [Coverband Demo site](https://coverband-demo.herokuapp.com/coverage?#_Coverage)
- View overall coverage information

- Drill into individual file coverage

- View individual file details

- Clear Coverage - disabled by default as it could be considered a danager operation in production. Enable with `config.web_enable_clear` or leave off and clear from [rake task](#clear-coverage).

- Clear coverage report

This will clear the coverage data. This wipes out all collected data.

- Clear individual file coverage

This will clear the details of the file you are looking at. This is helpful if you don't want to lose all Coverage data but made a change that you expect would impact a particular file.

- Force coverage collection

This triggers coverage collection on the current webserver process. Useful in development but confusing in production environments where many ruby processes are usually running.

### Mounting as a Rack App

Coverband comes with a mountable rack app for viewing reports. For Rails this can be done in `config/routes.rb` with:

Expand All @@ -97,49 +121,23 @@ Rails.application.routes.draw do
end
```

or you can enable basic auth by setting `ENV['COVERBAND_PASSWORD']` or via your configuration `config.password = 'my_pass'`

### Coverband Web Endpoint

The web endpoint is a barebones endpoint that you can either expose direct (after authentication) or you can just link to the actions you wish to expose. The index is intended as a example to showcase all the features.

![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_web_ui.png)

> The web index as available on the Coverband Demo site
- **force coverage collection:** This triggers coverage collection on the current webserver process
- **clear coverage report:** This will clear the coverage data. This wipes out all collected data (**dangerous**)
- View individual file details
- **clear individual file coverage:** This will clear the details of the file you are looking at. This is helpful if you don't want to lose all Coverage data but made a change that you expect would impact a particular file.

### JRuby Support

Coverband is compatible with JRuby. If you want to run on JRuby note that I haven't benchmarked and I believe the perf impact on older versions of JRuby could be significant, improved Coverage support is in [JRuby master](https://github.com/jruby/jruby/pull/6180), and will be in the next release.
or you can enable basic auth by setting `ENV['COVERBAND_PASSWORD']` or via your configuration `config.password = <YOUR_COMPLEX_UNGUESSABLE_PASSWORD>`

- older versions of JRuby need tracing enabled to work (and this could cause bad performance)
- run Jruby with the `--debug` option
- add into your `.jrubyrc` the `debug.fullTrace=true` setting
- For best performance the `oneshot_lines` is recommended, and in the latest releases should have very low overhead
- See JRuby support in a Rails app configured to run via JRuby, in [Coverband Demo](https://github.com/coverband-service/coverband_demo)
- JRuby is tested via CI against Rails 5 and 6
### Standalone

### Rake Tasks
The coverage server can also be started standalone with a rake task:

The rake task generates a report locally and opens a browser pointing to `coverage/index.html`.

`rake coverband:coverage`

This is mostly useful in your local development environment.

##### Example Output
```
bundle exec rake coverband:coverage_server
```

Since Coverband is [Simplecov](https://github.com/colszowka/simplecov) output compatible it should work with any of the `SimpleCov::Formatter`'s available. The output below is produced using the default Simplecov HTML formatter.
The web UI should then be available here: http://localhost:1022/

Index Page
![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_index.png)
This is especially useful for projects that are api only and cannot support the mounted rack app. To get production coverage, point coverband at your production redis server and ensure to checkout the production version of your project code locally.

Details on an example Sinatra app
![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_details.png)
```
COVERBAND_REDIS_URL=redis://username:password:redis_production_server:2322 bundle exec rake coverband:coverage_server
```

# Coverband Demo

Expand Down Expand Up @@ -245,19 +243,18 @@ end

### Avoiding Cache Stampede

If you have many servers and they all hit Redis at the same time you can see spikes in your Redis CPU, and memory. This is do to a concept called [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede). It is better to spread out the reporting across your servers. A simple way to do this is to add a random wiggle on your background reporting. This configuration option allows a wiggle. The right amount of wiggle depends on the numbers of servers you have and how willing you are to have delays in your coverage reporting. I would recommend at least 1 second per server.
If you have many servers and they all hit Redis at the same time you can see spikes in your Redis CPU, and memory. This is do to a concept called [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede). It is better to spread out the reporting across your servers. A simple way to do this is to add a random wiggle on your background reporting. This configuration option allows a wiggle. The right amount of wiggle depends on the numbers of servers you have and how willing you are to have delays in your coverage reporting. I would recommend at least 1 second per server. Note, the default wiggle is set to 30 seconds.

Add a wiggle (in seconds) to the background thread to avoid all your servers reporting at the same time:

`config.reporting_wiggle = 30`

### Redis Hash Store

Coverband on very high volume sites with many server processes reporting can have a race condition. To resolve the race condition and reduce Ruby memory overhead we have introduced a new Redis storage option. This moves the some of the work from the Ruby processes to Redis. It is worth noting because of this, it has a larger demands on the Redis server. So adjust your Redis instance accordingly. To help reduce the extra redis load you can also change the background reporting time period.
Coverband on very high volume sites with many server processes reporting can have a race condition which can cause hit counts to be inaccurate. To resolve the race condition and reduce Ruby memory overhead we have introduced a new Redis storage option. This moves the some of the work from the Ruby processes to Redis. It is worth noting because of this, it has a larger demands on the Redis server. So adjust your Redis instance accordingly. To help reduce the extra redis load you can also change the background reporting frequency.

- set the new Redis store: `config.store = Coverband::Adapters::HashRedisStore.new(Redis.new(url: redis_url))`
- adjust from default 30s reporting `config.background_reporting_sleep_seconds = 120`
- reminder it is recommended to have a unique Redis per workload (background jobs, caching, Coverband), for this store, it may be more important to have a dedicated Redis.
- Use a dedicated coverband redis instance: `config.store = Coverband::Adapters::HashRedisStore.new(Redis.new(url: redis_url))`
- Adjust from default 30s reporting `config.background_reporting_sleep_seconds = 120`

See more discussion [here](https://github.com/danmayer/coverband/issues/384).

Expand All @@ -267,6 +264,8 @@ Now that Coverband uses MD5 hashes there should be no reason to manually clear c

`rake coverband:clear`

This can also be done through the web if `config.web_enable_clear` is enabled.

### Coverage Data Migration

Between the release of 4.0 and 4.1 our data format changed. This resets all your coverage data. If you want to restore your previous coverage data, feel free to migrate.
Expand Down Expand Up @@ -341,6 +340,17 @@ We will match Heroku & Ruby's support lifetime, supporting the last 3 major Ruby

For Rails, we will follow the policy of the [Rails team maintenance policy](https://guides.rubyonrails.org/maintenance_policy.html). We officially support the last two major release versions, while providing minimal support (major bugs / security fixes) for an additional version. This means at the moment we primaryly target Rails 6.x, 5.x, and will try to keep current functionality working for Rails 4.x but may release new features that do not work on that target.

### JRuby Support

Coverband is compatible with JRuby. If you want to run on JRuby note that I haven't benchmarked and I believe the perf impact on older versions of JRuby could be significant. Improved Coverage support is in [JRuby master](https://github.com/jruby/jruby/pull/6180), and will be in the next release.

- older versions of JRuby need tracing enabled to work (and this could cause bad performance)
- run Jruby with the `--debug` option
- add into your `.jrubyrc` the `debug.fullTrace=true` setting
- For best performance the `oneshot_lines` is recommended, and in the latest releases should have very low overhead
- See JRuby support in a Rails app configured to run via JRuby, in [Coverband Demo](https://github.com/coverband-service/coverband_demo)
- JRuby is tested via CI against Rails 5 and 6

# Contributing To Coverband

If you are working on adding features, PRs, or bugfixes to Coverband this section should help get you going.
Expand Down

0 comments on commit 830e7db

Please sign in to comment.