Skip to content

Why use Automatic Code Review Tools?

Jump to bottom Edit New page
Ivan Di Francesco edited this page Nov 18, 2019 · 2 revisions

Codebeat

Codebeat is a dynamically growing tool that covers major technologies and programming languages. It has evolved substantially within the last few months and it’s worth noting that the team is open for feedback and implementation of new features suggested by their users. However, there are still some things missing. The tool does not allow performing any security checks (perhaps a “simple” integration with Brakeman would do the job). It also does not support any open-source tools or linters (in fact, we are still using Hound) and does not support analysing CSS/SCSS.

Pros:

  1. support for most of the languages we use (Ruby, JS, Swift, Objective-C),
  2. metrics customization,
  3. measuring tools with own algorithms nicely described in the docs (not just a bunch of open-source projects combined together),
  4. very good support from their team,
  5. small but well-documented API, which facilitates management e.g. it provides accesses to users working on selected projects (via teams),
  6. unique quick wins tab,
  7. codeclimate-test-reporter gem that integrates Codebeat with Simplecov coverage reports,
  8. customer suggestions considered and implemented in the product,
  9. It is a dynamically growing tool,
  10. support for Kotlin and Elixir (beta).

Cons:

  1. still missing some things in the documentation (e.g. explanation of code duplication detection here),
  2. no possible security issues check,
  3. no CSS/SCSS analysis.

Codacy

Codacy has an awesome UI, lots of features and it’s very flexible thanks to dozens of options. There is also a tool (in beta) which allows you to define your own patterns and implement it to be checked automatically. Frankly, it wasn’t easy to find disadvantages of this tool or any clients’ complaints. However, there is still some area for improvement and potential to growth for Codacy.

Pros:

  1. used by big players like Paypal or Adobe,
  2. great and intuitive UI,
  3. the possibility to define issue-based goals to improve the codebase,
  4. checking lots of security issues (like assigning strange values to private APIs which may lead to unexpected app behaviour),
  5. a nice feature of browsing commits and monitoring related issues,
  6. docker analysis,
  7. huge flexibility thanks to disabling/enabling patterns or whole packages and even ignoring certain patterns in selected files,
  8. time to fix estimation for each issue,
  9. small company but growing fast, delivering fresh features frequently,
  10. well-described issues with examples right below each case (no need to browse the documentation to find out why the issue actually occurred).

Cons:

  1. Incomplete documentation in some parts (some images are hard to read and the amount of information is insufficient sometimes),
  2. unintuitive one-page charts to track code quality changes over time (it’s a new feature, maybe not fully implemented yet),
  3. no hotspots or quick wins,
  4. no issues searching, only a few dropdown filters.

CodeClimate

CodeClimate is a well-developed and very stable solution with a great number of features. It has many advantages over its competitors and many big players recommend it as the best option. However, it lacked some crucial functionalities that we required for the Netguru code review process, so we switched to an alternative solution.

Pros:

  1. a great number of supported languages, technologies and frameworks,
  2. used by the biggest players, including Pivotal, New Relic for enterprise and Rails, jQuery for open-source,
  3. very stable,
  4. nice new UI,
  5. well-maintained test coverage feature gem,
  6. browser extensions,
  7. trends charts,
  8. test coverage out of the box,
  9. hotspots - a quick wins list.

Cons:

  1. seemingly an integrated bunch of open-source projects,
  2. pricing - it seems to be the most expensive tool in this comparison,
  3. still unpredictable API (in beta),
  4. no support for Objective-C,
  5. no distinct types for total issues number,
  6. no detailed description of the issue, only a header with source code,
  7. no issue searching/filtering, just a paginated list with all of them

Comparison of Automated Code Review Tools: Codebeat, Codacy, Codeclimate and Scrutinizer

Add a custom footer
Add a custom sidebar
Clone this wiki locally