Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Automatic API compatibility validation #105
Producers and consumers of software libraries don’t have an easy life. As a consumer of a library I don’t know if new version of the library brings API incompatibilities until I try to use it and it blows up, hopefully during compilation or testing, and not on production! As a producer of a library I can unintentionally sneak in API incompatibilities, unless I am very careful and/or great engineers review my code and spot the problems.
Let’s make the life of producers and consumers of libraries delightful!!!
What if producer gets nice report or even build failure if he changes the API without bumping major version? What if consumer gets nice report, or even early build failure if known API incompatibilities exist in the new version of a library he attempts to use? What if the incompatibilities are computable statically and it is not necessary to compile code/run tests to identify issues? What if the tools can tell you with good accuracy what version of a library you can consume safely?
While we don’t necessarily need to scope the entire solution we can to scope down an initial feature set that can help us validate if the problem is worth solving and how. Below features are suggestions and should be critically reviewed by whoever decides to own this feature.
Does my change break API compatibility?
To get started, we can create a Gradle task that compares 2 binaries, identifies API incompatibilities and reports them. Tools that identify API incompatibilities already exist and can be leveraged (mockito/mockito#738). In the project we already have code that pulls down the previously released binaries (implemented as part of #84).
Suggested implementation should be enough to get started. However, please discuss and suggest different approach as you see fit.
Below future ideas are wild brainstorming :)
I've came across this by a complete accident, but you might want to take a look at what I'm doing with https://github.com/revapi/revapi.
There's no gradle plugin for it just now, but the maven plugin does quite a bit:
If you find it interesting, I would love to answer any questions you might have!
Japicmp is a good tool for detecting binary incompatibilities but IMHO is a little bit lacking in terms of pluggability and extensibility.
The above mentioned https://diff.revapi.org uses a custom "reporter" to output the findings in json that is then consumed by the webpage. The Spoon project for example uses Revapi to check the API changes in all of their PRs and uses a single Freemarker template (used by Revapi) to produce nice reports such as one at INRIA/spoon#1771.
This would have been much harder with japicmp.
Also japicmp seems to be focused on binary compatibility and doesn't detect as many source incompatibilities as Revapi does as far as I know..
On the other hand, japicmp seems to have a larger community..
@metlos Japicmp looks like exactly what is needed, it even comes with a
Config options in Gradle Japicmp that fit requested features:
referenced this issue
Dec 20, 2017
I'm obviously biased because I'm the author of Revapi, but :-)
While japicmp certainly fits the bill, I really do think that Revapi might be a better choice because of its pluggable design. Japicmp is a single purpose tool with not much choice when it comes to reporting, while Revapi was from the beginning designed with pluggable"reporters".
I also think that reducing the compatibility to just binary compatibility is not the right choice in today's containerized world where stuff gets rebuilt from source. Revapi seems to have a richer set of detected problems in both binary and especially source area. Uniquely, Revapi is able to detect problems like nonpublic classes sneaking into the API through its use chain tracking abilities.
Japicmp on the other hand has a Gradle plugin that Revapi lacks as of now.
All in all I think both solutions would fit your use case, each with its own set of drawbacks ;-)
Whatever you choose in the end I'm very much excited about the prospect of shipkit...
@mockitoguy I've been mulling over this feature for some time now and I feel that implementing it may require a fundamental change in the way versions are incremented. This is how I see Shipkit currently incrementing versions:
Japicmp/Revapi can determine the
I can see several ways around this issue:
Do you have any other solutions for this? Which do you think is the most promising?