-
Notifications
You must be signed in to change notification settings - Fork 173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add gem version and SemVer compatibility badges to README #473
Conversation
So dependabot upgrades other people's packages automatically and then if their CI fails it decreases our semver score? Even if the failure has nothing to do with us (just flakey tests or whatever)? |
Yep - the idea here is to test SemVer by checking whether updates to bugsnag cause test failures for your users. The "score" is just the percentage of CI runs that pass when updating between SemVer compatible versions, but it still gives a pretty clear indicator that bugsnag updates won't require code changes for your users if/when the upgrade it. There are some difficulties with flaky tests that I haven't 100% ironed out yet (I should be able to filter out flaky test suites based on all the other updates Dependabot is doing for that repo) but in practice it seems flaky tests only affect the score very slightly. Of the 163 updates Dependabot has done of bugsnag only 3 have ever "caused" test failures. |
Yeah - I just don't think that a 98% score is particularly clear to people who don't know what it is and how it works. I think that we'll likely get confused users if we merge this. Maybe you could give a grading system instead, but as it is I dont think it's clearer than us saying we use semver versioning. |
👍, although sorry to hear that. I've renamed the badge to be "semver stability" to make it clearer what it does, and I'm going to try to filter out the flaky specs later today / tomorrow. Keep up the great work with Bugsnag - great product! ❤️ |
You could do something like 95%+ is given an excellent rating? |
The feedback I've had from others is that they like the percentage (generally because it's honest about release processes not being perfect, or because they think it's easily understandable if you click through). It's colour-coded based on the % so there's some "this is good" conditioning in there. I'll definitely take your feedback on board, though - if others tell me the same I'll look at making it less granular. |
Yeah fair - do you have examples of other projects that use it? |
I'll reopen it and see what @kattrali thinks when she is back from pto next week - thanks for all the context! |
@kattrali - any thoughts? The folks at Webpack suggested a version that only displayed updates to the latest version, so there's this option now, too: |
This is pretty cool! The context discussion helps too, so thanks @greysteil, @snmaynard. |
Goal
Make it easier for new users to add
bugsnag
to their Gemfile.Design
Badges that show the version and how SemVer compatible this gem is (very) give users all the information they need to pick the right gem requirement.
This PR adds a version number and SemVer compatibility badge. Data comes from Dependabot (disclosure: I built it).
Here's what the badges look like:
Tests
Discussion
Alternative Approaches
Outstanding Questions
Linked issues
Review
For the submitter, initial self-review:
For the pull request reviewer(s), this changeset has been reviewed for: