Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI against Ruby 2.7.0-dev #1285

Open
wants to merge 1 commit into
base: master
from
Open

CI against Ruby 2.7.0-dev #1285

wants to merge 1 commit into from

Conversation

@koic
Copy link
Contributor

koic commented Nov 1, 2019

Summary

CI fails as follows when using Ruby 2.7.0-dev.

% ruby 2.7.0dev (2019-10-31T08:11:23Z master 5f6fbf8725) [x86_64-darwin17]
% bundle exec rake
(snip)

Finished in 6.71 seconds (files took 1.19 seconds to load)
1910 examples, 29 failures, 3 pending

The cause of failures has not been identified, but changes in Ripper or something may have affected it.

First of all, it is opened as a PR so that CI failure can be shown.

Other Information

I noticed this issue because RuboCop documentation generation failed when using Ruby 2.7.0-dev.

e.g. https://circleci.com/gh/rubocop-hq/rubocop-performance/855

### Summary

CI fails as follows when using Ruby 2.7.0-dev.

```console
% ruby 2.7.0dev (2019-10-31T08:11:23Z master 5f6fbf8725) [x86_64-darwin17]
% bundle exec rake
(snip)

Finished in 6.71 seconds (files took 1.19 seconds to load)
1910 examples, 29 failures, 3 pending
```

The cause of failures has not been identified, but changes in Ripper
or something may have affected it.

First of all, it is opened as a PR so that CI failure can be shown.

## Other Information

I noticed this issue because RuboCop documentation generation failed
when using Ruby 2.7.0-dev.

e.g. https://circleci.com/gh/rubocop-hq/rubocop-performance/855
@coveralls

This comment has been minimized.

Copy link

coveralls commented Nov 1, 2019

Coverage Status

Coverage decreased (-0.4%) to 93.372% when pulling 6f90499 on koic:ci_against_ruby_head into 4b5c875 on lsegal:master.

@kaclak
kaclak approved these changes Nov 22, 2019
@MSP-Greg

This comment has been minimized.

Copy link
Contributor

MSP-Greg commented Nov 24, 2019

@koic

I just ran Travis CI w/ruby-head in my fork with the code from #1290.

Passed, see https://travis-ci.org/MSP-Greg/yard/jobs/616413468. Hence, it's passing using 2.7.0 on both Windows & Ubuntu...


matrix:
allow_failures:
- rvm: ruby-head

This comment has been minimized.

Copy link
@lsegal

lsegal Nov 24, 2019

Owner

I would generally say that allowing failures wouldn't be much more helpful than not running it since it's unlikely any failures would get detected and fixed in this way.

This comment has been minimized.

Copy link
@MSP-Greg

MSP-Greg Nov 25, 2019

Contributor

So generally, does that mean that:

You'll accept testing with ruby-head without allow-failure

or

You don't want to test against ruby-head

This comment has been minimized.

Copy link
@lsegal

lsegal Nov 25, 2019

Owner

We've previously had to turn ruby-head off because of weird temporal issues that were not baked behavior (i.e. not YARD issues).

That was a while ago. If things are more stable now I'd consider it given the benefit of keeping up with ripper changes.

I'm still unclear on whether this will just end up getting turned off again.

This comment has been minimized.

Copy link
@lsegal

lsegal Nov 25, 2019

Owner

If there was a way to support pre-releases or RCs without directly testing head I'd rather see that happen. That would provide a lot more stability and meaningfulness to the test results.

This comment has been minimized.

Copy link
@MSP-Greg

MSP-Greg Nov 25, 2019

Contributor

The Windows ruby-head build I maintain is at build 2687. A few years ago it was built twice a day, then switched to three times a day. Before that, I was building locally, and pushing to BinTray. It has been used in several repos during the last two years.

Given that, I would suggest trying CI against ruby-head. I personally don't consider the pre-releases or RCs of much use, as they aren't really based on any criteria.

Just like any other repo, if CI is failing in Ruby core, people get irritated. Hence, failures are usually fixed in short order.

JFYI, I believe the builds available on Travis are not tested. The windows build is setup to use the latest build that passes all the Ruby core test suites.

Feel free to ping me if ruby-head fails.

This comment has been minimized.

Copy link
@koic

koic Nov 25, 2019

Author Contributor

Thanks for your comments.

Given that, I would suggest trying CI against ruby-head. I personally don't consider the pre-releases or RCs of much use, as they aren't really based on any criteria.

Yeah, I think so and opened this PR. Release candidates are a snapshot of the trunk at a specific point.
And I think it's also useful to indicate failure using a persistent URL.

This comment has been minimized.

Copy link
@lsegal

lsegal Nov 26, 2019

Owner

I would suggest trying CI against ruby-head. I personally don't consider the pre-releases or RCs of much use, as they aren't really based on any criteria.

I would disagree here. YARD doesn't support ruby-head because it's not at all battle tested and subject to change. It might be a useful notification of what's to come, but we wouldn't be accepting any changes to YARD code just because something is failing on Ruby master; it's too early to tell if the change is intended behavior or a regression, and YARD isn't going to be tracking against unreleased Ruby versions.

For that reason, .pre or .rc releases are a better indicator that behavior is locked in and expected to ship.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.