Skip to content

Rust checks are required + MQ#21941

Open
blaginin wants to merge 1 commit intoapache:mainfrom
blaginin:db/mq-rust
Open

Rust checks are required + MQ#21941
blaginin wants to merge 1 commit intoapache:mainfrom
blaginin:db/mq-rust

Conversation

@blaginin
Copy link
Copy Markdown
Member

@blaginin blaginin commented Apr 29, 2026

Closes #6880
Follow up on #17538 and #21239

#21239 seems to be working fine - time to make all the rust changes required to merge it.

This brings two cool things:

  1. The most important one - the "merge when ready" button now works 🚀
image

You can click this button any time, even right after opening a PR. It will be automatically merged when:

  • all required CI checks (which now include rust) are passing
  • you get an approval

I think this should help quite a bit. A lot of people get an approval, push a nit, and then sit waiting for CI to finish. No more waiting!!

  1. We protect ourselves from logical conflicts: cases where develop was passing before, but someone merged something in the meantime and now your change is broken on develop - and you don't find out until after you merge. We hit this almost every month.

One downside: before, clicking merge would happen instantly. With this PR, you'll have to wait 32 minutes. But you don't actually have to wait - github will do the work itself.

@blaginin blaginin self-assigned this Apr 29, 2026
Comment on lines -36 to -40
paths-ignore:
- "docs/**"
- "**.md"
- ".github/ISSUE_TEMPLATE/**"
- ".github/pull_request_template.md"
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit annoying that we'll run those on every pr even on those that changes docs. but:

  • we don't have many of those anyway
  • i really tried to fix this, but it makes the CI very complicated to install and support in the future

@github-actions github-actions Bot added the development-process Related to development process of DataFusion label Apr 29, 2026
@blaginin blaginin requested a review from alamb April 29, 2026 22:24
@blaginin
Copy link
Copy Markdown
Member Author

blaginin commented Apr 29, 2026

Now it will take 2x16 minutes to perform a merge (16 mins to put the PR in the queue + 16 mins to merge in the queue). I think that's OK since it's all automated. But we can actually make it MUCH faster - 2x6 mins instead.

tg_image_2949983809

You can see that the actual bottleneck is the Mac OS tests - that's because runs-on doesn't support them. My ideas:

  • kill the Mac OS tests. Although every three months or so there's 1-2 PRs that fail JUST on them, so probably not a good idea.
  • run macos tests just for a subset of tests, not everything?
  • introduce an "Optional Mac OS Check": this will be a no-on on push to a PR, but will run when the PR is in the merge queue. So instead of 2x16 mins it'll be just 1x16 mins (only in the merge queue). We can do the same with extended tests, which currently aren't required and only run on develop in full.
  • work with the infra team to get access to more performant github runners. Can take a while but i'm happy to do that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable merge queue in github to avoid commit confliction.

1 participant