Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move codegen into src/ folder #3333

Closed
wants to merge 2 commits into from

Conversation

avantgardnerio
Copy link
Contributor

Which issue does this PR close?

Closes #3332.

Rationale for this change

Described in issue.

What changes are included in this PR?

A change to the generated code location.

Are there any user-facing changes?

no

@github-actions github-actions bot added the core Core datafusion crate label Sep 1, 2022
@andygrove
Copy link
Member

I would love to see code completion working with protobuf generated code. That has been frustrating not being able to do that.

The only concern I have with this change is that cargo clean will no longer remove generated source files. I don't know how much of a problem that will be in practice though.

@avantgardnerio
Copy link
Contributor Author

Example of it working in CLion:

image

@codecov-commenter
Copy link

codecov-commenter commented Sep 1, 2022

Codecov Report

Merging #3333 (8ffc83f) into master (b175f9a) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3333   +/-   ##
=======================================
  Coverage   85.48%   85.48%           
=======================================
  Files         294      294           
  Lines       54115    54100   -15     
=======================================
- Hits        46259    46248   -11     
+ Misses       7856     7852    -4     
Impacted Files Coverage Δ
datafusion/core/tests/sql/timestamp.rs 99.65% <ø> (-0.01%) ⬇️
datafusion/proto/src/lib.rs 93.52% <ø> (ø)
datafusion/expr/src/logical_plan/plan.rs 77.77% <0.00%> (+0.33%) ⬆️
datafusion/core/src/physical_plan/metrics/value.rs 87.43% <0.00%> (+0.50%) ⬆️
datafusion/expr/src/window_frame.rs 93.27% <0.00%> (+0.84%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@avantgardnerio avantgardnerio marked this pull request as ready for review September 1, 2022 21:08
@avantgardnerio
Copy link
Contributor Author

@tustvold @alamb What do you two think? This seems (to me) to be the best of all worlds:

  1. generated code is not checked in
  2. generated code is in source folders so IDEs work again

The only downside is that protoc is required, but that was already a requirement for datafusion. If anyone is interested I can (in another PR), have build.rs download and unzip the appropriate protoc for the platform.

@alamb
Copy link
Contributor

alamb commented Sep 2, 2022 via email

@alamb
Copy link
Contributor

alamb commented Sep 3, 2022

Sorry I ran out of time today to give this a careful look. It is on my list...

There was all sorts of good stuff happening in DataFusion I still haven't fully caught up

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea and I think we should try it. If something about generating the code into the source tree causes unforseen issues we can always roll this back.

Thanks a lot @avantgardnerio

datafusion/proto/build.rs Show resolved Hide resolved
@@ -277,7 +277,9 @@ jobs:
rustup default stable
rustup component add rustfmt
- name: Run
run: ci/scripts/rust_fmt.sh
run: |
echo '' > datafusion/proto/src/generated/datafusion.rs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed so it works on a clean checkout, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI jobs are parallelized, so Lint doesn't get a copy of the files generated by Build, but lint did complain I did a mod datafusion for a file that didn't exist. Since this is a CI-only issue, this seemed like an acceptable fix for CI ... ?

@alamb
Copy link
Contributor

alamb commented Sep 4, 2022

This PR appears to need a merge or rebase against master

Brent Gardner and others added 2 commits September 4, 2022 18:42
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
@alamb
Copy link
Contributor

alamb commented Sep 5, 2022

I believe these changes were also included in #3311 so closing this PR -- @avantgardnerio please let me know if that is incorrect and reopen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core datafusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow IDEs to recognize generated code
4 participants