Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn if include macro fails to include entire file #64284

Open
wants to merge 1 commit into
base: master
from

Conversation

@Mark-Simulacrum
Copy link
Member

commented Sep 8, 2019

This currently introduces an error, mainly because that was just simpler, and I'm not entirely certain if we can introduce a lint without an RFC and such.

This is primarily to get feedback on the approach and overall aim -- in particular, do we think this is helpful? If so, we probably will need lang-team sign off and decide if it should be an error (as currently introduced by this PR), a lint, or a warning.

r? @petrochenkov

cc #35560

Some(panictry!(self.p.parse_expr()))
let r = Some(panictry!(self.p.parse_expr()));
if self.p.token != token::Eof {
self.p.sess.span_diagnostic.span_fatal(

This comment has been minimized.

Copy link
@Centril

Centril Sep 8, 2019

Member

Why fatal?

This comment has been minimized.

Copy link
@Mark-Simulacrum

Mark-Simulacrum Sep 8, 2019

Author Member

As I mentioned in the PR description, I think it's an open question what we want here, a fatal diagnostic was just the easiest thing to immediately implement.

@Centril

This comment has been minimized.

Copy link
Member

commented Sep 8, 2019

Ostensibly we could do what the user wants by switching to parsing a block implicitly and putting a bunch of statements in there. On the other hand, this might actually change the dynamic semantics of some programs (but that's probably a bug-fix if it does...).

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member Author

commented Sep 8, 2019

As far as I know, we can't just switch to parsing it as a block of statements because of how macro expansion works, but I could well be wrong about that.

@rust-highfive

This comment was marked as resolved.

Copy link
Collaborator

commented Sep 8, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-09-08T14:04:18.5712221Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-08T14:04:19.1231077Z ##[command]git config gc.auto 0
2019-09-08T14:04:19.1234231Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-08T14:04:19.1238117Z ##[command]git config --get-all http.proxy
2019-09-08T14:04:19.1240934Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64284/merge:refs/remotes/pull/64284/merge
---
2019-09-08T15:06:55.2664405Z .................................................................................................... 1500/9010
2019-09-08T15:07:01.3497022Z .................................................................................................... 1600/9010
2019-09-08T15:07:14.6085009Z ......................................................i...............i............................. 1700/9010
2019-09-08T15:07:22.7830712Z .................................................................................................... 1800/9010
2019-09-08T15:07:37.8033660Z .............................................iiiii.................................................. 1900/9010
2019-09-08T15:07:49.2113822Z .................................................................................................... 2100/9010
2019-09-08T15:07:51.9008483Z .................................................................................................... 2200/9010
2019-09-08T15:07:55.6027288Z .................................................................................................... 2300/9010
2019-09-08T15:08:03.8255475Z .................................................................................................... 2400/9010
---
2019-09-08T15:11:08.7747209Z ....................................i...............i............................................... 4700/9010
2019-09-08T15:11:20.7531977Z .................................................................................................... 4800/9010
2019-09-08T15:11:27.5177727Z .................................................................................................... 4900/9010
2019-09-08T15:11:38.7041884Z .................................................................................................... 5000/9010
2019-09-08T15:11:44.9406332Z ..................ii.ii............................................................................. 5100/9010
2019-09-08T15:11:55.4317892Z .................................................................................................... 5300/9010
2019-09-08T15:12:05.5932347Z .................................................................................i.................. 5400/9010
2019-09-08T15:12:13.6481149Z .................................................................................................... 5500/9010
2019-09-08T15:12:19.5536481Z .................................................................................................... 5600/9010
2019-09-08T15:12:19.5536481Z .................................................................................................... 5600/9010
2019-09-08T15:12:30.2442665Z ...........................................................................ii...i..ii...........i... 5700/9010
2019-09-08T15:12:55.6672312Z .................................................................................................... 5900/9010
2019-09-08T15:13:05.9877704Z .................................................................................................... 6000/9010
2019-09-08T15:13:05.9877704Z .................................................................................................... 6000/9010
2019-09-08T15:13:12.1997583Z .............................................................................i..ii.................. 6100/9010
2019-09-08T15:13:42.3286823Z .................................................................................................... 6300/9010
2019-09-08T15:13:44.5266327Z ....................................i............................................................... 6400/9010
2019-09-08T15:13:46.7752707Z .................................................................................................... 6500/9010
2019-09-08T15:13:49.4428699Z ........i........................................................................................... 6600/9010
---
2019-09-08T15:18:40.4477551Z  finished in 20.631
2019-09-08T15:18:40.4674552Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:18:40.6368657Z 
2019-09-08T15:18:40.6369791Z running 150 tests
2019-09-08T15:18:44.0381111Z i....iii......iii..iiii....i.............................i..i..................i....i.........ii.i.i 100/150
2019-09-08T15:18:46.1175664Z ..iiii..............i.........iii.i.......ii......
2019-09-08T15:18:46.1176211Z 
2019-09-08T15:18:46.1186830Z  finished in 5.651
2019-09-08T15:18:46.1398038Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:18:46.3122578Z 
---
2019-09-08T15:18:48.4946458Z  finished in 2.354
2019-09-08T15:18:48.5161047Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:18:48.6905867Z 
2019-09-08T15:18:48.6908948Z running 9 tests
2019-09-08T15:18:48.6911995Z iiiiiiiii
2019-09-08T15:18:48.6913071Z 
2019-09-08T15:18:48.6914557Z  finished in 0.175
2019-09-08T15:18:48.7119034Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:18:48.8884926Z 
---
2019-09-08T15:19:07.8357788Z  finished in 19.120
2019-09-08T15:19:07.8523364Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:19:08.0263915Z 
2019-09-08T15:19:08.0264191Z running 123 tests
2019-09-08T15:19:32.5293892Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....ii..........iiii..........i...ii...i.......ii. 100/123
2019-09-08T15:19:37.1391967Z i.i.i......iii.i.....ii
2019-09-08T15:19:37.1394078Z 
2019-09-08T15:19:37.1397366Z  finished in 29.287
2019-09-08T15:19:37.1407666Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:19:37.1407991Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-09-08T15:33:35.2144178Z 
2019-09-08T15:33:35.2145496Z    Doc-tests core
2019-09-08T15:33:40.4703679Z 
2019-09-08T15:33:40.4704763Z running 2397 tests
2019-09-08T15:33:51.6803935Z ......iiiii......................................................................................... 100/2397
2019-09-08T15:34:02.5789657Z ...........................................................................ii....................... 200/2397
2019-09-08T15:34:14.9547328Z .................................................................................................i.. 300/2397
2019-09-08T15:34:28.4206131Z .................................................................................................... 400/2397
2019-09-08T15:34:38.9199343Z .........................................i..i.................iiii.................................. 500/2397
2019-09-08T15:34:59.0869248Z .................................................................................................... 700/2397
2019-09-08T15:35:09.4896362Z .................................................................................................... 800/2397
2019-09-08T15:35:19.9219821Z .................................................................................................... 900/2397
2019-09-08T15:35:30.4559422Z .................................................................................................... 1000/2397
---
2019-09-08T15:40:34.9966160Z 
2019-09-08T15:40:34.9967940Z running 991 tests
2019-09-08T15:40:56.5240671Z i................................................................................................... 100/991
2019-09-08T15:41:08.3312912Z .................................................................................................... 200/991
2019-09-08T15:41:16.7055947Z .................iii......i......i...i......i....................................................... 300/991
2019-09-08T15:41:22.6627641Z .................................................................................................... 400/991
2019-09-08T15:41:30.7494088Z ..................................i..i.................................ii........................... 500/991
2019-09-08T15:41:46.0596628Z .................................................................................................... 700/991
2019-09-08T15:41:46.0596628Z .................................................................................................... 700/991
2019-09-08T15:41:54.2941175Z .................iiii............................................................................... 800/991
2019-09-08T15:42:09.4575259Z .................................................................................................... 900/991
2019-09-08T15:42:17.1620426Z .......................................iiii................................................
2019-09-08T15:42:17.1621956Z 
2019-09-08T15:42:17.1714938Z  finished in 249.419
2019-09-08T15:42:17.1732586Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-08T15:42:17.5085834Z    Compiling term v0.0.0 (/checkout/src/libterm)
---
2019-09-08T15:57:30.0517914Z Building stage2 tool error_index_generator (x86_64-unknown-linux-gnu)
2019-09-08T15:57:30.2560446Z    Compiling same-file v1.0.4
2019-09-08T15:57:30.3723515Z    Compiling walkdir v2.2.7
2019-09-08T15:57:31.8101763Z    Compiling error_index_generator v0.0.0 (/checkout/src/tools/error_index_generator)
2019-09-08T15:57:32.9460804Z error: include macro expected single expression in source
2019-09-08T15:57:32.9462281Z    --> /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/build/error_index_generator-87d009c7e595704d/out/error_0.rs:455:5
2019-09-08T15:57:32.9463672Z 455 | })();
2019-09-08T15:57:32.9463948Z     |     ^
2019-09-08T15:57:32.9465956Z 
2019-09-08T15:57:32.9470884Z error: aborting due to previous error
---
2019-09-08T15:57:32.9685469Z == clock drift check ==
2019-09-08T15:57:32.9726524Z   local time: Sun Sep  8 15:57:32 UTC 2019
2019-09-08T15:57:33.0561600Z   network time: Sun, 08 Sep 2019 15:57:33 GMT
2019-09-08T15:57:33.0564184Z == end clock drift check ==
2019-09-08T15:57:37.9632405Z ##[error]Bash exited with code '1'.
2019-09-08T15:57:37.9671669Z ##[section]Starting: Checkout
2019-09-08T15:57:37.9674459Z ==============================================================================
2019-09-08T15:57:37.9674520Z Task         : Get sources
2019-09-08T15:57:37.9674591Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@Mark-Simulacrum Mark-Simulacrum force-pushed the Mark-Simulacrum:include-warn branch from 1eb6922 to 2a92dc7 Sep 8, 2019

@petrochenkov

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2019

Seems like an obvious bugfix to me (compare with with fn make_items below which does perform the token::Eof check for item position include!s).

The end goal is to migrate built-in macros to the token-based model (which is currently in effect for proc macros).
In that model the macro returns tokens, and those tokens are parsed into an AST fragment depending on the macro's position (expresstion, type, items or whatever). If the tokens don't fit into that fragment's grammar, then it's an error. (In this particular case the tokens don't fit into expression grammar.)

Why include!("include-single-expr-helper.rs"); is treated as an expression position macro and not a statement position macro is an separate question though, probably related to #61733.
It should be addressed by the upcoming invocation collector rewrite.
I'd expect include-single-expr.rs to compile successfully and expand into three statements.

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member Author

commented Sep 8, 2019

Yeah, I agree that the specific test case might not be quite ideal -- open to suggestions, I'm not sure how to structure a good test case. Maybe just 10 10 10 would be fine since it's multiple expressions (and invalid Rust grammar, I believe, as such, since expressions need some delimiter).

@petrochenkov

This comment has been minimized.

Copy link
Contributor

commented Sep 9, 2019

Maybe just 10 10 10 would be fine since it's multiple expressions

I'd change the test to something like that, yes.

@joshtriplett

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

I can see two reasonable behaviors here:

  • Error (not warn/lint) if we don't parse and include the entire file. (Note, though, that comments shouldn't cause an error here.)
  • Actually parse and include the entire file.
@nikomatsakis

This comment has been minimized.

Copy link
Contributor

commented Sep 12, 2019

@Mark-Simulacrum

do we have a test for trailing comments -- can we add a test for a file like

22; /* this is a comment */

I don't think this should error. I'm not sure whether it will or not, I think it will actually work.

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member Author

commented Sep 12, 2019

I think we don't formally have such a test but I believe I locally tested that behavior (and can/will add such a test).

Did lang team get a chance to discuss this yet? I'd prefer to avoid throwing work at it if there's no need to (if lang team decides we're not ready to turn this into an error/warning).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.