cleanup shebang handling in the lexer #59687

matklad · 2019-04-04T08:10:29Z

No description provided.

rust-highfive · 2019-04-04T08:10:32Z

(rust_highfive has picked a reviewer for you, use r? to override)

rust-highfive · 2019-04-04T10:05:34Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:013ec130:start=1554365829450513934,finish=1554365905777002284,duration=76326488350
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
Setting environment variables from .travis.yml
---
travis_time:start:test_assembly
Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:21:08] 
[01:21:08] running 9 tests
[01:21:08] iiiiiiiii
[01:21:08] 
[01:21:08]  finished in 0.153
[01:21:08] travis_fold:end:test_assembly

---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:21:24] 
[01:21:24] running 121 tests
[01:21:50] .iiiii...i.....i..i...i..i.i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i. 100/121
[01:21:54] i.i......iii.i.....ii
[01:21:54] 
[01:21:54]  finished in 30.587
[01:21:54] travis_fold:end:test_debuginfo

---
[01:46:41]    Compiling syntax v0.0.0 (/checkout/src/libsyntax)
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:1949:46
[01:46:44]      |
[01:46:44] 1949 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:1997:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 1997 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2007:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2007 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2017:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2017 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2027:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2027 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2037:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2037 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2047:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2047 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2057:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2057 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2067:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2067 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2077:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2077 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2089:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2089 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2135:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2135 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:44] error[E0433]: failed to resolve: use of undeclared type or module `FilePathMapping`
[01:46:44]     --> src/libsyntax/parse/lexer/mod.rs:2150:46
[01:46:44]      |
[01:46:44]      |
[01:46:44] 2150 |             let sm = Lrc::new(SourceMap::new(FilePathMapping::empty()));
[01:46:44]      |                                              ^^^^^^^^^^^^^^^ use of undeclared type or module `FilePathMapping`
[01:46:53] error: aborting due to 13 previous errors
[01:46:53] 
[01:46:53] For more information about this error, try `rustc --explain E0433`.
[01:46:53] error: Could not compile `syntax`.
[01:46:53] error: Could not compile `syntax`.
[01:46:53] 
[01:46:53] To learn more, run the command again with --verbose.
[01:46:53] 
[01:46:53] 
[01:46:53] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "syntax" "--" "--quiet"
[01:46:53] 
[01:46:53] 
[01:46:53] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:46:53] Build completed unsuccessfully in 0:37:35
[01:46:53] Build completed unsuccessfully in 0:37:35
[01:46:53] make: *** [check] Error 1
[01:46:53] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:077eb17c
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Thu Apr  4 10:05:28 UTC 2019
---
travis_time:end:187e2b1c:start=1554372329838296970,finish=1554372329910896610,duration=72599640
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:1123798f
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0f76a322
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

petrochenkov · 2019-04-04T21:59:50Z

src/libsyntax/parse/lexer/mod.rs

-                debug!("Skipping a shebang");
-                if loc.line == 1 && loc.col == CharPos(0) {
-                    // FIXME: Add shebang "token", return it
+                let is_beginning_of_file = self.pos == self.source_file.start_pos;


So, the new code accepts shebang at the start of anything that compiler treats as a source file - out-of-line mod, perhaps command line or macro?
And I'm not even sure what the old code does.
I rather wish this thing never existed in the language.

My reading is that the old code did the same, but in a more elaborate way, by binary searching in a single-element slice. I’ve manually checked that shebangs are accepted in the outofline modules today.

For the referrnce, here’s the original commit that added the code: #2430

Accidentally, it also added this _adj function, which I would like to get rid of next

petrochenkov · 2019-04-05T07:02:14Z

@bors r+ rollup

bors · 2019-04-05T07:02:15Z

📌 Commit fdb8752 has been approved by petrochenkov

cleanup shebang handling in the lexer

@ghost

Rollup of 5 pull requests Successful merges: - #59665 (improve worst-case performance of HashSet.is_subset) - #59687 (cleanup shebang handling in the lexer) - #59690 (Mark unix::ffi::OsStrExt methods as inline) - #59702 (Use declare_lint_pass! and impl_lint_pass! in more places) - #59712 (wasm32: Default to a "static" relocation model) Failed merges: r? @ghost

rust-highfive assigned petrochenkov Apr 4, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 4, 2019

matklad force-pushed the shebang branch from ab52d5e to 78aaaba Compare April 4, 2019 08:16

matklad mentioned this pull request Apr 4, 2019

Make some of lexer's API private #59671

Merged

cleanup shebang handling in the lexer

fdb8752

matklad force-pushed the shebang branch from 78aaaba to fdb8752 Compare April 4, 2019 10:32

petrochenkov reviewed Apr 4, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 5, 2019

Centril added a commit to Centril/rust that referenced this pull request Apr 5, 2019

Rollup merge of rust-lang#59687 - matklad:shebang, r=petrochenkov

2e7be1e

cleanup shebang handling in the lexer

Centril mentioned this pull request Apr 5, 2019

Rollup of 5 pull requests #59721

Merged

bors merged commit fdb8752 into rust-lang:master Apr 5, 2019

matklad deleted the shebang branch July 9, 2019 12:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cleanup shebang handling in the lexer #59687

cleanup shebang handling in the lexer #59687

matklad commented Apr 4, 2019

rust-highfive commented Apr 4, 2019

rust-highfive commented Apr 4, 2019

petrochenkov Apr 4, 2019

matklad Apr 5, 2019

matklad Apr 5, 2019

petrochenkov commented Apr 5, 2019

bors commented Apr 5, 2019

cleanup shebang handling in the lexer #59687

cleanup shebang handling in the lexer #59687

Conversation

matklad commented Apr 4, 2019

rust-highfive commented Apr 4, 2019

rust-highfive commented Apr 4, 2019

petrochenkov Apr 4, 2019

Choose a reason for hiding this comment

matklad Apr 5, 2019

Choose a reason for hiding this comment

matklad Apr 5, 2019

Choose a reason for hiding this comment

petrochenkov commented Apr 5, 2019

bors commented Apr 5, 2019