Fix grammar verifcation

[dns2utf8]

I wanted to improve the grammar-section but I can not build it.

r? @steveklabnik @brson

EDIT: I resolved some of the issues, removed the compiler output from the entry.

[dns2utf8]

I was able to fix the errors, yay 😁 The API of libsyntax was changed (eg 24e7491) but the veryfy.rs is not getting build.

Should we include the antlr4-complete.jar or just build the verify.rs with the compiler? So further changes to the API will triger an error.

[dbrgn]

This should probably say "antlr", not "antler".

[dns2utf8]

Ah, yes thank you 👍

[dns2utf8]

I run the check-lexer target and the result was this:

test result: failed. 4764 passed; 125 failed; 2 skipped

I am curious, what are the next steps to fix the failed tests?

[steveklabnik]

hm, is this always going to be where this is? Feels system-specific

[dns2utf8]

It is the place in Arch Linux, but not on Mac OS X. I have not tested other distros.

[steveklabnik]

@alexcrichton what do you think of this PR?

[alexcrichton]

It does seems system specific, perhaps that could be refactored? That being said I know very little about all of this, and we don't run it on the bots, so I don't mind merging basically whatever here.

[brson]

I'm cool with pushing these improvements through even with the hardcoded path. It's better than it was, and I'm glad somebody is looking at this stuff. Would be good to get this onto a bot #28592

[dns2utf8]

The system specific path could be obmittet with an environement variable, so the javac can find the antlr4-complete.jar but this would require the user to set it manually or the configure script to search for it.

Another alternative would be to include the jar file and just ship it. Is this a good idea?

cc: @nikomatsakis

[dns2utf8]

I add the target check-build-lexer-verifier to make tidy, so it will build the verifier with every build and catch future changes in the API.
This does not require javac or antlr.

I think this is a good state freeze/merge for now. I planed to add some documumentation about the process in the next weeks, but I am no sure whether it will be in a good shape before the 18. august or not.

[steveklabnik]

@bors: r+ rollup

now, we can make it better eventually 😄

[bors]

📌 Commit 76026d1 has been approved by steveklabnik

[jonathandturner]

This does not handle an out-of-tree build directory, which our bots use.

cleaning grammar verification
/bin/sh: 1: cd: can't cd to src/grammar
/buildslave/rust-buildbot/slave/auto-linux-64-nopt-t/build/mk/clean.mk:51: recipe for target 'clean-grammar' failed
make: *** [clean-grammar] Error 2
program finished with exit code 2
elapsedTime=0.958165

[dns2utf8]

Interesting, looking on it.

[jonathandturner]

@bors r-

[dns2utf8]

I moved the in-tree part to the README of the manual process, since it was just for convenience.

I was looking for the structure of the build bot, can I run one locally?

[steveklabnik]

@alexcrichton ping re last comment here, can @dns2utf8 do something to test this out?

[alexcrichton]

We've got a number of docker images which our bots are running, so running the tests inside those containers would perhaps be enough to test out?

[dns2utf8]

I was able to start a container and access it with this command:

docker run --rm -it --entrypoint=/bin/bash alexcrichton/rust-slave-linux:2016-08-04

then inside clone my branch like the script:

git clone https://github.com/dns2utf8/rust.git --branch doc_grammar

I can then build my branch:

cd rust
./configure && make -j$(nproc) docs

but I am not sure this is the same as you do.
Am I doing it wrong? @jonathandturner

EDIT: I am doing this, because the setup-slave.sh uses some internal server which I can not reach so I am strongly guessing

[alexcrichton]

@dns2utf8 yeah that's basically the same as what we do, this'd just be testing for presence of the relevant tools.

Is this running by default on make check though? I thought you had a run a custom check target?

[alexcrichton]

Closing due to inactivity, but feel free to resubmit with a rebase!

[dns2utf8]

Thanks for the reminder, I rebased it.

Can I reopen the issue just with a comment?