macros: Clean up code with non-optional NonterminalKind

[tgross35]

Since 1, the fragment specifier is unconditionally required in all
editions. This means NonTerminalKind no longer needs to be optional,
as we can reject this code during the expansion of macro_rules! rather
than handling it throughout the code. Do this cleanup here.

r? @petrochenkov

[tgross35]

Can be reviewed by-commit, the first commit is just refactoring a heavily nested function.

[tgross35]

I don't know why these new "unexpected end" errors are showing up at the invocation rather than at macro_rules! or not at all. Any idea how to get rid of this?

[tgross35]

Probably need to emulate this behavior somewhere

rust/compiler/rustc_expand/src/mbe/macro_parser.rs

Lines 631 to 705 in 77ec48f

	let res = self.parse_tt_inner(
	matcher,
	&parser.token,
	parser.approx_token_stream_pos(),
	track,
	);
	if let Some(res) = res {
	return res;
	}
	// `parse_tt_inner` handled all of `cur_mps`, so it's empty.
	assert!(self.cur_mps.is_empty());
	// Error messages here could be improved with links to original rules.
	match (self.next_mps.len(), self.bb_mps.len()) {
	(0, 0) => {
	// There are no possible next positions AND we aren't waiting for the black-box
	// parser: syntax error.
	return Failure(T::build_failure(
	parser.token,
	parser.approx_token_stream_pos(),
	"no rules expected this token in macro call",
	));
	}
	(_, 0) => {
	// Dump all possible `next_mps` into `cur_mps` for the next iteration. Then
	// process the next token.
	self.cur_mps.append(&mut self.next_mps);
	parser.to_mut().bump();
	}
	(0, 1) => {
	// We need to call the black-box parser to get some nonterminal.
	let mut mp = self.bb_mps.pop().unwrap();
	let loc = &matcher[mp.idx];
	if let &MatcherLoc::MetaVarDecl {
	span,
	kind: Some(kind),
	next_metavar,
	seq_depth,
	..
	} = loc
	{
	// We use the span of the metavariable declaration to determine any
	// edition-specific matching behavior for non-terminals.
	let nt = match parser.to_mut().parse_nonterminal(kind) {
	Err(err) => {
	let guarantee = err.with_span_label(
	span,
	format!(
	"while parsing argument for this `{kind}` macro fragment"
	),
	)
	.emit();
	return ErrorReported(guarantee);
	}
	Ok(nt) => nt,
	};
	mp.push_match(next_metavar, seq_depth, MatchedSingle(nt));
	mp.idx += 1;
	} else {
	unreachable!()
	}
	self.cur_mps.push(mp);
	}
	(_, _) => {
	// Too many possibilities!
	return self.ambiguity_error(matcher, parser.token.span);
	}
	}
	assert!(!self.cur_mps.is_empty());

since parse_tt_inner used to return an Err if there was no fragment specifier.

[tgross35]

If we emit a InvalidFragmentSpecifier (below) the fallback is Ident rather than TT. Any reason to prefer one or the other?