Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unstable Literal::subspan(). #56120

Merged
merged 1 commit into from
Nov 23, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 46 additions & 1 deletion src/libproc_macro/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ mod diagnostic;
#[unstable(feature = "proc_macro_diagnostic", issue = "54140")]
pub use diagnostic::{Diagnostic, Level, MultiSpan};

use std::ops::{Bound, RangeBounds};
use std::{ascii, fmt, iter};
use std::path::PathBuf;
use rustc_data_structures::sync::Lrc;
Expand All @@ -59,7 +60,7 @@ use syntax::errors::DiagnosticBuilder;
use syntax::parse::{self, token};
use syntax::symbol::Symbol;
use syntax::tokenstream::{self, DelimSpan};
use syntax_pos::{Pos, FileName};
use syntax_pos::{Pos, FileName, BytePos};

/// The main type provided by this crate, representing an abstract stream of
/// tokens, or, more specifically, a sequence of token trees.
Expand Down Expand Up @@ -1168,6 +1169,50 @@ impl Literal {
pub fn set_span(&mut self, span: Span) {
self.span = span;
}

/// Returns a `Span` that is a subset of `self.span()` containing only the
/// source bytes in range `range`. Returns `None` if the would-be trimmed
/// span is outside the bounds of `self`.
// FIXME(SergioBenitez): check that the byte range starts and ends at a
// UTF-8 boundary of the source. otherwise, it's likely that a panic will
// occur elsewhere when the source text is printed.
// FIXME(SergioBenitez): there is no way for the user to know what
// `self.span()` actually maps to, so this method can currently only be
// called blindly. For example, `to_string()` for the character 'c' returns
// "'\u{63}'"; there is no way for the user to know whether the source text
// was 'c' or whether it was '\u{63}'.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is correct. Literal::to_string should show the exact source contents, if the literal isn't synthetic.

Copy link
Contributor Author

@SergioBenitez SergioBenitez Nov 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct. Try:

extern crate proc_macro;

use crate::proc_macro::Literal;
use crate::proc_macro::TokenStream;

#[proc_macro]
pub fn go(_: TokenStream) -> TokenStream {
    let literal = Literal::string("hel\u{6c}o");
    println!("literal: {}", literal.to_string());

    let literal = Literal::character('c');
    println!("literal: {}", literal.to_string());

    let literal = Literal::character(100 as char);
    println!("literal: {}", literal.to_string());

    let literal = Literal::byte_string(b"hello");
    println!("literal: {}", literal.to_string());

    let literal = Literal::byte_string(b"he\x42");
    println!("literal: {}", literal.to_string());

    let literal = Literal::byte_string(&['h' as u8, 'e' as u8, ]);
    println!("literal: {}", literal.to_string());

    let literal = Literal::f32_unsuffixed(1.0);
    println!("literal: {}", literal.to_string());

    let literal = Literal::f32_unsuffixed(1.134100000022);
    println!("literal: {}", literal.to_string());

    TokenStream::new()
}

Edit: This may be different if the Literal comes from a TokenStream parsed from source. Nevertheless, this method needs to be correct regardless of where the Literal comes from.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's synthetic then the span doesn't matter, I was talking about Literals from the source. But also, we shouldn't be using escape_unicode like that.

#[unstable(feature = "proc_macro_span", issue = "54725")]
pub fn subspan<R: RangeBounds<usize>>(&self, range: R) -> Option<Span> {
let inner = self.span().0;
let length = inner.hi().to_usize() - inner.lo().to_usize();

let start = match range.start_bound() {
Bound::Included(&lo) => lo,
Bound::Excluded(&lo) => lo + 1,
Bound::Unbounded => 0,
};

let end = match range.end_bound() {
Bound::Included(&hi) => hi + 1,
Bound::Excluded(&hi) => hi,
Bound::Unbounded => length,
};

// Bounds check the values, preventing addition overflow and OOB spans.
if start > u32::max_value() as usize
|| end > u32::max_value() as usize
|| (u32::max_value() - start as u32) < inner.lo().to_u32()
|| (u32::max_value() - end as u32) < inner.lo().to_u32()
|| start >= end
|| end > length
{
return None;
}

let new_lo = inner.lo() + BytePos::from_usize(start);
let new_hi = inner.lo() + BytePos::from_usize(end);
Some(Span(inner.with_lo(new_lo).with_hi(new_hi)))
}
}

/// Prints the literal as a string that should be losslessly convertible
Expand Down
47 changes: 47 additions & 0 deletions src/test/ui-fulldeps/auxiliary/subspan.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// Copyright 2018 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

// no-prefer-dynamic

#![crate_type = "proc-macro"]
#![feature(proc_macro_diagnostic, proc_macro_span)]

extern crate proc_macro;

use proc_macro::{TokenStream, TokenTree, Span, Diagnostic};

fn parse(input: TokenStream) -> Result<(), Diagnostic> {
if let Some(TokenTree::Literal(lit)) = input.into_iter().next() {
let mut spans = vec![];
let string = lit.to_string();
for hi in string.matches("hi") {
let index = hi.as_ptr() as usize - string.as_ptr() as usize;
let subspan = lit.subspan(index..(index + hi.len())).unwrap();
spans.push(subspan);
}

if !spans.is_empty() {
Err(Span::call_site().error("found 'hi's").span_note(spans, "here"))
} else {
Ok(())
}
} else {
Err(Span::call_site().error("invalid input: expected string literal"))
}
}

#[proc_macro]
pub fn subspan(input: TokenStream) -> TokenStream {
if let Err(diag) = parse(input) {
diag.emit();
}

TokenStream::new()
}
37 changes: 37 additions & 0 deletions src/test/ui-fulldeps/subspan.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright 2018 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

// aux-build:subspan.rs
// ignore-stage1

extern crate subspan;

use subspan::subspan;

// This one emits no error.
subspan!("");

// Exactly one 'hi'.
subspan!("hi"); //~ ERROR found 'hi's

// Now two, back to back.
subspan!("hihi"); //~ ERROR found 'hi's

// Now three, back to back.
subspan!("hihihi"); //~ ERROR found 'hi's

// Now several, with spacing.
subspan!("why I hide? hi!"); //~ ERROR found 'hi's
subspan!("hey, hi, hidy, hidy, hi hi"); //~ ERROR found 'hi's
subspan!("this is a hi, and this is another hi"); //~ ERROR found 'hi's
subspan!("how are you this evening"); //~ ERROR found 'hi's
subspan!("this is highly eradic"); //~ ERROR found 'hi's

fn main() { }
98 changes: 98 additions & 0 deletions src/test/ui-fulldeps/subspan.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
error: found 'hi's
--> $DIR/subspan.rs:22:1
|
LL | subspan!("hi"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:22:11
|
LL | subspan!("hi"); //~ ERROR found 'hi's
| ^^

error: found 'hi's
--> $DIR/subspan.rs:25:1
|
LL | subspan!("hihi"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:25:11
|
LL | subspan!("hihi"); //~ ERROR found 'hi's
| ^^^^

error: found 'hi's
--> $DIR/subspan.rs:28:1
|
LL | subspan!("hihihi"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:28:11
|
LL | subspan!("hihihi"); //~ ERROR found 'hi's
| ^^^^^^

error: found 'hi's
--> $DIR/subspan.rs:31:1
|
LL | subspan!("why I hide? hi!"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:31:17
|
LL | subspan!("why I hide? hi!"); //~ ERROR found 'hi's
| ^^ ^^

error: found 'hi's
--> $DIR/subspan.rs:32:1
|
LL | subspan!("hey, hi, hidy, hidy, hi hi"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:32:16
|
LL | subspan!("hey, hi, hidy, hidy, hi hi"); //~ ERROR found 'hi's
| ^^ ^^ ^^ ^^ ^^

error: found 'hi's
--> $DIR/subspan.rs:33:1
|
LL | subspan!("this is a hi, and this is another hi"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:33:12
|
LL | subspan!("this is a hi, and this is another hi"); //~ ERROR found 'hi's
| ^^ ^^ ^^ ^^

error: found 'hi's
--> $DIR/subspan.rs:34:1
|
LL | subspan!("how are you this evening"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:34:24
|
LL | subspan!("how are you this evening"); //~ ERROR found 'hi's
| ^^

error: found 'hi's
--> $DIR/subspan.rs:35:1
|
LL | subspan!("this is highly eradic"); //~ ERROR found 'hi's
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: here
--> $DIR/subspan.rs:35:12
|
LL | subspan!("this is highly eradic"); //~ ERROR found 'hi's
| ^^ ^^

error: aborting due to 8 previous errors