UTF-8 routes. Forms revamp. Temp files. Capped.

So. Many. Changes.

This is an insane commit: simultaneously one of the best (because of all
the wonderful improvements!) and one of the worst (because it is just
massive) in the project's history.

Routing:
  * All UTF-8 characters are accepted everywhere in route paths. (#998)
  * `path` is now `uri` in `route` attribute: `#[route(GET, path = "..")]`
    becomes `#[route(GET, uri = "..")]`.

Forms Revamp
  * All form related types now reside in a new `form` module.
  * Multipart forms are supported. (resolves #106)
  * Collections are supported in forms and queries. (resolves #205)
  * Nested structures in forms and queries are supported. (resolves #313)
  * Form fields can be ad-hoc validated with `#[field(validate = expr)]`.
  * `FromFormValue` is now `FromFormField`, blanket implements `FromForm`.
  * Form field values are always percent-decoded apriori.

Temporary Files
  * A new `TempFile` data and form guard allows streaming data directly to a
    file which can then be persisted.
  * A new `temp_dir` config parameter specifies where to store `TempFile`.
  * The limits `file` and `file/$ext`, where `$ext` is the file extension,
    determines the data limit for a `TempFile`.

Capped
  * A new `Capped` type is used to indicate when data has been truncated due to
    incoming data limits. It allows checking whether data is complete or
    truncated.
  * `DataStream` methods return `Capped` types.
  * `DataStream` API has been revamped to account for `Capped` types.
  * Several `Capped<T>` types implement `FromData`, `FromForm`.
  * HTTP 413 (Payload Too Large) errors are now returned when data limits are
    exceeded. (resolves #972)

Hierarchical Limits
  * Data limits are now hierarchical, delimited with `/`. A limit of `a/b/c`
    falls back to `a/b` then `a`.

Core
  * `&RawStr` no longer implements `FromParam`.
  * `&str` implements `FromParam`, `FromData`, `FromForm`.
  * `FromTransformedData` was removed.
  * `FromData` gained a lifetime for use with request-local data.
  * The default error HTML is more compact.
  * `&Config` is a request guard.
  * The `DataStream` interface was entirely revamped.
  * `State` is only exported via `rocket::State`.
  * A `request::local_cache!()` macro was added for storing values in
    request-local cache without consideration for type uniqueness by using a
    locally generated anonymous type.
  * `Request::get_param()` is now `Request::param()`.
  * `Request::get_segments()` is now `Request::segments()`, takes a range.
  * `Request::get_query_value()` is now `Request::query_value()`, can parse any
    `FromForm` including sequences.
  * `std::io::Error` implements `Responder` like `Debug<std::io::Error>`.
  * `(Status, R)` where `R: Responder` implements `Responder` by overriding the
    `Status` of `R`.
  * The name of a route is printed first during route matching.
  * `FlashMessage` now only has one lifetime generic.

HTTP
  * `RawStr` implements `serde::{Serialize, Deserialize}`.
  * `RawStr` implements _many_ more methods, in particular, those related to the
    `Pattern` API.
  * `RawStr::from_str()` is now `RawStr::new()`.
  * `RawStr::url_decode()` and `RawStr::url_decode_lossy()` only allocate as
    necessary, return `Cow`.
  * `Status` implements `Default` with `Status::Ok`.
  * `Status` implements `PartialEq`, `Eq`, `Hash`, `PartialOrd`, `Ord`.
  * Authority and origin part of `Absolute` can be modified with new
    `Absolute::{with,set}_authority()`, `Absolute::{with,set}_origin()` methods.
  * `Origin::segments()` was removed in favor of methods split into query and
    path parts and into raw and decoded versions.
  * The `Segments` iterator is smarter, returns decoded `&str` items.
  * `Segments::into_path_buf()` is now `Segments::to_path_buf()`.
  * A new `QuerySegments` is the analogous query segment iterator.
  * Once set, `expires` on private cookies is not overwritten. (resolves #1506)
  * `Origin::path()` and `Origin::query()` return `&RawStr`, not `&str`.

Codegen
  * Preserve more spans in `uri!` macro.
  * Preserve spans `FromForm` field types.
  * All dynamic parameters in a query string must typecheck as `FromForm`.
  * `FromFormValue` derive removed; `FromFormField` added.
  * The `form` `FromForm` and `FromFormField` field attribute is now named
    `field`. `#[form(field = ..)]` is now `#[field(name = ..)]`.

Contrib
  * `Json` implements `FromForm`.
  * `MsgPack` implements `FromForm`.
  * The `json!` macro is exported as `rocket_contrib::json::json!`.
  * Added clarifying docs to `StaticFiles`.

Examples
  * `form_validation` and `form_kitchen_sink` removed in favor of `forms`.
  * The `hello_world` example uses unicode in paths.
  * The `json` example only allocates as necessary.

Internal
  * Codegen uses new `exports` module with the following conventions:
    - Locals starts with `__` and are lowercased.
    - Rocket modules start with `_` and are lowercased.
    - `std` types start with `_` and are titlecased.
    - Rocket types are titlecased.
  * A `header` module was added to `http`, contains header types.
  * `SAFETY` is used as doc-string keyword for `unsafe` related comments.
  * The `Uri` parser no longer recognizes Rocket route URIs.

Cargo.toml

@@ -1,6 +1,3 @@
-[profile.dev]
-codegen-units = 4
-
 [workspace]
 members = [
   "core/lib/",
@@ -11,7 +8,7 @@ members = [
   "site/tests",
   "examples/cookies",
   "examples/errors",
-  "examples/form_validation",
+  "examples/forms",
   "examples/hello_person",
   "examples/query_params",
   "examples/hello_world",
@@ -30,7 +27,6 @@ members = [
   "examples/msgpack",
   "examples/handlebars_templates",
   "examples/tera_templates",
-  "examples/form_kitchen_sink",
   "examples/config",
   "examples/raw_upload",
   "examples/pastebin",

contrib/codegen/Cargo.toml

@@ -19,7 +19,7 @@ proc-macro = true
 
 [dependencies]
 quote = "1.0"
-devise = { git = "https://github.com/SergioBenitez/Devise.git", rev = "3648468" }
+devise = { git = "https://github.com/SergioBenitez/Devise.git", rev = "bd221a4" }
 
 [dev-dependencies]
 rocket = { version = "0.5.0-dev", path = "../../core/lib" }

contrib/lib/src/json.rs

@@ -14,40 +14,41 @@
 //! features = ["json"]
 //! ```
 
-use std::ops::{Deref, DerefMut};
 use std::io;
+use std::ops::{Deref, DerefMut};
 use std::iter::FromIterator;
 
-use rocket::request::Request;
-use rocket::outcome::Outcome::*;
-use rocket::data::{Data, ByteUnit, Transform::*, Transformed};
-use rocket::data::{FromTransformedData, TransformFuture, FromDataFuture};
-use rocket::http::Status;
+use rocket::request::{Request, local_cache};
+use rocket::data::{ByteUnit, Data, FromData, Outcome};
 use rocket::response::{self, Responder, content};
+use rocket::http::Status;
+use rocket::form::prelude as form;
 
 use serde::{Serialize, Serializer};
-use serde::de::{Deserialize, Deserializer};
+use serde::de::{Deserialize, DeserializeOwned, Deserializer};
 
 #[doc(hidden)]
 pub use serde_json::{json_internal, json_internal_vec};
 
-/// The JSON type: implements [`FromTransformedData`] and [`Responder`], allowing you to
-/// easily consume and respond with JSON.
+/// The JSON data guard: easily consume and respond with JSON.
 ///
 /// ## Receiving JSON
 ///
-/// If you're receiving JSON data, simply add a `data` parameter to your route
-/// arguments and ensure the type of the parameter is a `Json<T>`, where `T` is
-/// some type you'd like to parse from JSON. `T` must implement [`Deserialize`]
-/// from [`serde`]. The data is parsed from the HTTP request body.
+/// `Json` is both a data guard and a form guard.
+///
+/// ### Data Guard
+///
+/// To parse request body data as JSON , add a `data` route argument with a
+/// target type of `Json<T>`, where `T` is some type you'd like to parse from
+/// JSON. `T` must implement [`serde::Deserialize`].
 ///
 /// ```rust
 /// # #[macro_use] extern crate rocket;
 /// # extern crate rocket_contrib;
 /// # type User = usize;
 /// use rocket_contrib::json::Json;
 ///
-/// #[post("/users", format = "json", data = "<user>")]
+/// #[post("/user", format = "json", data = "<user>")]
 /// fn new_user(user: Json<User>) {
 ///     /* ... */
 /// }
@@ -58,6 +59,30 @@ pub use serde_json::{json_internal, json_internal_vec};
 /// "application/json" as its `Content-Type` header value will not be routed to
 /// the handler.
 ///
+/// ### Form Guard
+///
+/// `Json<T>`, as a form guard, accepts value and data fields and parses the
+/// data as a `T`. Simple use `Json<T>`:
+///
+/// ```rust
+/// # #[macro_use] extern crate rocket;
+/// # extern crate rocket_contrib;
+/// # type Metadata = usize;
+/// use rocket::form::{Form, FromForm};
+/// use rocket_contrib::json::Json;
+///
+/// #[derive(FromForm)]
+/// struct User<'r> {
+///     name: &'r str,
+///     metadata: Json<Metadata>
+/// }
+///
+/// #[post("/user", data = "<form>")]
+/// fn new_user(form: Form<User<'_>>) {
+///     /* ... */
+/// }
+/// ```
+///
 /// ## Sending JSON
 ///
 /// If you're responding with JSON data, return a `Json<T>` type, where `T`
@@ -94,6 +119,22 @@ pub use serde_json::{json_internal, json_internal_vec};
 #[derive(Debug)]
 pub struct Json<T>(pub T);
 
+/// An error returned by the [`Json`] data guard when incoming data fails to
+/// serialize as JSON.
+#[derive(Debug)]
+pub enum JsonError<'a> {
+    /// An I/O error occurred while reading the incoming request data.
+    Io(io::Error),
+
+    /// The client's data was received successfully but failed to parse as valid
+    /// JSON or as the requested type. The `&str` value in `.0` is the raw data
+    /// received from the user, while the `Error` in `.1` is the deserialization
+    /// error from `serde`.
+    Parse(&'a str, serde_json::error::Error),
+}
+
+const DEFAULT_LIMIT: ByteUnit = ByteUnit::Mebibyte(1);
+
 impl<T> Json<T> {
     /// Consumes the JSON wrapper and returns the wrapped item.
     ///
@@ -110,52 +151,38 @@ impl<T> Json<T> {
     }
 }
 
-/// An error returned by the [`Json`] data guard when incoming data fails to
-/// serialize as JSON.
-#[derive(Debug)]
-pub enum JsonError<'a> {
-    /// An I/O error occurred while reading the incoming request data.
-    Io(io::Error),
-
-    /// The client's data was received successfully but failed to parse as valid
-    /// JSON or as the requested type. The `&str` value in `.0` is the raw data
-    /// received from the user, while the `Error` in `.1` is the deserialization
-    /// error from `serde`.
-    Parse(&'a str, serde_json::error::Error),
-}
+impl<'r, T: Deserialize<'r>> Json<T> {
+    fn from_str(s: &'r str) -> Result<Self, JsonError<'r>> {
+        serde_json::from_str(s).map(Json).map_err(|e| JsonError::Parse(s, e))
+    }
 
-const DEFAULT_LIMIT: ByteUnit = ByteUnit::Mebibyte(1);
+    async fn from_data(req: &'r Request<'_>, data: Data) -> Result<Self, JsonError<'r>> {
+        let size_limit = req.limits().get("json").unwrap_or(DEFAULT_LIMIT);
+        let string = match data.open(size_limit).into_string().await {
+            Ok(s) if s.is_complete() => s.into_inner(),
+            Ok(_) => {
+                let eof = io::ErrorKind::UnexpectedEof;
+                return Err(JsonError::Io(io::Error::new(eof, "data limit exceeded")));
+            },
+            Err(e) => return Err(JsonError::Io(e)),
+        };
 
-impl<'a, T: Deserialize<'a>> FromTransformedData<'a> for Json<T> {
-    type Error = JsonError<'a>;
-    type Owned = String;
-    type Borrowed = str;
-
-    fn transform<'r>(r: &'r Request<'_>, d: Data) -> TransformFuture<'r, Self::Owned, Self::Error> {
-        Box::pin(async move {
-            let size_limit = r.limits().get("json").unwrap_or(DEFAULT_LIMIT);
-            match d.open(size_limit).stream_to_string().await {
-                Ok(s) => Borrowed(Success(s)),
-                Err(e) => Borrowed(Failure((Status::BadRequest, JsonError::Io(e))))
-            }
-        })
+        Self::from_str(local_cache!(req, string))
     }
+}
+
+#[rocket::async_trait]
+impl<'r, T: Deserialize<'r>> FromData<'r> for Json<T> {
+    type Error = JsonError<'r>;
 
-    fn from_data(_: &'a Request<'_>, o: Transformed<'a, Self>) -> FromDataFuture<'a, Self, Self::Error> {
-        Box::pin(async move {
-            let string = try_outcome!(o.borrowed());
-            match serde_json::from_str(&string) {
-                Ok(v) => Success(Json(v)),
-                Err(e) => {
-                    error_!("Couldn't parse JSON body: {:?}", e);
-                    if e.is_data() {
-                        Failure((Status::UnprocessableEntity, JsonError::Parse(string, e)))
-                    } else {
-                        Failure((Status::BadRequest, JsonError::Parse(string, e)))
-                    }
-                }
-            }
-        })
+    async fn from_data(req: &'r Request<'_>, data: Data) -> Outcome<Self, Self::Error> {
+        match Self::from_data(req, data).await {
+            Ok(value) => Outcome::Success(value),
+            Err(JsonError::Io(e)) if e.kind() == io::ErrorKind::UnexpectedEof => {
+                Outcome::Failure((Status::PayloadTooLarge, JsonError::Io(e)))
+            },
+            Err(e) => Outcome::Failure((Status::BadRequest, e)),
+        }
     }
 }
 
@@ -190,6 +217,26 @@ impl<T> DerefMut for Json<T> {
     }
 }
 
+impl From<JsonError<'_>> for form::Error<'_> {
+    fn from(e: JsonError<'_>) -> Self {
+        match e {
+            JsonError::Io(e) => e.into(),
+            JsonError::Parse(_, e) => form::Error::custom(e)
+        }
+    }
+}
+
+#[rocket::async_trait]
+impl<'v, T: DeserializeOwned + Send> form::FromFormField<'v> for Json<T> {
+    fn from_value(field: form::ValueField<'v>) -> Result<Self, form::Errors<'v>> {
+        Ok(Self::from_str(field.value)?)
+    }
+
+    async fn from_data(f: form::DataField<'v, '_>) -> Result<Self, form::Errors<'v>> {
+        Ok(Self::from_data(f.request, f.data).await?)
+    }
+}
+
 /// An arbitrary JSON value.
 ///
 /// This structure wraps `serde`'s [`Value`] type. Importantly, unlike `Value`,
@@ -397,3 +444,5 @@ macro_rules! json {
         $crate::json::JsonValue($crate::json::json_internal!($($json)+))
     };
 }
+
+pub use json;