Rewrite our internal type system

(Sorry for the really large diff but this touches nearly all parts of
diesel)

This commit does in two things:
* Refactor our handling of nullable values. Instead of having a marker
trait for non nullable sql types we now indicate if a sql type is
nullable by using a associated type on a new fundamental trait named
SqlType. This allows us to reason if an type is nullable or not in a
much precise way without running in conflicting implementation
issues. This allows us to address diesel-rs#104 in a much more fundamental
way. (The correct way as mentioned there by sgrif).
* Refactor our handling of typed and untyped sql fragments. Instead of
having two separate code paths for `Queryable` (sql types known) and
`QueryableByName` (sql types not known) we now have only one code path
and indicate if a query is typed or untyped as part of the expression
sql type. This is required to address diesel-rs#2150. As part of this change we
unify `Queryable`, `QueryableByName` and `FromSqlRow` into only
`FromSqlRow`. Additionally we separate the static size component of
`FromSqlRow` to allow dynamically sized rows there (but only at the
last position for tuple impls.)

I should probably have implement those changes in different commits
but as both changes basically requiring touching most of our code base
this would have required much more work...

Both changes are theoretically big major breaking changes. For
application code I expect the actual breakage to be minimal, see the
required changes in `diesel_tests` and `diesel_cli` for examples. For
highly generic code I would expect quite a few required changes.

Additionally there are a few small drive by fixes.

Fixes diesel-rs#104
Fixes diesel-rs#2274
Fixes diesel-rs#2161

CHANGELOG.md

@@ -39,6 +39,9 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
   functionality of `NonAggregate`. See [the upgrade
   notes](#2-0-0-upgrade-non-aggregate) for details.
 
+* It is now possible to inspect the type of values returned from the database
+  in such a way to support constructing a dynamic value depending on this type.
+
 
 ### Removed
 
@@ -47,6 +50,12 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
 * Support for `bigdecimal` < 0.0.13 has been removed.
 * Support for `pq-sys` < 0.4.0 has been removed.
 * Support for `mysqlclient-sys` < 0.2.0 has been removed.
+* The `Queryable<ST, DB>` trait is replaced in favour of `FromSqlRow<ST, DB>`.
+  If you use `#[derive(Queryable)]`  a compatible implementation will be generated.
+  For manual implementations see the relevant section of [the migration guide][2-0-migration]
+* The `QueryableByName<DB>` is replaced by in favour of `FromSqlRow<Untyped, DB>`.
+  If you use `#[derive(QueryableByName)]`  a compatible implementation will be generated.
+  For manual implementations see the relevant section of [the migration guide][2-0-migration]
 
 ### Changed
 
@@ -93,6 +102,10 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
   `#[non_exhaustive]`. If you matched on one of those variants explicitly you need to
   introduce a wild card match instead.
 
+* `FromSql::from_sql` is changed to construct value from non nullable database values.
+   To construct a rust value for nullable values use the new `FromSql::from_nullable_sql`
+   method instead.
+
 
 ### Fixed
 
@@ -129,6 +142,10 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
 
 * We've refactored our type translation layer for Mysql to handle more types now.
 
+* We've refactored our type level representation of nullable values. This allowed us to
+  fix multiple long standing bugs regarding the correct handling of nullable values in some
+  corner cases (#104, #2274)
+
 ### Deprecated
 
 * `diesel_(prefix|postfix|infix)_operator!` have been deprecated. These macros
@@ -171,6 +188,32 @@ Key points:
   NonAggregate` no longer implies `(OtherType, T): NonAggregate`.
   - With `feature = "unstable"`, `(T, OtherType): NonAggregate` is still implied.
 
+#### Replacement of `Queryable` and `QueryableByName` with `FromSqlRow`
+
+FIXME: This should probably be on the website, but I wanted to document it in
+the PR adding the changes.
+
+Key points:
+
+- Unified deserializing rows into structs between `Queryable`, `QueryableByName` and `FromSqlRow`
+- Allows to write code abstracting over `sql_query` and normal query dsl
+- Derives continue to work, they will just generate the corresponding `FromSqlRow` impl
+- For usage in trait bounds:
+    + `Queryable<ST, DB>` is now `FromSqlRow<ST, DB>`
+    + `QueryableByName<DB>` is now `FromSqlRow<Untyped, DB>`
+- For manual implementations:
+    + `Queryable<ST, DB>` should implement `FromSqlRow<ST, DB>` in such a way
+       that the the `FromSqlRow<ST, DB>` impl for the old `Queryable::Row` type
+       is called internally. After that ~same construction as before
+    + `QueryableByName<DB>` should implement `FromSqlRow<Untyped, DB>` in such a way
+       that for each field the following code is called:
+
+       ```rust
+       let field = row.get("field name").ok_or("Column with name `field name` not found")?;
+       let value = <FieldType as FromSql<SqlTypeOfField, DB>>::from_nullable_sql(field.value())?;
+       ```
+
+
 [2-0-migration]: FIXME write a migration guide
 
 ## [1.4.5] - 2020-06-09

diesel/Cargo.toml

@@ -32,6 +32,7 @@ num-integer = { version = "0.1.39", optional = true }
 bigdecimal = { version = ">= 0.0.13, < 0.2.0", optional = true }
 bitflags = { version = "1.2.0", optional = true }
 r2d2 = { version = ">= 0.8, < 0.9", optional = true }
+itoa = "0.4"
 
 [dependencies.diesel_derives]
 version = "~2.0.0"

diesel/src/associations/belongs_to.rs

@@ -4,6 +4,7 @@ use crate::expression::array_comparison::AsInExpression;
 use crate::expression::AsExpression;
 use crate::prelude::*;
 use crate::query_dsl::methods::FilterDsl;
+use crate::sql_types::SqlType;
 
 use std::borrow::Borrow;
 use std::hash::Hash;
@@ -139,6 +140,7 @@ where
     Id<&'a Parent>: AsExpression<<Child::ForeignKeyColumn as Expression>::SqlType>,
     Child::Table: FilterDsl<Eq<Child::ForeignKeyColumn, Id<&'a Parent>>>,
     Child::ForeignKeyColumn: ExpressionMethods,
+    <Child::ForeignKeyColumn as Expression>::SqlType: SqlType,
 {
     type Output = FindBy<Child::Table, Child::ForeignKeyColumn, Id<&'a Parent>>;
 
@@ -154,6 +156,7 @@ where
     Vec<Id<&'a Parent>>: AsInExpression<<Child::ForeignKeyColumn as Expression>::SqlType>,
     <Child as HasTable>::Table: FilterDsl<EqAny<Child::ForeignKeyColumn, Vec<Id<&'a Parent>>>>,
     Child::ForeignKeyColumn: ExpressionMethods,
+    <Child::ForeignKeyColumn as Expression>::SqlType: SqlType,
 {
     type Output = Filter<Child::Table, EqAny<Child::ForeignKeyColumn, Vec<Id<&'a Parent>>>>;
 

diesel/src/connection/mod.rs

@@ -6,10 +6,10 @@ mod transaction_manager;
 use std::fmt::Debug;
 
 use crate::backend::Backend;
-use crate::deserialize::{Queryable, QueryableByName};
+use crate::deserialize::{FromSqlRow, IsCompatibleType};
+use crate::expression::TypedExpressionType;
 use crate::query_builder::{AsQuery, QueryFragment, QueryId};
 use crate::result::*;
-use crate::sql_types::HasSqlType;
 
 #[doc(hidden)]
 pub use self::statement_cache::{MaybeCached, StatementCache, StatementCacheKey};
@@ -169,18 +169,13 @@ pub trait Connection: SimpleConnection + Sized + Send {
     fn execute(&self, query: &str) -> QueryResult<usize>;
 
     #[doc(hidden)]
-    fn query_by_index<T, U>(&self, source: T) -> QueryResult<Vec<U>>
+    fn load<T, U, ST>(&self, source: T) -> QueryResult<Vec<U>>
     where
         T: AsQuery,
         T::Query: QueryFragment<Self::Backend> + QueryId,
-        Self::Backend: HasSqlType<T::SqlType>,
-        U: Queryable<T::SqlType, Self::Backend>;
-
-    #[doc(hidden)]
-    fn query_by_name<T, U>(&self, source: &T) -> QueryResult<Vec<U>>
-    where
-        T: QueryFragment<Self::Backend> + QueryId,
-        U: QueryableByName<Self::Backend>;
+        U: FromSqlRow<ST, Self::Backend>,
+        T::SqlType: IsCompatibleType<Self::Backend, Compatible = ST>,
+        ST: TypedExpressionType;
 
     #[doc(hidden)]
     fn execute_returning_count<T>(&self, source: &T) -> QueryResult<usize>