Improve Inspect for structs

CHANGELOG.md

@@ -108,7 +108,9 @@ TODO.
   * [Enum] Allow slicing with steps in `Enum.slice/2`
   * [Float] Do not show floats in scientific notation if below `1.0e16` and the fractional value is precisely zero
   * [Inspect] Improve error reporting when there is a faulty implementation of the `Inspect` protocol
-  * [Inspect] Use expression-based inspection for `Date.Range`, `MapSet`, `Version`, and `Version.Requirement`
+  * [Inspect] Allow `:optional` when deriving the Inspect protocol
+  * [Inspect] Inspect struct fields in the order they are defined
+  * [Inspect] Use expression-based inspection for `Date.Range`, `MapSet`, and `Version.Requirement`
   * [Kernel] Allow any guard expression as the size of a bitstring in a pattern match
   * [Kernel] Allow composite types with pins as the map key in a pattern match
   * [Kernel] Print escaped version of control chars when they show up as unexpected tokens

lib/elixir/lib/inspect.ex

@@ -8,16 +8,85 @@ defprotocol Inspect do
   The `Inspect` protocol converts an Elixir data structure into an
   algebra document.
 
+  This is typically done when you want to customize how your own
+  structs are inspected in logs and the terminal.
+
   This documentation refers to implementing the `Inspect` protocol
   for your own data structures. To learn more about using inspect,
   see `Kernel.inspect/2` and `IO.inspect/2`.
 
-  The `inspect/2` function receives the entity to be inspected
-  followed by the inspecting options, represented by the struct
-  `Inspect.Opts`. Building of the algebra document is done with
-  `Inspect.Algebra`.
+  ## Inspect representation
+
+  There are typically three choices of inspect representation. In order
+  to understand them, let's imagine we have the following `User` struct:
+
+      defmodule User do
+        defstruct [:id, :name, :address]
+      end
+
+  Our choices are:
+
+    1. Print the struct using Elixir's struct syntax, for example:
+       `%User{address: "Earth", id: 13, name: "Jane"}`. This is the
+       default representation and best choice if all struct fields
+       are public.
+
+    2. Print using the `#User<...>` notation, for example: `#User<id: 13, name: "Jane", ...>`.
+       This notation does not emit valid Elixir code and is typically
+       used when the struct has private fields (for example, you may want
+       to hide the field `:address` to redact person identifiable information).
+
+    3. Print the struct using the expression syntax, for example:
+       `User.new(13, "Jane", "Earth")`. This assumes there is a `User.new/3`
+       function. This option is mostly used as an alternative to option 2
+       for representing custom data structures, such as `MapSet`, `Date.Range`,
+       and others.
+
+  You can implement the Inspect protocol for your own structs while
+  adhering to the conventions above. Option 1 is the default representation
+  and you can quickly achieve option 2 by deriving the `Inspect` protocol.
+  For option 3, you need your custom implementation.
+
+  ## Deriving
+
+  The `Inspect` protocol can be derived to customize the order of fields
+  (the default is alphabetical) and hide certain fields from structs,
+  so they don't show up in logs, inspects and similar. The latter is
+  especially useful for fields containing private information.
+
+  The supported options are:
+
+    * `:only` - only include the given fields when inspecting.
+
+    * `:except` - remove the given fields when inspecting.
+
+    * `:optional` - (since v1.14.0) do not include a field if it
+      matches its default value. This can be used to simplify the
+      struct representation at the cost of hiding information.
 
-  ## Examples
+  Whenever `:only` or `:except` are used to restrict fields,
+  the struct will be printed using the `#User<...>` notation,
+  as the struct can no longer be copy and pasted as valid Elixir
+  code. Let's see an example:
+
+      defmodule User do
+        @derive {Inspect, only: [:id, :name]}
+        defstruct [:id, :name, :address]
+      end
+
+      inspect(%User{id: 1, name: "Jane", address: "Earth"})
+      #=> #User<id: 1, name: "Jane", ...>
+
+  If you use only the `:optional` option, the struct will still be
+  printed as `%User{...}`.
+
+  ## Custom implementation
+
+  You can also define your custom protocol implementation by
+  defining the `inspect/2` function. The function receives the
+  entity to be inspected followed by the inspecting options,
+  represented by the struct `Inspect.Opts`. Building of the
+  algebra document is done with `Inspect.Algebra`.
 
   Many times, inspecting a structure can be implemented in function
   of existing entities. For example, here is `MapSet`'s `inspect/2`
@@ -27,23 +96,24 @@ defprotocol Inspect do
         import Inspect.Algebra
 
         def inspect(map_set, opts) do
-          concat(["#MapSet<", to_doc(MapSet.to_list(map_set), opts), ">"])
+          concat(["MapSet.new(", Inspect.List.inspect(MapSet.to_list(map_set), opts), ")"])
         end
       end
 
   The [`concat/1`](`Inspect.Algebra.concat/1`) function comes from
   `Inspect.Algebra` and it concatenates algebra documents together.
-  In the example above it is concatenating the string `"#MapSet<"`,
+  In the example above it is concatenating the string `"MapSet.new("`,
   the document returned by `Inspect.Algebra.to_doc/2`, and the final
-  string `">"`. We prefix the module name `#` to denote the inspect
-  presentation is not actually valid Elixir syntax.
+  string `")"`. Therefore, the MapSet with the numbers 1, 2, and 3
+  will be printed as:
 
-  Finally, note strings themselves are valid algebra documents that
-  keep their formatting when pretty printed. This means your `Inspect`
-  implementation may simply return a string, although that will devoid
-  it of any pretty-printing.
+      iex> MapSet.new([1, 2, 3], fn x -> x * 2 end)
+      MapSet.new([2, 4, 6])
 
-  ## Error handling
+  In other words, `MapSet`'s inspect representation returns an expression
+  that, when evaluated, builds the `MapSet` itself.
+
+  ### Error handling
 
   In case there is an error while your structure is being inspected,
   Elixir will raise an `ArgumentError` error and will automatically fall back
@@ -55,24 +125,6 @@ defprotocol Inspect do
 
       Inspect.MapSet.inspect(MapSet.new(), %Inspect.Opts{})
 
-  ## Deriving
-
-  The `Inspect` protocol can be derived to hide certain fields from
-  structs, so they don't show up in logs, inspects and similar. This
-  is especially useful for fields containing private information.
-
-  The options `:only` and `:except` can be used with `@derive` to
-  specify which fields should and should not appear in the
-  algebra document:
-
-      defmodule User do
-        @derive {Inspect, only: [:id, :name]}
-        defstruct [:id, :name, :address]
-      end
-
-      inspect(%User{id: 1, name: "Homer", address: "742 Evergreen Terrace"})
-      #=> #User<id: 1, name: "Homer", ...>
-
   """
 
   # Handle structs in Any
@@ -250,29 +302,34 @@ end
 
 defimpl Inspect, for: Map do
   def inspect(map, opts) do
-    inspect(map, "", opts)
-  end
+    list = Map.to_list(map)
 
-  def inspect(map, name, opts) do
-    map = Map.to_list(map)
-    open = color("%" <> name <> "{", :map, opts)
-    sep = color(",", :map, opts)
-    close = color("}", :map, opts)
-    container_doc(open, map, close, opts, traverse_fun(map, opts), separator: sep, break: :strict)
+    fun =
+      if Inspect.List.keyword?(list) do
+        &Inspect.List.keyword/2
+      else
+        sep = color(" => ", :map, opts)
+        &to_assoc(&1, &2, sep)
+      end
+
+    map_container_doc(list, "", opts, fun)
   end
 
-  defp traverse_fun(list, opts) do
-    if Inspect.List.keyword?(list) do
-      &Inspect.List.keyword/2
-    else
-      sep = color(" => ", :map, opts)
-      &to_map(&1, &2, sep)
-    end
+  def inspect(map, name, infos, opts) do
+    fun = fn %{field: field}, opts -> Inspect.List.keyword({field, Map.get(map, field)}, opts) end
+    map_container_doc(infos, name, opts, fun)
   end
 
-  defp to_map({key, value}, opts, sep) do
+  defp to_assoc({key, value}, opts, sep) do
     concat(concat(to_doc(key, opts), sep), to_doc(value, opts))
   end
+
+  defp map_container_doc(list, name, opts, fun) do
+    open = color("%" <> name <> "{", :map, opts)
+    sep = color(",", :map, opts)
+    close = color("}", :map, opts)
+    container_doc(open, list, close, opts, fun, separator: sep, break: :strict)
+  end
 end
 
 defimpl Inspect, for: Integer do
@@ -454,72 +511,99 @@ defimpl Inspect, for: Any do
     fields = Map.keys(struct) -- [:__exception__, :__struct__]
     only = Keyword.get(options, :only, fields)
     except = Keyword.get(options, :except, [])
+    optional = Keyword.get(options, :optional, [])
 
-    :ok = validate_option(only, fields, module)
-    :ok = validate_option(except, fields, module)
+    :ok = validate_option(:only, only, fields, module)
+    :ok = validate_option(:except, except, fields, module)
+    :ok = validate_option(:optional, optional, fields, module)
+
+    inspect_module =
+      if fields == only and except == [] do
+        Inspect.Map
+      else
+        Inspect.Any
+      end
 
     filtered_fields =
       fields
       |> Enum.reject(&(&1 in except))
       |> Enum.filter(&(&1 in only))
 
-    inspect_module =
-      if fields == only and except == [] do
-        Inspect.Map
+    optional? =
+      if optional == [] do
+        false
       else
-        Inspect.Any
+        optional_map = for field <- optional, into: %{}, do: {field, Map.fetch!(struct, field)}
+
+        quote do
+          case unquote(Macro.escape(optional_map)) do
+            %{^var!(field) => var!(default)} ->
+              var!(default) == Map.get(var!(struct), var!(field))
+
+            %{} ->
+              false
+          end
+        end
       end
 
     quote do
       defimpl Inspect, for: unquote(module) do
         def inspect(var!(struct), var!(opts)) do
-          var!(map) = Map.take(var!(struct), unquote(filtered_fields))
+          var!(infos) =
+            for %{field: var!(field)} = var!(info) <- unquote(module).__info__(:struct),
+                var!(field) in unquote(filtered_fields) and not unquote(optional?),
+                do: var!(info)
+
           var!(name) = Macro.inspect_atom(:literal, unquote(module))
-          unquote(inspect_module).inspect(var!(map), var!(name), var!(opts))
+          unquote(inspect_module).inspect(var!(struct), var!(name), var!(infos), var!(opts))
         end
       end
     end
   end
 
-  defp validate_option(option_list, fields, module) do
+  defp validate_option(option, option_list, fields, module) do
     case option_list -- fields do
       [] ->
         :ok
 
       unknown_fields ->
         raise ArgumentError,
-              "unknown fields #{Kernel.inspect(unknown_fields)} given when deriving the Inspect protocol for #{Kernel.inspect(module)}. :only and :except values must match struct fields"
+              "unknown fields #{Kernel.inspect(unknown_fields)} in #{Kernel.inspect(option)} " <>
+                "when deriving the Inspect protocol for #{Kernel.inspect(module)}"
     end
   end
 
   def inspect(%module{} = struct, opts) do
     try do
-      module.__struct__()
+      {module.__struct__(), module.__info__(:struct)}
     rescue
       _ -> Inspect.Map.inspect(struct, opts)
     else
-      dunder ->
+      {dunder, fields} ->
         if Map.keys(dunder) == Map.keys(struct) do
-          pruned = Map.drop(struct, [:__struct__, :__exception__])
-          Inspect.Map.inspect(pruned, Macro.inspect_atom(:literal, module), opts)
+          infos =
+            for %{field: field} = info <- fields,
+                field not in [:__struct__, :__exception__],
+                do: info
+
+          Inspect.Map.inspect(struct, Macro.inspect_atom(:literal, module), infos, opts)
         else
           Inspect.Map.inspect(struct, opts)
         end
     end
   end
 
-  def inspect(map, name, opts) do
-    map = Map.to_list(map) ++ [:...]
+  def inspect(map, name, infos, opts) do
     open = color("#" <> name <> "<", :map, opts)
     sep = color(",", :map, opts)
     close = color(">", :map, opts)
 
     fun = fn
-      {key, value}, opts -> Inspect.List.keyword({key, value}, opts)
+      %{field: field}, opts -> Inspect.List.keyword({field, Map.get(map, field)}, opts)
       :..., _opts -> "..."
     end
 
-    container_doc(open, map, close, opts, fun, separator: sep, break: :strict)
+    container_doc(open, infos ++ [:...], close, opts, fun, separator: sep, break: :strict)
   end
 end
 

lib/elixir/lib/inspect/algebra.ex

@@ -469,8 +469,9 @@ defmodule Inspect.Algebra do
 
   defp container_each([term | terms], limit, opts, fun, acc, simple?)
        when is_list(terms) and is_limit(limit) do
-    limit = decrement(limit)
-    doc = fun.(term, %{opts | limit: limit})
+    new_limit = decrement(limit)
+    doc = fun.(term, %{opts | limit: new_limit})
+    limit = if doc == :doc_nil, do: limit, else: new_limit
     container_each(terms, limit, opts, fun, [doc | acc], simple? and simple?(doc))
   end
 

lib/elixir/lib/kernel.ex

@@ -5119,13 +5119,21 @@ defmodule Kernel do
   list before invoking `defstruct/1`:
 
       defmodule User do
-        @derive [MyProtocol]
-        defstruct name: nil, age: 10 + 11
+        @derive MyProtocol
+        defstruct name: nil, age: nil
       end
 
       MyProtocol.call(john) # it works!
 
-  For each protocol in the `@derive` list, Elixir will assert the protocol has
+  A common example is to `@derive` the `Inspect` protocol to hide certain fields
+  when the struct is printed:
+
+      defmodule User do
+        @derive {Inspect, only: :name}
+        defstruct name: nil, age: nil
+      end
+
+  For each protocol in `@derive`, Elixir will assert the protocol has
   been implemented for `Any`. If the `Any` implementation defines a
   `__deriving__/3` callback, the callback will be invoked and it should define
   the implementation module. Otherwise an implementation that simply points to