Skip to content

treble37/nested_filter

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

136 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NestedFilter

Build Hex.pm Hex.pm Downloads GitHub license

Structure-preserving filter/reject for nested maps and lists: drop or take keys and values at any depth without flattening or losing data. Zero runtime dependencies.

Map.take/2 and Map.drop/2 only see the top level. NestedFilter walks the whole structure — maps inside maps, maps inside lists — and never merges sibling branches or invents values: what survives is always at the path where it appeared in the input.

Every example below is copied verbatim from a doctest, so it runs exactly as shown.

Recipes

Clean params before insert

Drop nil and blank values at any depth before handing user input to a changeset or query:

iex> params = %{"name" => "Ada", "bio" => nil, "address" => %{"city" => "London", "zip" => ""}}
iex> NestedFilter.drop_by_value(params, [nil, ""])
%{"name" => "Ada", "address" => %{"city" => "London"}}

Strip nils before JSON encoding

Remove every nil entry so encoded payloads carry no null noise:

iex> payload = %{id: 7, tags: ["a", "b"], meta: %{source: nil, ip: "1.2.3.4"}}
iex> NestedFilter.reject(payload, fn _k, v -> is_nil(v) end)
%{id: 7, tags: ["a", "b"], meta: %{ip: "1.2.3.4"}}

Drop sensitive keys everywhere

Remove known-bad keys wherever they appear, however deeply nested:

iex> event = %{user: %{email: "ada@example.com", password: "s3cret"}, session: %{token: "abc", ttl: 60}}
iex> NestedFilter.drop_by_key(event, [:password, :token])
%{user: %{email: "ada@example.com"}, session: %{ttl: 60}}

Take fields, structure preserved

Keep only the fields you care about without flattening or losing duplicates across branches:

iex> order = %{buyer: %{id: 1, name: "Ada"}, items: [%{id: 10, sku: "X"}, %{id: 11, sku: "Y"}]}
iex> NestedFilter.take_by_key(order, [:id])
%{buyer: %{id: 1}, items: [%{id: 10}, %{id: 11}]}

Sanitize logs

Redact by pattern when the exact key names aren't known up front:

iex> log = %{"msg" => "login ok", "user_password" => "hunter2", "ctx" => %{"api_token" => "xyz"}}
iex> NestedFilter.reject(log, fn k, _v -> is_binary(k) and (k =~ "password" or k =~ "token") end)
%{"msg" => "login ok", "ctx" => %{}}

Installation

Add nested_filter to your list of dependencies in mix.exs:

def deps do
  [{:nested_filter, "~> 2.0"}]
end

Requires Elixir 1.15 or later. Full documentation is at hexdocs.pm/nested_filter.

API

Two engine functions take a predicate receiving each key and value:

  • NestedFilter.reject/3 — recursively remove matching entries
  • NestedFilter.filter/3 — recursively keep matching entries, pruning branches without a match; a matched entry is kept whole

Three convenience functions cover the common cases:

  • NestedFilter.drop_by_value/3 — remove entries whose value is in a list
  • NestedFilter.drop_by_key/3 — remove entries whose key is in a list
  • NestedFilter.take_by_key/3 — keep entries whose key is in a list, structure preserved

Semantics worth knowing

  • Structure is sacred. Matches stay at the path where they were found. Sibling branches are never merged, so duplicate keys in different branches never clobber each other.
  • reject preserves empty maps; filter prunes empty branches. Rejecting every entry of a nested map leaves %{} at its path (add %{} to drop_by_value/3's list to remove those too), while filter drops any branch with no surviving content.
  • Lists are traversed, not filtered by value. reject leaves non-map list elements untouched; filter prunes list elements with no surviving content.
  • Structs are opaque leaves by default. Pass structs: :convert to recurse into them as plain maps, or structs: :error to raise if one is encountered. See the :structs option on reject/3.

Upgrading from 1.x

Version 2.0 changed take_by_key/3 from flattening (which silently lost data on duplicate keys) to structure-preserving, removed the undocumented drop_by/2 and take_by/2, and raised the Elixir floor to 1.15. See the CHANGELOG for the full migration table.

License

MIT — see LICENSE.

About

Providing Map#drop (by key or value) and Map#take functionality for nested maps

Topics

Resources

License

Stars

35 stars

Watchers

1 watching

Forks

Packages

 
 
 

Contributors

Languages