Huginn

Performant, elegant ActiveRecord datatables and tolerant search.
Huginn is the raven of Odin that represents thought and remembrance — the mate of Muninn.

🇺🇸 English · 🇧🇷 Português

Huginn is a lightweight query layer for Rails that turns a raw datatable request into a lean count, a paginated subset and one preload — instead of a massive JOIN materialized in memory. It also ships a PostgreSQL fuzzy-search builder (pg_trgm similarity with unaccent and ILIKE fallback) that is tolerant to typos and accents.

• Two-phase execution — association filters/orders/range become reflection-secured subqueries, then a lean count and preload only on the paginated subset.
• Lean counts — COUNT(DISTINCT pk) through a stripped relation; no JOIN materialization.
• SQL injection safe ordering/filtering — every column reference is resolved through Arel reflection, never string-interpolated.
• Accent/typo tolerant search — pg_trgm similarity OR unaccent+ILIKE, with a configurable fallback chain.
• Rails conventions — works with ActionController::Parameters, Railtie auto-includes both concerns (toggleable), zero boilerplate.

The root Gemfile keeps only the tooling (rspec, appraisal, pry) — each supported Rails series lives in its own Appraisal. Use these to run the suite:

bundle install
bundle exec appraisal install        # generates gemfiles/*.gemfile + resolves
bundle exec appraisal rspec          # runs the full matrix (Rails 7.1/7.2/8.0)
bundle exec appraisal rails-8.0 rspec   # or a single series
bundle exec rake matrix              # alias for the full matrix

A bare bundle exec rspec needs an active environment: export BUNDLE_GEMFILE=gemfiles/rails_8.0.gemfile.

The suite is verified against Rails 7.1, 7.2 and 8.0 across supported Rubies via Appraisal. Run the full matrix locally:

bundle exec appraisal install
bundle exec appraisal rspec

The gemfiles/*.gemfile are generated by Appraisal (committed); their .lock files are not — each CI cell resolves for its own Ruby/Rails pair.

gem "huginn"

# config/initializers/huginn.rb
Huginn.configure do |config|
  # :pg_trgm  (recommended) — trigram similarity OR unaccent+ILIKE
  # :unaccent               — unaccent + ILIKE only
  # :simple                 — plain LIKE
  config.search_strategy = :pg_trgm

  config.fuzzy_threshold = 0.3   # similarity() cutoff used by :pg_trgm
  config.pagy_items = 10         # default page size
  config.pagy_max_items = 500    # hard cap for per_page
end

By default the Railtie includes Huginn::Datatable and Huginn::Searchable into every ActiveRecord::Base model. You do not need include statements unless you opt in selectively:

Huginn.configure { |c| c.auto_include_datatable = false; c.auto_include_searchable = false }

class Plano < ApplicationRecord
  # Datatable + Searchable are auto-included via Railtie
end

result = Plano.datatable(
  params,
  allowed_paths: [:grupo, { operadora: [:pessoa] }], # associations filters/orders may use
  includes: [{ operadora: { pessoa: [:endereco, :contatos] } }] # preloaded on the page only
)

result[:total_count] # Integer (lean COUNT DISTINCT pk)
result[:data]        # ActiveRecord::Relation (paged + preloaded)

Supported params:

Any column or association.column reference is validated and mapped to its real reflected table:

Plano.datatable({ orders: [{ "operadora.pessoa.nome" => "asc" }] }, allowed_paths: [{ operadora: :pessoa }])

Association filters/range and ordering use reflection-resolved subqueries (see the "allowlist of associations" section below). The main relation stays singular and the count is COUNT(DISTINCT pk).

To protect the schema and keep the query lean, the datatable does not materialize left_joins to filter/order by associations. Instead:

• Filters/ranges over association columns become pk IN (SELECT DISTINCT pk …) subqueries — the main relation is never multiplied;
• Ordering by an association column uses a correlated scalar subquery (ORDER BY (SELECT … ORDER BY col ASC LIMIT 1)), which is deterministic even for has_many (smallest value);
• Only authorized associations may be referenced. Pass allowed_paths: with the associations the caller may use in the query:

result = Plano.datatable(
  params,
  allowed_paths: [:grupo, { operadora: :pessoa }],  # associations filters/orders may use
  includes:      [{ operadora: { pessoa: [:endereco, :contatos] } }] # preload only the page
)

• Deny-all by default: without allowed_paths:, no association is authorized for filtering/ordering — only columns of the table itself.
• allowed_paths: accepts the same shapes Rails knows (:symbol, "string", nested Hash, mixed Array). Table names ("companies") are recognized as the matching association (:company).
• includes: stays independent of allowed_paths: — it only controls the preload of the paginated page.

Public APIs should not expose the database schema. Declare a mapping of public names to real columns/tables with huginn_attributes:

class User < ApplicationRecord
  # Public API name -> real column/table (association name or table name)
  huginn_attributes(
    name:         "users.name",
    email:        "users.email",
    created_at:   "users.created_at",
    company_name: "companies.name"   # association column, resolved via subquery
  )
end

• Callers then filter/order/range only by the aliases: { filters: { company_name: "Acme Corp" } }, { orders: [{ company_name: "asc" }] }.
• Strict by default: fields outside the mapping are silently rejected (they never reach SQL and are never answered). The schema stays hidden from API consumers.
• Scoped aliases (companies.name) resolve through the association only if it is authorized in allowed_paths: (the same allowlist applies to aliases).
• Without huginn_attributes, the model falls back to plain/reflected columns (name, company.name).
• Without allowed_paths:, association-scoped aliases are denied; only plain columns are usable.
• huginn_attributes({ ... }, strict: false) keeps alias translation but also accepts raw columns.

# Default: searches every :string / :text column of the model.
Person.search("kayky")            # typo/accent tolerant, case-insensitive

# Override which columns (including through associations) are searched:
class Person < ApplicationRecord
  searchable_columns :name, company: [:name, :cnpj]
end

Person.search("globex")                            # matches company.name via a left_join
Person.search("kayky", distinct: false)            # disable the implicit DISTINCT

Huginn::Datatable reuses Huginn::Searchable.search automatically when the model responds to search.

phase 1  build the relation          subqueries (pk IN … / ORDER BY (SELECT …)) + search + filters + order   (no data in memory)
phase 2  count                       SELECT COUNT(DISTINCT "<pk column>") ... (subquery, pk-indexed)
          paginate                   offset / limit
          preload                    SELECT ... WHERE id IN (subset)        (2nd lightweight query)

For a Plano datatable with deep includes:, this is exactly 2 extra queries on the small page instead of one enormous JOIN.

lib/huginn.rb                       entry, Huginn.configure, Huginn.instrument
lib/huginn/configuration.rb         search_strategy, fuzzy_threshold, pagy_*
lib/huginn/railtie.rb               auto-includes concerns into ActiveRecord
lib/huginn/datatable.rb             Huginn::Datatable (aggregator)
lib/huginn/datatable/datatable.rb   the datatable Concern
lib/huginn/datatable/validator.rb   column/association validation + Arel resolution
lib/huginn/datatable/association_path.rb   resolution of association chains + correlated subqueries
lib/huginn/datatable/allowed_paths.rb      `allowed_paths:` allowlist expansion/authorization
lib/huginn/datatable/filter_normalizer.rb  functional param normalization
lib/huginn/datatable/paginator.rb    lean count, pagination, isolated preload
lib/huginn/searchable.rb            Huginn::Searchable (aggregator)
lib/huginn/searchable/searchable.rb the search Concern + DSL
lib/huginn/searchable/query.rb      tolerant search builder (joins + OR)
lib/huginn/searchable/fuzzy.rb      pg_trgm / unaccent / simple predicates

Huginn.instrument wraps ActiveSupport::Notifications events under the huginn namespace (e.g. datatable.call.huginn). Subscribe with ActiveSupport::Notifications.subscribe(/\.huginn/).

MIT