[C++][Dataset] Minimize Expression to a wrapper around compute::Function

[asfimport]

The Expression class hierarchy was originally intended to provide generic, structured representations of compute functionality. On the former point they have been superseded by compute::

{Function, Kernel, ...}

which encapsulates validation and execution. In light of this Expression can be drastically simplified and improved by composition with these classes. Each responsibility which can be deferred implies less boilerplate when exposing a new compute function for use in datasets. Ideally any compute function will be immediately available to use in a filter or projection.

struct Expression {
  using Literal = std::shared_ptr<Scalar>;

  struct Call {
    std::shared_ptr<ScalarFunction> function;
    std::shared_ptr<FunctionOptions> options;
    std::vector<Expression> arguments;
  };

  util::variant<Literal, FieldRef, Call> value;
};

A simple discriminated union as above should be sufficient to represent arbitrary filters and projections: any expression which results in type bool is a valid filter, and any expression which is a Projection may be used to map one record batch to another.

Expression simplification (currently implemented in Expression::Assume) is an optimization used for example in predicate pushdown, and therefore need not exhaustively cover the full space of available compute functions.

Reporter: Ben Kietzman / @bkietz
Assignee: Ben Kietzman / @bkietz

Related issues:

• [C++] Implement casts from date types to Timestamp (fixes)
• [C++] Persist SetLookupState in between usages of IsIn when filtering dataset batches (fixes)
• [C++][Dataset] Refactor Expression::Assume to return a Result (supercedes)

PRs and other links:

• GitHub Pull Request #8894

_{Note: This issue was originally created as ARROW-10322. Please see the migration documentation for further details.}

[asfimport]

Neal Richardson / @nealrichardson:
Issue resolved by pull request 8894
#8894