Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expressions in AggregationPicker without legacy query #36031

Merged
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
e48ab06
Add expression-clause-for-legacy-expression
metamben Nov 17, 2023
e4df56e
Add legacy-expression-for-expression-clause
metamben Nov 20, 2023
e93d114
Fix wrapper function name
metamben Nov 20, 2023
76054bf
Merge remote-tracking branch 'origin/master' into 34830-expression-cl…
metamben Nov 20, 2023
0994d76
Include ExpressionClause in Aggregatable type
kamilmielnik Nov 21, 2023
55b4f31
Use Aggregatable type in aggregate function
kamilmielnik Nov 21, 2023
95697f7
Rename query to legacyQuery in Expressions (part 1)
kamilmielnik Nov 21, 2023
d6eb1dc
Rename query to legacyQuery in Expressions (part 2)
kamilmielnik Nov 22, 2023
eb24d0d
Rename query to legacyQuery in Expressions (part 3)
kamilmielnik Nov 22, 2023
3233e98
Merge branch 'master' into 35947-legacy-expression-clause-filters-and…
kamilmielnik Nov 22, 2023
9ffa599
Merge branch 'refactor/rename-query-to-legacy-query' into 35947-legac…
kamilmielnik Nov 22, 2023
3d59ab3
Merge branch 'master' into 35947-legacy-expression-clause-filters-and…
kamilmielnik Nov 22, 2023
3e4fb53
Support MLv2 expressionClause in processSource and ExpressionWidget
kamilmielnik Nov 22, 2023
4eb9eda
Revert changes to ExpressionStep as it's out of scope of this task
kamilmielnik Nov 22, 2023
3eb2f3c
Merge branch 'master' into 35947-legacy-expression-clause-filters-and…
kamilmielnik Nov 23, 2023
cadf4ed
Use new expressionClause in ExpressionWidget
kamilmielnik Nov 23, 2023
20e9406
Use onSelect instead of onSelectLegacy in handleExpressionChange
kamilmielnik Nov 23, 2023
248009a
Remove onSelectLegacy from AggregationPicker
kamilmielnik Nov 23, 2023
35dbf4b
Merge branch 'master' into 35947-legacy-expression-clause-filters-and…
kamilmielnik Nov 24, 2023
7ae92bd
Use nullish coalescing operator
kamilmielnik Nov 24, 2023
8646317
Assert new onChangeExpression argument in test
kamilmielnik Nov 24, 2023
71ac3aa
Assert new onChangeExpression argument in test
kamilmielnik Nov 24, 2023
30b9b27
Update AggregationPicker tests
kamilmielnik Nov 24, 2023
2f50b3f
Replace toEqual + expect.objectContaining with toMatchObject
kamilmielnik Nov 24, 2023
24a131c
Add stageIndex to setup, use destructuring for onSelect arguments
kamilmielnik Nov 24, 2023
5c75c83
Deal with awkward assertions
kamilmielnik Nov 24, 2023
d64c7c4
Use props in a conservative way
kamilmielnik Nov 24, 2023
89041b9
Replace legacyQuery.database() with query + metadata
kamilmielnik Nov 24, 2023
39496c1
Add a TODO comment
kamilmielnik Nov 24, 2023
221e950
Add clause prop to AggregationPicker
kamilmielnik Nov 24, 2023
0573ab3
Migrate clause name
kamilmielnik Nov 24, 2023
a9d7b5b
Replace 3rd argument in onChangeExpression with onChangeExpressionClause
kamilmielnik Nov 24, 2023
079abb5
Use destructuring
kamilmielnik Nov 27, 2023
7658ec9
Allow AggregationClause, drop expression from ExpressionEditorTextfield
kamilmielnik Nov 27, 2023
1f13259
Make withExpressionName work with AggregationClause
kamilmielnik Nov 27, 2023
94bc358
Rename expressionClause to clause
kamilmielnik Nov 27, 2023
f36fc18
Update tests with new interface
kamilmielnik Nov 27, 2023
3953f85
Use overloading instead of generics
kamilmielnik Nov 27, 2023
2074e8b
Add function body
kamilmielnik Nov 27, 2023
ee9d538
Rename operator to clause
kamilmielnik Nov 27, 2023
f18cd75
Revert "Use overloading instead of generics"
kamilmielnik Nov 27, 2023
7baa434
Fix tests failing due to useSelect usage in AggregationPicker
kamilmielnik Nov 27, 2023
f28b10a
Omit aggregation options converting expressions to legacy
metamben Nov 27, 2023
bdad147
Merge branch '36120-omit-aggregation-options-converting-exprs-to-lega…
kamilmielnik Nov 28, 2023
1e2485c
Remove temporary hack
kamilmielnik Nov 28, 2023
505d11f
Get rid of props spread
kamilmielnik Nov 28, 2023
bacf371
Migrate isExpressionEditorInitiallyOpen to MLv2
kamilmielnik Nov 28, 2023
95ddd4c
Omit aggregation options converting expressions to legacy
metamben Nov 27, 2023
58bd581
Normalize legacy expressions as MBQL expressions
metamben Nov 28, 2023
c4866fd
Merge branch '36120-omit-aggregation-options-converting-exprs-to-lega…
kamilmielnik Nov 29, 2023
8afeac3
Fix isExpressionEditorInitiallyOpen
kamilmielnik Nov 29, 2023
7f7152e
Update expressionName signature
kamilmielnik Nov 29, 2023
3eb2181
Pass props in ExpressionStep tests in a usual way
kamilmielnik Nov 29, 2023
c36cebd
Drop a conditional statement
kamilmielnik Nov 29, 2023
594b389
Bring back expression prop for backwards-compatibility
kamilmielnik Nov 29, 2023
dcd7781
Update ExpressionWidget validation & tests
kamilmielnik Nov 29, 2023
0a0802a
Bring back using expression in componentWillReceiveProps
kamilmielnik Nov 29, 2023
f7d14cf
Bring back removed assertion
kamilmielnik Nov 29, 2023
92aa48f
Remove legacyQuery prop from AggregationPicker
kamilmielnik Nov 29, 2023
e719fc2
Add a test case for isExpressionEditorInitiallyOpen (which uses Lib.e…
kamilmielnik Nov 29, 2023
0aec027
Fix legacyQuery creation
kamilmielnik Nov 29, 2023
45a50d3
Add assertion for expression name
kamilmielnik Nov 29, 2023
b5bfee8
Fix committing expression with done button
kamilmielnik Nov 29, 2023
db71907
Improve assertions in tests
kamilmielnik Nov 29, 2023
3b8f15b
Make with-expression-name set the display-name of the clause
metamben Nov 29, 2023
fd32c5f
Add expression name assertion
kamilmielnik Nov 30, 2023
33aec8f
Add expression name assertion
kamilmielnik Nov 30, 2023
e80ac60
Add expression name assertion
kamilmielnik Nov 30, 2023
6a0c758
Add expression name assertion
kamilmielnik Nov 30, 2023
2795c3d
Add expression name assertion
kamilmielnik Nov 30, 2023
9f85ea9
Add expression name assertion
kamilmielnik Nov 30, 2023
a7a9ec6
Add expression name assertion
kamilmielnik Nov 30, 2023
49dfc00
Unskip an e2e test
kamilmielnik Nov 30, 2023
a7ed491
Add expression name assertion
kamilmielnik Nov 30, 2023
fc35005
Add expression name assertion
kamilmielnik Nov 30, 2023
da11fb9
Use displayInfo instead of displayName
kamilmielnik Nov 30, 2023
42d5ff1
Deprecate displayName in favor of displayInfo
kamilmielnik Nov 30, 2023
9acc7ba
Use clearer notation
kamilmielnik Nov 30, 2023
91be09a
Format code
kamilmielnik Nov 30, 2023
cb68901
Use displayInfo instead of expressionName in tests
kamilmielnik Dec 1, 2023
f699c90
Add isNamed attribute to ColumnDisplayInfo and ClauseDisplayInfo
kamilmielnik Dec 1, 2023
73283d9
Use displayInfo instead of expressionName in AggregationPicker
kamilmielnik Dec 1, 2023
2d6ca04
Drop expressionName
kamilmielnik Dec 1, 2023
31ce8e7
Drop redundant cast
kamilmielnik Dec 1, 2023
86c6a80
Simplify isExpressionEditorInitiallyOpen interface
kamilmielnik Dec 1, 2023
3796af4
Merge remote-tracking branch 'origin/master' into 36167-with-expressi…
metamben Dec 1, 2023
1ee3df6
Use display-info with :named? instead of expression-name
metamben Dec 1, 2023
04e4c16
Merge remote-tracking branch 'origin/master' into 36167-with-expressi…
metamben Dec 1, 2023
0bfcedd
Improve comment
metamben Dec 1, 2023
c3e015e
Add :named? to the schema of display-info
metamben Dec 1, 2023
f6d7c0a
Use conditional types for ExpressionWidget prop
kamilmielnik Dec 4, 2023
7cf0bb7
Use getNotebookStep
kamilmielnik Dec 4, 2023
e6a0bed
Simplify condition
kamilmielnik Dec 4, 2023
8780eee
Simplify assertions
kamilmielnik Dec 4, 2023
98ab29f
Use .lastCall
kamilmielnik Dec 4, 2023
34b87a2
Remove isNamed attribute from ColumnDisplayInfo
kamilmielnik Dec 4, 2023
dd0da11
Use .lastCall
kamilmielnik Dec 4, 2023
a0777d8
Merge branch '36167-with-expression-name-to-set-display-name' into 35…
kamilmielnik Dec 4, 2023
8d14a11
Fix typing
kamilmielnik Dec 4, 2023
3ef1bcb
Bring back legacyQuery prop to AggregationPicker
kamilmielnik Dec 4, 2023
da95422
Fix isExpressionEditorInitiallyOpen for new clauses
kamilmielnik Dec 4, 2023
277766a
Fix unit tests
kamilmielnik Dec 4, 2023
e582696
Remove wrong assertion
kamilmielnik Dec 4, 2023
db821e0
Fix assertion
kamilmielnik Dec 4, 2023
254913d
Trigger a change
kamilmielnik Dec 4, 2023
a76248a
Revert "Trigger a change"
kamilmielnik Dec 4, 2023
13571b1
Fix assertion
kamilmielnik Dec 4, 2023
6563742
Merge branch 'master' into 36167-with-expression-name-to-set-display-…
kamilmielnik Dec 5, 2023
f6901a6
Merge branch '36167-with-expression-name-to-set-display-name' into 35…
kamilmielnik Dec 5, 2023
24937fd
Merge branch 'master' into 35947-legacy-expression-clause-filters-and…
kamilmielnik Dec 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions frontend/src/metabase-lib/aggregation.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import * as ML from "cljs/metabase.lib.js";
import type {
Aggregatable,
AggregationClause,
AggregationOperator,
ColumnMetadata,
MetricMetadata,
Query,
} from "./types";

Expand All @@ -30,7 +30,7 @@ export function selectedAggregationOperators(
export function aggregate(
query: Query,
stageIndex: number,
clause: AggregationClause | MetricMetadata,
clause: Aggregatable,
): Query {
return ML.aggregate(query, stageIndex, clause);
}
Expand Down
17 changes: 15 additions & 2 deletions frontend/src/metabase-lib/expressions/process.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { t } from "ttag";

import * as Lib from "metabase-lib";

import { parse, adjustBooleans } from "./recursive-parser";
import { resolve } from "./resolver";

Expand Down Expand Up @@ -31,13 +33,23 @@ export function processSource(options) {
}
};

const { source, startRule } = options;
const { source, query, stageIndex, startRule } = options;

let expression;
let expression = null;
let expressionClause = null;
Comment on lines +38 to +39
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusting to match onChange and onCommit props in ExpressionEditorTextfield which were/are typed with null.

let compileError;
try {
const parsed = parse(source);
expression = adjustBooleans(resolve(parsed, startRule, resolveMBQLField));

// query and stageIndex are not available outside of notebook editor (e.g. in Metrics or Segments).
if (typeof query !== "undefined" && typeof stageIndex !== "undefined") {
kamilmielnik marked this conversation as resolved.
Show resolved Hide resolved
expressionClause = Lib.expressionClauseForLegacyExpression(
query,
stageIndex,
expression,
);
}
} catch (e) {
console.warn("compile error", e);
compileError = e;
Expand All @@ -46,6 +58,7 @@ export function processSource(options) {
return {
source,
expression,
expressionClause,
compileError,
};
}
5 changes: 4 additions & 1 deletion frontend/src/metabase-lib/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@ export type MetricMetadata = unknown & { _opaque: typeof MetricMetadata };
declare const AggregationClause: unique symbol;
export type AggregationClause = unknown & { _opaque: typeof AggregationClause };

export type Aggregatable = AggregationClause | MetricMetadata;
export type Aggregatable =
| AggregationClause
| MetricMetadata
| ExpressionClause;

declare const AggregationOperator: unique symbol;
export type AggregationOperator = unknown & {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,7 @@ import { useToggle } from "metabase/hooks/use-toggle";
import { ExpressionWidget } from "metabase/query_builder/components/expressions/ExpressionWidget";
import { ExpressionWidgetHeader } from "metabase/query_builder/components/expressions/ExpressionWidgetHeader";

import type {
Aggregation as LegacyAggregationClause,
Expression as LegacyExpressionClause,
} from "metabase-types/api";
import type { Expression as LegacyExpressionClause } from "metabase-types/api";
import * as Lib from "metabase-lib";
import * as AGGREGATION from "metabase-lib/queries/utils/aggregation";
import type LegacyAggregation from "metabase-lib/queries/structured/Aggregation";
Expand Down Expand Up @@ -40,7 +37,6 @@ interface AggregationPickerProps {
legacyClause?: LegacyAggregation;
maxHeight?: number;
onSelect: (operator: Lib.Aggregatable) => void;
onSelectLegacy: (operator: LegacyAggregationClause) => void;
onClose?: () => void;
}

Expand Down Expand Up @@ -75,7 +71,6 @@ export function AggregationPicker({
legacyClause,
maxHeight = DEFAULT_MAX_HEIGHT,
onSelect,
onSelectLegacy,
onClose,
}: AggregationPickerProps) {
const [
Expand Down Expand Up @@ -202,18 +197,27 @@ export function AggregationPicker({
);

const handleExpressionChange = useCallback(
(name: string, expression: LegacyExpressionClause) => {
const aggregation = AGGREGATION.setName(expression, name);
onSelectLegacy(aggregation as LegacyAggregationClause);
(
name: string,
_expression: LegacyExpressionClause,
expressionClause: Lib.ExpressionClause,
) => {
const updatedExpressionClause = Lib.withExpressionName(
expressionClause,
name,
);
onSelect(updatedExpressionClause);
onClose?.();
},
[onSelectLegacy, onClose],
[onSelect, onClose],
);

if (isEditingExpression) {
return (
<ExpressionWidget
legacyQuery={legacyQuery}
query={query}
stageIndex={stageIndex}
name={AGGREGATION.getName(legacyClause)}
expression={AGGREGATION.getContent(legacyClause)}
withName
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,6 @@ function setup({
: baseOperators;

const onSelect = jest.fn();
const onSelectLegacy = jest.fn();

function handleSelect(clause: Lib.Aggregatable) {
const nextQuery = Lib.aggregate(query, 0, clause);
Expand All @@ -159,7 +158,6 @@ function setup({
operators={operators}
hasExpressionInput={hasExpressionInput}
onSelect={handleSelect}
onSelectLegacy={onSelectLegacy}
/>,
);

Expand All @@ -168,7 +166,7 @@ function setup({
return lastCall?.[0];
}

return { metadata, getRecentClause, onSelectLegacy };
return { metadata, getRecentClause, onSelect };
}

describe("AggregationPicker", () => {
Expand Down Expand Up @@ -392,15 +390,15 @@ describe("AggregationPicker", () => {

describe("custom expressions", () => {
it("should allow to enter a custom expression", async () => {
const { onSelectLegacy } = setup();
const { onSelect } = setup();

userEvent.click(screen.getByText("Custom Expression"));
userEvent.type(screen.getByLabelText("Expression"), "1 + 1");
userEvent.type(screen.getByLabelText("Name"), "My expression");
userEvent.click(screen.getByRole("button", { name: "Done" }));

await waitFor(() =>
expect(onSelectLegacy).toHaveBeenCalledWith([
expect(onSelect).toHaveBeenCalledWith([
"aggregation-options",
["+", 1, 1],
{ "display-name": "My expression", name: "My expression" },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import * as ace from "ace-builds/src-noconflict/ace";
import type { Ace } from "ace-builds";
import type { Expression } from "metabase-types/api";
import ExplicitSize from "metabase/components/ExplicitSize";
import type * as Lib from "metabase-lib";
import { format } from "metabase-lib/expressions/format";
import { processSource } from "metabase-lib/expressions/process";
import { diagnose } from "metabase-lib/expressions/diagnostics";
Expand Down Expand Up @@ -47,15 +48,23 @@ interface ExpressionEditorTextfieldProps {
expression: Expression | undefined;
name: string;
legacyQuery: StructuredQuery;
query?: Lib.Query;
stageIndex?: number;
startRule?: string;
width?: number;
reportTimezone?: string;
textAreaId?: string;

onChange: (expression: Expression | null) => void;
onChange: (
expression: Expression | null,
expressionClause: Lib.ExpressionClause | null,
) => void;
onError: (error: ErrorWithMessage | null) => void;
onBlankChange: (isBlank: boolean) => void;
onCommit: (expression: Expression | null) => void;
onCommit: (
expression: Expression | null,
expressionClause: Lib.ExpressionClause | null,
) => void;
helpTextTarget: RefObject<HTMLElement>;
}

Expand Down Expand Up @@ -279,23 +288,29 @@ class ExpressionEditorTextfield extends React.Component<
return;
}

const { onChange, onError } = this.props;

this.clearSuggestions();

const errorMessage = this.diagnoseExpression();
this.setState({ errorMessage });

// whenever our input blurs we push the updated expression to our parent if valid
if (errorMessage) {
this.props.onError(errorMessage);
onError(errorMessage);
} else {
const expression = this.compileExpression();
if (expression) {
const compiledExpression = this.compileExpression();

if (compiledExpression) {
const { expression, expressionClause } = compiledExpression;

if (!isExpression(expression)) {
console.warn("isExpression=false", expression);
}
this.props.onChange(expression);

onChange(expression, expressionClause);
} else {
this.props.onError({ message: t`Invalid expression` });
onError({ message: t`Invalid expression` });
}
}
};
Expand Down Expand Up @@ -331,18 +346,20 @@ class ExpressionEditorTextfield extends React.Component<

compileExpression() {
const { source } = this.state;
const { legacyQuery, startRule, name } = this.props;
const { legacyQuery, query, stageIndex, startRule, name } = this.props;
if (!source || source.length === 0) {
return null;
}
const { expression } = processSource({
const { expression, expressionClause } = processSource({
name,
source,
query,
legacyQuery,
stageIndex,
startRule,
});

return expression;
return { expression, expressionClause };
}

diagnoseExpression(): ErrorWithMessage | null {
Expand All @@ -362,6 +379,8 @@ class ExpressionEditorTextfield extends React.Component<
const {
legacyQuery,
startRule = ExpressionEditorTextfield.defaultProps.startRule,
onCommit,
onError,
} = this.props;
const { source } = this.state;
const errorMessage = diagnose(
Expand All @@ -372,12 +391,16 @@ class ExpressionEditorTextfield extends React.Component<
this.setState({ errorMessage });

if (errorMessage) {
this.props.onError(errorMessage);
onError(errorMessage);
} else {
const expression = this.compileExpression();
const compiledExpression = this.compileExpression();

if (compiledExpression) {
kamilmielnik marked this conversation as resolved.
Show resolved Hide resolved
const { expression, expressionClause } = compiledExpression;

if (isExpression(expression)) {
this.props.onCommit(expression);
if (isExpression(expression)) {
onCommit(expression, expressionClause);
}
}
}
}
Expand Down