-
Notifications
You must be signed in to change notification settings - Fork 6.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(editor): Execution filter #5496
Merged
Merged
Changes from 75 commits
Commits
Show all changes
76 commits
Select commit
Hold shift + click to select a range
80cf38e
wip: workflow execution filtering
valya 24f6ac2
fix: import type failing to build
valya 80c7ac8
fix: remove console.logs
valya 677126b
feat: execution metadata migrations
valya 4729b5c
fix(editor): Move global executions filter to its own component
cstuncsik e385f70
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik f58e071
fix(editor): Using the same filter component in workflow level
cstuncsik fddb840
fix(editor): a small housekeeping
cstuncsik 7885743
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik 3357a74
Merge remote-tracking branch 'origin/master' into pay-36-store-execut…
cstuncsik 2c42a0d
checking workflowId in filter applied
cstuncsik 8871c38
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik 6455ef2
Merge remote-tracking branch 'origin/master' into pay-36-store-execut…
cstuncsik bdfc3d3
fix(editor): update filter after resolving merge conflicts
cstuncsik e77f27e
fix(editor): unify empy filter status
cstuncsik a1be679
Merge remote-tracking branch 'origin/master' into pay-36-store-execut…
cstuncsik 7958e3e
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik 38b04a9
feat(editor): add datetime picker to filter
cstuncsik 4497ea0
feat(editor): add meta fields
cstuncsik 8a1b9b3
fix: fix button override in datepicker panel
alexgrozav 44555da
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik e724de0
feat(editor): add filter metadata
cstuncsik de56350
Merge remote-tracking branch 'origin/master' into pay-36-store-execut…
cstuncsik b11077b
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik fbcb4bc
feat(core): add 'startedBefore' execution filter prop
cstuncsik a7b9680
feat(core): add 'tags' execution query filter
cstuncsik b787c13
Merge remote-tracking branch 'origin/pay-36-store-execution-metadata'…
cstuncsik d8911b3
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik 48c4923
Revert "feat(core): add 'tags' execution query filter"
cstuncsik 7cfdcc2
feat(editor): add translations and tooltip and counting selected filt…
cstuncsik e0a4afd
fix(editor): fix label layouts
cstuncsik 35e8887
fix(editor): update custom data docs link
cstuncsik cd67e1a
fix(editor): update custom data tooltip position
cstuncsik 67aaa42
fix(editor): update tooltip text
cstuncsik dfb3fcd
fix(editor): fix after merge conflict
cstuncsik 00a9682
Merge remote-tracking branch 'origin/master' into pay-36-store-execut…
cstuncsik 86e37e8
refactor: Ignore metadata if not enabled by license
krynble a08e4f4
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik fd3cf93
Merge remote-tracking branch 'origin/pay-36-store-execution-metadata'…
cstuncsik 102d718
fix(editor): Add paywall states to advanced execution filter
cstuncsik 4f8a324
refactor: Save custom data also for worker mode
krynble 203ebf6
fix: Remove duplicate migration name from list
krynble f500671
fix(editor): Reducing filter complexity and add debounce to text inputs
cstuncsik c932ceb
Merge branch 'pay-149-execution-filter' of github.com:n8n-io/n8n into…
cstuncsik b972bdc
fix(editor): Remove unused import, add comment
cstuncsik 7eb9bdb
fix(editor): simplify event listener
cstuncsik 9c028c4
fix: Prevent error when there are running executions
krynble 11b438c
test(editor): Add advanced execution filter basic unit test
cstuncsik a202b0e
Merge branch 'pay-149-execution-filter' of github.com:n8n-io/n8n into…
cstuncsik a22d5e2
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik 789b199
test(editor): Add advanced execution filter state change unit test
cstuncsik 940d95d
fix: Small lint issue
krynble f1b44b2
feat: Add indices to speed up queries
krynble b671849
feat: add customData limits
valya fc511b9
Merge branch 'pay-149-execution-filter' of github.com:n8n-io/n8n into…
valya a1083b9
refactor: put metadata save in transaction
valya 3d5a4d2
chore: remove unneed comment
valya bc2d4cb
test: add tests for execution metadata
valya acd7a26
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik bb2cab1
Merge branch 'pay-149-execution-filter' of github.com:n8n-io/n8n into…
cstuncsik 41ddbd5
fix(editor): Fixes after merge conflict
cstuncsik 38c39d5
fix(editor): Remove unused import
cstuncsik d058006
wordings and ui fixes
romainminaud 2616fea
fix(editor): type fixes
cstuncsik cf4d37a
feat: add code node autocompletions for customData
valya cabc8be
fix: Prevent transaction issues and ambiguous ID in sql clauses
krynble d240626
Merge branch 'pay-149-execution-filter' of github.com:n8n-io/n8n into…
krynble b75b52d
Merge remote-tracking branch 'origin/master' into pay-149-execution-f…
cstuncsik 0abfcf9
fix(editor): Suppress requesting current executions if metadata is us…
cstuncsik 4037eb9
refactor: Improve performance by correcting database indices
krynble 6db72e9
Merge branch 'pay-149-execution-filter' of github.com:n8n-io/n8n into…
krynble 9b5d643
fix: Lint issue
krynble 19237bb
test: Fix broken test
krynble 41a669c
fix: Broken test
krynble 2eee5fd
test: add call data check for saveExecutionMetadata test
valya 55f02dd
Merge branch 'master' into pay-149-execution-filter
krynble File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
import { Column, Entity, ManyToOne, PrimaryGeneratedColumn, RelationId } from 'typeorm'; | ||
import { ExecutionEntity } from './ExecutionEntity'; | ||
|
||
@Entity() | ||
export class ExecutionMetadata { | ||
@PrimaryGeneratedColumn() | ||
id: number; | ||
|
||
@ManyToOne('ExecutionEntity', 'metadata', { | ||
onDelete: 'CASCADE', | ||
}) | ||
execution: ExecutionEntity; | ||
|
||
@RelationId((executionMetadata: ExecutionMetadata) => executionMetadata.execution) | ||
executionId: number; | ||
|
||
@Column('text') | ||
key: string; | ||
|
||
@Column('text') | ||
value: string; | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
69 changes: 69 additions & 0 deletions
69
packages/cli/src/databases/migrations/mysqldb/1679416281779-CreateExecutionMetadataTable.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
import { MigrationInterface, QueryRunner, Table } from 'typeorm'; | ||
import { getTablePrefix, logMigrationEnd, logMigrationStart } from '@db/utils/migrationHelpers'; | ||
|
||
export class CreateExecutionMetadataTable1679416281779 implements MigrationInterface { | ||
name = 'CreateExecutionMetadataTable1679416281779'; | ||
|
||
public async up(queryRunner: QueryRunner): Promise<void> { | ||
logMigrationStart(this.name); | ||
const tablePrefix = getTablePrefix(); | ||
|
||
await queryRunner.query( | ||
`CREATE TABLE ${tablePrefix}execution_metadata ( | ||
id int(11) auto_increment NOT NULL PRIMARY KEY, | ||
executionId int(11) NOT NULL, | ||
\`key\` TEXT NOT NULL, | ||
value TEXT NOT NULL, | ||
CONSTRAINT \`${tablePrefix}execution_metadata_FK\` FOREIGN KEY (\`executionId\`) REFERENCES \`${tablePrefix}execution_entity\` (\`id\`) ON DELETE CASCADE, | ||
INDEX \`IDX_${tablePrefix}6d44376da6c1058b5e81ed8a154e1fee106046eb\` (\`executionId\` ASC) | ||
) | ||
ENGINE=InnoDB`, | ||
); | ||
|
||
// Remove indices that are no longer needed since the addition of the status column | ||
await queryRunner.query( | ||
`DROP INDEX \`IDX_${tablePrefix}06da892aaf92a48e7d3e400003\` ON \`${tablePrefix}execution_entity\``, | ||
); | ||
await queryRunner.query( | ||
`DROP INDEX \`IDX_${tablePrefix}78d62b89dc1433192b86dce18a\` ON \`${tablePrefix}execution_entity\``, | ||
); | ||
await queryRunner.query( | ||
`DROP INDEX \`IDX_${tablePrefix}1688846335d274033e15c846a4\` ON \`${tablePrefix}execution_entity\``, | ||
); | ||
await queryRunner.query( | ||
`DROP INDEX \`IDX_${tablePrefix}cefb067df2402f6aed0638a6c1\` ON \`${tablePrefix}execution_entity\``, | ||
); | ||
|
||
// Add index to the new status column | ||
await queryRunner.query( | ||
`CREATE INDEX \`IDX_${tablePrefix}8b6f3f9ae234f137d707b98f3bf43584\` ON \`${tablePrefix}execution_entity\` (\`status\`, \`workflowId\`)`, | ||
); | ||
|
||
logMigrationEnd(this.name); | ||
} | ||
|
||
public async down(queryRunner: QueryRunner): Promise<void> { | ||
const tablePrefix = getTablePrefix(); | ||
|
||
await queryRunner.query(`DROP TABLE "${tablePrefix}execution_metadata"`); | ||
await queryRunner.query( | ||
`CREATE INDEX \`IDX_${tablePrefix}06da892aaf92a48e7d3e400003\` ON \`${tablePrefix}execution_entity\` (\`workflowId\`, \`waitTill\`, \`id\`)`, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX \`IDX_${tablePrefix}78d62b89dc1433192b86dce18a\` ON \`${tablePrefix}execution_entity\` (\`workflowId\`, \`finished\`, \`id\`)`, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX \`IDX_${tablePrefix}1688846335d274033e15c846a4\` ON \`${tablePrefix}execution_entity\` (\`finished\`, \`id\`)`, | ||
); | ||
await queryRunner.query( | ||
'CREATE INDEX `IDX_' + | ||
tablePrefix + | ||
'cefb067df2402f6aed0638a6c1` ON `' + | ||
tablePrefix + | ||
'execution_entity` (`stoppedAt`)', | ||
); | ||
await queryRunner.query( | ||
`DROP INDEX \`IDX_${tablePrefix}8b6f3f9ae234f137d707b98f3bf43584\` ON \`${tablePrefix}execution_entity\``, | ||
); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
62 changes: 62 additions & 0 deletions
62
...ges/cli/src/databases/migrations/postgresdb/1679416281778-CreateExecutionMetadataTable.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
import { MigrationInterface, QueryRunner, Table } from 'typeorm'; | ||
import { getTablePrefix, logMigrationEnd, logMigrationStart } from '@db/utils/migrationHelpers'; | ||
|
||
export class CreateExecutionMetadataTable1679416281778 implements MigrationInterface { | ||
name = 'CreateExecutionMetadataTable1679416281778'; | ||
|
||
public async up(queryRunner: QueryRunner): Promise<void> { | ||
logMigrationStart(this.name); | ||
const tablePrefix = getTablePrefix(); | ||
|
||
await queryRunner.query( | ||
`CREATE TABLE ${tablePrefix}execution_metadata ( | ||
"id" serial4 NOT NULL PRIMARY KEY, | ||
"executionId" int4 NOT NULL, | ||
"key" text NOT NULL, | ||
"value" text NOT NULL, | ||
CONSTRAINT ${tablePrefix}execution_metadata_fk FOREIGN KEY ("executionId") REFERENCES ${tablePrefix}execution_entity(id) ON DELETE CASCADE | ||
)`, | ||
); | ||
|
||
await queryRunner.query( | ||
`CREATE INDEX "IDX_${tablePrefix}6d44376da6c1058b5e81ed8a154e1fee106046eb" ON "${tablePrefix}execution_metadata" ("executionId");`, | ||
); | ||
|
||
// Remove indices that are no longer needed since the addition of the status column | ||
await queryRunner.query(`DROP INDEX IF EXISTS "IDX_${tablePrefix}33228da131bb1112247cf52a42"`); | ||
await queryRunner.query(`DROP INDEX IF EXISTS "IDX_${tablePrefix}72ffaaab9f04c2c1f1ea86e662"`); | ||
await queryRunner.query(`DROP INDEX IF EXISTS "IDX_${tablePrefix}58154df94c686818c99fb754ce"`); | ||
await queryRunner.query(`DROP INDEX IF EXISTS "IDX_${tablePrefix}4f474ac92be81610439aaad61e"`); | ||
|
||
// Create new index for status | ||
await queryRunner.query( | ||
`CREATE INDEX "IDX_${tablePrefix}8b6f3f9ae234f137d707b98f3bf43584" ON "${tablePrefix}execution_entity" ("status", "workflowId");`, | ||
); | ||
|
||
logMigrationEnd(this.name); | ||
} | ||
|
||
public async down(queryRunner: QueryRunner): Promise<void> { | ||
const tablePrefix = getTablePrefix(); | ||
|
||
// Re-add removed indices | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS "IDX_${tablePrefix}33228da131bb1112247cf52a42" ON ${tablePrefix}execution_entity ("stoppedAt") `, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS "IDX_${tablePrefix}72ffaaab9f04c2c1f1ea86e662" ON ${tablePrefix}execution_entity ("finished", "id") `, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS "IDX_${tablePrefix}58154df94c686818c99fb754ce" ON ${tablePrefix}execution_entity ("workflowId", "waitTill", "id") `, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS "IDX_${tablePrefix}4f474ac92be81610439aaad61e" ON ${tablePrefix}execution_entity ("workflowId", "finished", "id") `, | ||
); | ||
|
||
await queryRunner.query( | ||
`DROP INDEX IF EXISTS "IDX_${tablePrefix}8b6f3f9ae234f137d707b98f3bf43584"`, | ||
); | ||
|
||
await queryRunner.query(`DROP TABLE "${tablePrefix}execution_metadata"`); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
64 changes: 64 additions & 0 deletions
64
packages/cli/src/databases/migrations/sqlite/1679416281777-CreateExecutionMetadataTable.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
import { MigrationInterface, QueryRunner, Table } from 'typeorm'; | ||
import { getTablePrefix, logMigrationEnd, logMigrationStart } from '@db/utils/migrationHelpers'; | ||
|
||
export class CreateExecutionMetadataTable1679416281777 implements MigrationInterface { | ||
name = 'CreateExecutionMetadataTable1679416281777'; | ||
|
||
public async up(queryRunner: QueryRunner): Promise<void> { | ||
logMigrationStart(this.name); | ||
const tablePrefix = getTablePrefix(); | ||
|
||
await queryRunner.query( | ||
`CREATE TABLE "${tablePrefix}execution_metadata" ( | ||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, | ||
executionId INTEGER NOT NULL, | ||
"key" TEXT NOT NULL, | ||
value TEXT NOT NULL, | ||
CONSTRAINT ${tablePrefix}execution_metadata_entity_FK FOREIGN KEY (executionId) REFERENCES ${tablePrefix}execution_entity(id) ON DELETE CASCADE | ||
)`, | ||
); | ||
|
||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS "IDX_${tablePrefix}6d44376da6c1058b5e81ed8a154e1fee106046eb" ON "${tablePrefix}execution_metadata" ("executionId");`, | ||
); | ||
|
||
// Re add some lost indices from migration DeleteExecutionsWithWorkflows.ts | ||
// that were part of AddExecutionEntityIndexes.ts | ||
// not all were needed since we added the `status` column to execution_entity | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS 'IDX_${tablePrefix}b94b45ce2c73ce46c54f20b5f9' ON '${tablePrefix}execution_entity' ('waitTill', 'id') `, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS 'IDX_${tablePrefix}81fc04c8a17de15835713505e4' ON '${tablePrefix}execution_entity' ('workflowId', 'id') `, | ||
); | ||
|
||
// Also add index to the new status column | ||
await queryRunner.query( | ||
`CREATE INDEX IF NOT EXISTS 'IDX_${tablePrefix}8b6f3f9ae234f137d707b98f3bf43584' ON '${tablePrefix}execution_entity' ('status', 'workflowId') `, | ||
); | ||
|
||
// Remove no longer needed index to waitTill since it's already covered by the index b94b45ce2c73ce46c54f20b5f9 above | ||
await queryRunner.query(`DROP INDEX IF EXISTS 'IDX_${tablePrefix}ca4a71b47f28ac6ea88293a8e2'`); | ||
// Remove index for stoppedAt since it's not used anymore | ||
await queryRunner.query(`DROP INDEX IF EXISTS 'IDX_${tablePrefix}cefb067df2402f6aed0638a6c1'`); | ||
|
||
logMigrationEnd(this.name); | ||
} | ||
|
||
public async down(queryRunner: QueryRunner): Promise<void> { | ||
const tablePrefix = getTablePrefix(); | ||
|
||
await queryRunner.query(`DROP TABLE "${tablePrefix}execution_metadata"`); | ||
await queryRunner.query(`DROP INDEX IF EXISTS 'IDX_${tablePrefix}b94b45ce2c73ce46c54f20b5f9'`); | ||
await queryRunner.query(`DROP INDEX IF EXISTS 'IDX_${tablePrefix}81fc04c8a17de15835713505e4'`); | ||
await queryRunner.query( | ||
`DROP INDEX IF EXISTS 'IDX_${tablePrefix}8b6f3f9ae234f137d707b98f3bf43584'`, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX "IDX_${tablePrefix}ca4a71b47f28ac6ea88293a8e2" ON "${tablePrefix}execution_entity" ("waitTill")`, | ||
); | ||
await queryRunner.query( | ||
`CREATE INDEX "IDX_${tablePrefix}cefb067df2402f6aed0638a6c1" ON "${tablePrefix}execution_entity" ("stoppedAt")`, | ||
); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should either use stricter typing on this column, and add an index for faster searching, or get move this data to a simple json column on the executions table.
in this current form, the additional table/entity feels unnecessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The goal of this field is not to be a json, it's intentionally simply basic text. About the performance, I'll check to see if something can be improved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let me rephrase: Does this really need to be a separate table?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think so - you can save multiple key-value pairs for each execution. The goal is to allow users to save and read data during execution and afterwards, search for this data in the executions panel.
When querying, you can query by some of the keys (initial implementation is a single key-value search but this can be changed in the future)