Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Temporal rewrite #146

Merged
merged 84 commits into from
May 29, 2024
Merged

feat: Temporal rewrite #146

merged 84 commits into from
May 29, 2024

Conversation

daryllimyt
Copy link
Contributor

@daryllimyt daryllimyt commented May 29, 2024 •
edited
Loading

Changes

  • All actions and integrations are now UDFs (user defined function)
  • UDF args and response schemas get parsed into pydantic models for validation
  • Workflow engine rewritten in Temporal
  • We can now operate the SOAR headless, using a yaml DSL interface
  • Added CLI for convenience

WIP

  • Frontend nocode -> code still WIP
  • UDF pydantic models will be used to generate UI form elements

Testing

  • Rewrote tests for major components
  • Unit tests all passing

topher-lo and others added 30 commits May 29, 2024 02:03
@topher-lo @daryllimyt
ci: Create GH action to push images to ECR
956ffb8
@topher-lo @daryllimyt
ci: Consolidate pub images workflows
7bad4b1
@topher-lo @daryllimyt
Delete LICENSE
3c2c49f
@topher-lo @daryllimyt
Update LICENSE
499033d
@topher-lo @daryllimyt
Change license to AGPL-3.0 (TracecatHQ#1)
19df567
@topher-lo @daryllimyt
docs: Change homepage url
65b5826
@topher-lo @daryllimyt
docs: Update classifiers
7f10671
@topher-lo @daryllimyt
docs: Fix broken link to new install video
7e2f0b7
@daryllimyt
feat(engine): Basic workflow DSL with pydantic
7b9eb89
@daryllimyt
build(engine): Udpate dependencies
ae6a50e
@daryllimyt
refactor(engine): Define dsl activities outside of worker
0496a59
@daryllimyt
feat(engine): Templated future WIP [1/N]
b2795e4
@daryllimyt
refactor(engine): Remove pydantic converter boilerplate and module
4df803a
@daryllimyt
refactor: Remove experimental dsl readme
cec8078
@daryllimyt
feat(engine): Add experimental templates module
2f5abf4
@daryllimyt
feat(engine): Add templating to workflow execution
ff2df34
@daryllimyt
feat(engine): Improve UDF registry [1/N]
4cf1055
@daryllimyt
feat(engine): Implement auth sandbox for udf registry
545c5cd
@daryllimyt
feat(engine): Move registry into actions and add core
b3dbbe8
@daryllimyt
refactor(engine): Condense dsl module
cb1cd9b
@daryllimyt
refactor: Rename exampel workflow dir and add workflow4
ea3c1a8
@daryllimyt
feat(engine): Update udf slug and parse description into jsonschema
03e7b3b
@daryllimyt
refactor(engine): Create actions module substructure with core and in…
c89c98c 
…tegrations
@daryllimyt
refactor(engine): Remove concurrency module
8f90308
@daryllimyt
feat(engine): Update auth module
d353358
@daryllimyt
feat+refactor(engine): Move a lot of stuff around to get registry wor…
7fa0f55 
…king, WIP [1/N]
@daryllimyt
feat(engine): Add RF graph parsing into DSL
c6f88bf
@daryllimyt
refactor(engine): Rename runner router to triggers
af92739
@daryllimyt
feat(engine): Use json in postgres
8cbca67
@daryllimyt
feat(ui): Pass RF objects as json objects
767cf2e
@daryllimyt
build: Pin all dependencies, remove colorlog
b22dfa0
@daryllimyt
build: Remove runner optional dependency
e39ccf2
@daryllimyt
refactor(engine): Move triggers into main API app
207e895
@daryllimyt
feat(ui): Generalize error handling
28d1ec7
@daryllimyt
refactor(ui): Remove event feed provider
d92dddb
@daryllimyt
feat(engine): Make http payload more flexible
ed49438
@daryllimyt
refactor: Silence logging on startup when running with auth
9269587
@daryllimyt
feat: Add CLI tool
27cfede
@daryllimyt
feat(engine): Improve DSLInput validation
0ea45c0
@daryllimyt
refactor: Print udf key inline
9799eb1
@daryllimyt
feat(engine): Add aws guardduty + slack playbook yaml
e38dcfb
@daryllimyt
refactor: Migrate guardduty integration into experimental registry
3144eff
@daryllimyt
build: Pin dotenv
3ac4c98
@daryllimyt
refactor: Remove croniter
c463b25
@daryllimyt
fix: Fix guardduty etl imports
7984849
@daryllimyt
build: Add colorlog 6.8.2
b2a9526
@daryllimyt
test: Fix tests
5ec0790
@daryllimyt
refactor(engine): Rename db.models to db.schemas
2219abb
@daryllimyt
refactor: Delete deprecated actions and integrations type
05b6db7
@daryllimyt
chore: Delete unused workflow yamls
2ac2d5e
@daryllimyt
feat(engine): Remove experimental status for actions, dsl, templates,…
7e39d83 
… registry
@daryllimyt
refactor: Remove custom logging, deprioritize
931613f
@daryllimyt daryllimyt added the enhancement New feature or request label May 29, 2024
@daryllimyt daryllimyt requested a review from topher-lo May 29, 2024 03:44
@daryllimyt daryllimyt self-assigned this May 29, 2024
@vercel Vercel
Copy link

vercel bot commented May 29, 2024

@daryllimyt is attempting to deploy a commit to the Tracecat Team on Vercel.

A member of the Team first needs to authorize it.

@daryllimyt daryllimyt merged commit 2816954 into TracecatHQ:main May 29, 2024
0 of 3 checks passed
@daryllimyt daryllimyt deleted the feat/pull-internal branch May 29, 2024 03:53
topher-lo
topher-lo reviewed May 29, 2024
View reviewed changes
Copy link
Contributor

@topher-lo topher-lo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as discussed on Slack

@topher-lo topher-lo mentioned this pull request Jun 16, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@topher-lo topher-lo topher-lo left review comments

Assignees

@daryllimyt daryllimyt

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@daryllimyt @topher-lo