Skip to content

feat(subs,gate): substantial integration#844

Merged
michael-0acf4 merged 6 commits intomainfrom
substantial-integration
Sep 23, 2024
Merged

feat(subs,gate): substantial integration#844
michael-0acf4 merged 6 commits intomainfrom
substantial-integration

Conversation

@michael-0acf4
Copy link
Copy Markdown
Contributor

@michael-0acf4 michael-0acf4 commented Sep 9, 2024

Supersedes #818

  • Backend constructs
  • Workflow logic
    • Exceptions/Interrupts + configurable SUBSTANTIAL_RELAUNCH_MS
    • Runs (start, stop, healthcheck queries)
  • Workflows in typescript for now
  • Events (receive, handle)
  • Std interrupts (ctx.sleep(..), ctx.ensure(..))

Also removed Substantial syntactic sugar from the Deno/PythonRuntime SDKs as it might introduce confusion on the inner workings of the Typegate.

Migration notes

None

  • The change comes with new or modified tests
  • Hard-to-understand functions have explanatory comments
  • End-user documentation is updated to reflect the change

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 10, 2024

Codecov Report

Attention: Patch coverage is 82.66254% with 112 lines in your changes missing coverage. Please review.

Project coverage is 77.25%. Comparing base (15c1afa) to head (196c769).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/typegate/src/runtimes/substantial.ts 84.09% 52 Missing ⚠️
...rc/runtimes/substantial/workflow_worker_manager.ts 85.94% 26 Missing ⚠️
src/typegate/src/runtimes/substantial/types.ts 53.84% 12 Missing ⚠️
src/typegraph/deno/src/runtimes/deno.ts 33.33% 10 Missing ⚠️
src/typegate/engine/bindings.ts 78.37% 8 Missing ⚠️
src/typegraph/deno/src/runtimes/python.ts 33.33% 2 Missing ⚠️
src/typegate/src/config.ts 85.71% 1 Missing ⚠️
src/typegate/src/utils.ts 94.11% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #844      +/-   ##
==========================================
+ Coverage   76.26%   77.25%   +0.98%     
==========================================
  Files         145      147       +2     
  Lines       16796    17226     +430     
  Branches     1649     1701      +52     
==========================================
+ Hits        12810    13308     +498     
+ Misses       3967     3899      -68     
  Partials       19       19              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@michael-0acf4 michael-0acf4 changed the title feat(sub,gate): substantial feat(sub,gate): substantial integration Sep 10, 2024
@michael-0acf4 michael-0acf4 changed the title feat(sub,gate): substantial integration feat(subs,gate): substantial integration Sep 11, 2024
@michael-0acf4 michael-0acf4 force-pushed the substantial-integration branch from 7002b36 to a1c48a5 Compare September 16, 2024 19:30
Yohe-Am
Yohe-Am previously approved these changes Sep 17, 2024
Copy link
Copy Markdown
Contributor

@Yohe-Am Yohe-Am left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. But as the complexity increases, we'll need to invest in even more massive test infra. A sound model can still have faulty implementations.

@michael-0acf4 michael-0acf4 force-pushed the substantial-integration branch 4 times, most recently from ee02244 to 6ea439f Compare September 20, 2024 13:57
@michael-0acf4 michael-0acf4 force-pushed the substantial-integration branch 3 times, most recently from 6e86f56 to 196c769 Compare September 23, 2024 09:39
@michael-0acf4 michael-0acf4 merged commit c56803c into main Sep 23, 2024
@michael-0acf4 michael-0acf4 deleted the substantial-integration branch September 23, 2024 13:18
michael-0acf4 added a commit that referenced this pull request Sep 24, 2024
Continuation of #844 

Note: 
There is currently no need for additional flags when running typegate,
existing runs should be picked automatically as long as the typegraph
owning the workflow is deployed. SubstantialRuntime will launch and
start the agent, the later will attempt to 'pick' jobs on the backend
and spawn workers if it can acquire it.
Workflows are identified relative to their names for now, the content
hash might be the way to go going forward, downside being that it will
change even on smaller modifications (e.g. spaces, missing semicolons,
..).

#### Migration notes

None

- [x] The change comes with new or modified tests
- [ ] Hard-to-understand functions have explanatory comments
- [ ] End-user documentation is updated to reflect the change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants