Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to (faster) graph based Span parsing #130

Merged
merged 20 commits into from
Jul 18, 2022

Conversation

matiasdahl
Copy link
Member

@matiasdahl matiasdahl commented Jul 15, 2022

Run times mnist demo pipeline "run-all-tests-and-pipeline"-job

Significant speedup for "Render Gantt and dependency graphs" (16mins -> 42 sec, !!!)

Before

image

After

image

Run times for Github action "Run unit tests" step in CI-pipeline for pynb-dag-runner repo

Large variety and no clear speedup.

Last 5 runs in development branch

  • 5:14 #675
  • 4:24 #671
  • 4:28 #669
  • 4:26 #667
  • 4:51 #665

5 runs after commit 29900d5 in this PR

  • 5:06 #689
  • 5:32 #690
  • 4:36 #691
  • 5:02 #692
  • 4:26 #693

@matiasdahl matiasdahl linked an issue Jul 15, 2022 that may be closed by this pull request
@matiasdahl matiasdahl changed the title Introduce graph structure for Span:s Move to (faster) graph based Span parsing Jul 18, 2022
@matiasdahl matiasdahl merged commit 22731c2 into development Jul 18, 2022
@matiasdahl matiasdahl deleted the tree-based-spans-data-structure branch July 18, 2022 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant