write pipelined tasty in parallel. #20153
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I have yet to benchmark any theoretical performance gain - but I would assume it to be fairly trivial to predict on main branch by measuring how long the present |
He-Pin
reviewed
Apr 10, 2024
sjrd
requested changes
Apr 11, 2024
bishabosha
force-pushed
the
topic/zinc-pipelining-async
branch
from
acbb1c6
to
3279a4c
Compare
|
There should probably be a cancellation signal also |
bishabosha
force-pushed
the
topic/zinc-pipelining-async
branch
from
3279a4c
to
97422d9
Compare
bishabosha
force-pushed
the
topic/zinc-pipelining-async
branch
from
97422d9
to
8f5e659
Compare
Now in backend we block on the composition of the Zinc API callbacks and tasty being written. We also now conditionally signal that we are "done" writing tasty, based on if any units were suspended. This works in line with the Zinc default, which will ignore the early output anyway under the presence of macros (user can override this). Also move async tasty holder to the Run, as it is now context dependent on suspending. TODO: give the user an option to optimise performance by preventing definition of such problematic macros (which would also avoid suspensions).
bishabosha
force-pushed
the
topic/zinc-pipelining-async
branch
from
8f5e659
to
a25d252
Compare
|
@sjrd all comments are addressed |
sjrd
requested changes
Apr 16, 2024
sjrd
approved these changes
Apr 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
sets up the executor in pickler to write the files as the final task, meaning we no longer read from
unit.pickled, but directly send the tasty bytes to theserializedholder, which queues up the internal name string of the class, and its bytes. When writing TASTy asynchronously, collect any logs into a buffer.When the task for writing is finished, complete a promise with info describing if the writing was successful, and any pending log messages.
when the promise completes, and no errors occur in writing tasty, then asynchronously call the Zinc callbacks which trigger downstream pipelined compilation.
await on the promise in GenBCode, at which point report any buffered errors
fixes #20101