Implement basic support for pure-Python tools #1091
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements support for pure-Python tools, a feature we talked about a while back when discussing how to more cleanly port other flows into SC. With both the ORFS and Caravel flows, we've noticed cases where we need to do some intermediate processing on files being passed around the flow. The existing options we explored each have flaws:
pre_process()
orpost_process()
in a tool driverrun()
calls that each use a partial steplist, with Python logic in betweenNatively supporting pure Python steps seems like a cleaner solution to this problem. As a basic approach, I changed SC to treat any tool driver that defines a
run(chip)
function as a pure-Python tool. The SC runtime will then call this function instead of an executable.The one problem with this implementation is that it introduces a third code path that doesn't include support for some nice features, like output redirection, memory tracking, and task timeouts. If we run into a situation where we want these features, I think we could explore running these tools in a separate process that can then be monitored by the runtime in order to provide this functionality, but this was the simplest first approach.
As a proof-of-concept, I updated the Caravel example to use these features. To validate the changes, I ran the flow before and after and made sure the input DEF to KLayout and that the final DEF passed into KLayout are the same. If you'd like more certainty that the changes don't break anything, I can try re-running the MPW prechecks using the new flow.