Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Extract a generalized V2 rule to inject `__init__.py` files #7722
Python requires packages to have an
#7696 does this for Pytest, but inlines the logic, even though it will likely be helpful for other Python rules as well.
Further, because this logic was originally written before being able to from
Will close #7715.
We take a
We return a
It will now be easier for other Python rules to work with Python packages.
The unnecessary memory usage is now fixed. The V2 Pytest runner now has a space complexity of
changed the title
Refactor to have a new generalized V2 rule to inject `__init__.py` files
May 16, 2019
cosmicexplorer left a comment
This is great, and I'm quite happy with this code! I would however really like to clamp down on uses of absolute paths to binaries, especially in remoteable process executions. It's used in test code but shouldn't (imho) really spread to production code. I propose one alternative (create an intermediate rule to encapsulate the workaround) which shouldn't be super hard to implement, and may also end up being useful for other rules as well (which is another reason why the v2 engine is so great!).
I strongly prefer not adding this intermediate type, given that Daniel and I next week plan to add a much more robust builtin Rust rule to do the same thing. This was already difficult to get everything working, and I do not want to spend much time on something that is only temporary.
I'd like to please keep following the "progress is better than perfection" approach we talked about it in #7696 (review). Because we let ourselves land that in an imperfect state, I was unblocked from several other PRs, including this memory over-usage investigation. Already, we've implemented 2 of the 4 followup PRs we agreed to. The 3rd followup is what we're discussing, and the 4th is blocked by #7710.
Tangibly, this PR is necessary for the memory over-usage work, because it removes a use of
cosmicexplorer left a comment
I think that there's real value to attempting to discuss and understand the novel v2 rulesets we're making, and especially to look, from the start, to find opportunities to make composable rulesets that others can hook into and to avoid repeated logic. For example, yesterday we found that we had missed such an existing composable ruleset in the native backend in #7735, and may have been about to write that logic all over again. I would really like to not have absolute paths to files sprouting without TODOs attached to them, and I think we should do that here and link to issues or a project if necessary so it can be clear to others what the timeline of these changes are and what upcoming work is going to invalidate the temporary workaround of using an absolute path (
Separately, if there is a PR which is blocking others and therefore is under pressure to review quickly, it would be extremely useful to me if that could be noted somehow in the PR title, label, or description (a label named
I agree! And I appreciate that you raised the concern originally in #7696. When writing this the first time, I did not know that it's bad to have to rely on touch.
My main point is that we can incrementally improve these rules via followup PRs (so long as we actually address the TODOs, which so far we have been doing). Which leads to the next point about not communicating this well enough:
Acknowledged and agreed. It would have been a lot less confusing all around if I communicated better the incremental plan to get this landed as the first improvement, and then to follow it up with the next step once Daniel lands #7739. I'm sorry for the confusion this caused!