Use get_parfor_writes to detect illegal array access that prevents fusion.

[DrTodd13]

Resolves #7578

In parfor fusion, there is a check that something written in one parfor not be used in the second parfor. There was a case in issue #7578 that an array was written in the first parfor and analysis of that parfor said that the array was "used" but not written in that parfor. Therefore, the second parfor which also used the same array didn't see a conflict and did the fusion. If you just make the array written in the first parfor loop body as a "def" then the array isn't passed to the parfor gufunc and ends up undefined. So, what I am trying here is to make the array both def and use so that the parfors don't fuse but the array is passed as a parameter to the gufunc.

This is in-progress at the moment as I expect this to break legitimate instances of fusion because the real problem with this example is that indices outside of the parfor index are used in the second parfor which is the only thing that should prevent fusion in this case. I want to see what the tests say to see which tests fail due to this and if none do then to create one that should be allowed.

[ehsantn]

Can something like get_stmt_writes be used instead? Use/def and liveness are used in many places and messing with them is potentially very dangerous. Setitem doesn't really have def semantics.

[DrTodd13]

Can something like get_stmt_writes be used instead? Use/def and liveness are used in many places and messing with them is potentially very dangerous. Setitem doesn't really have def semantics.

I did something similar now. Please have a look.

[ehsantn]

Didn't go through the details but seems like this is on the right track. Thanks.

[sklam]

Can it default to be unsafe instead for a more conservative behavior?

It seems it just need all overlap variables to be of ArrayCompatible and in p2arraynotindexed

[DrTodd13]

I can do the following:

unsafe_var = [not isinstance(typemap[x], types.ArrayCompatible) or x in p2arraynotindexed for x in overlap]
if any(unsafe_var):

That is simpler looking but this does have potential to be slower than what I wrote that gives up as soon as it finds the first problem. The new line above would potentially compute some useless stuff but the amount of work is trivial so I've checked in this change. See if you like it.

[lgarrison]

Would a generator comprehension solve this problem? If you change the brackets to parentheses, then the body will be lazily evaluated by any(): unsafe_var = (not isinstance(typemap[x], types.ArrayCompatible) or x in p2arraynotindexed for x in overlap)

[DrTodd13]

Thanks. Done.

[esc]

towncrier support was fixed with #9069

I think you'll need to merge in main to get towncrier to work again with the patches on main.

[esc]

Closing to open again in an attempt to re-trigger towncrier. It should "just work" on this PR.

[esc]

@DrTodd13 there is a bug in the filename for towncrier: 7582.big_fix.rst should be 7582.bug_fix.rst, can you fix?

[sklam]

LGTM

[esc]

@DrTodd13 thank you for the patch. @sklam thank you for the review. I have marked this as RTM now.

[esc]

@DrTodd13 after merging #9037 this now conflicts with main.

[DrTodd13]

@DrTodd13 after merging #9037 this now conflicts with main.

Resolved.

[sklam]

(CI failed due to conda timeout. i've restarted failing builds)