feature: hash source with AST

[majidaldo]

It's a bit annoying that the cache gets invalidated by trivial non-functional code changes. Has anyone thought of hashing the ast to identify a function?

[ogrisel]

That's an interesting suggestion. I wonder if it could have a performance impact though.

@pierreglaser wants to investigate with a bad interaction that prevent caching to work as expected when:

• the function is dynamically defined in __main__
• the function is cached with memory
• the function is called by Parallel with the loky backend

all 3 at the same time.

Using the AST instead of the source might be a fix for the above problem.

[majidaldo]

• the function is cached with memory

you mean the Memory object?

[pierreglaser]

I agree that it's a good suggestion. That could also simplify the codebase.
@ogrisel if you think that could be an option, I can investigate this.

[ogrisel]

@pierreglaser feel free to investigate, in particular in the context of the bug you described to me IRL and that I tried to summarize in #1074 (comment). Please open a dedicated issue for the bug itself if you can come up with a simple reproducer.

[majidaldo]

I went ahead and gave it a shot. I only did a quick test and it seems to work.

It's crude though and just demonstrates the concept. The ast hashing was straightforward but I couldn't follow all the (existing) code. Function source refs seems to be entangled with its hashing.

[majidaldo]

bump @pierreglaser