Serialize wrapped global functions by value

[stiepan]

Signed-off-by: Kamil Tokarski ktokarski@nvidia.com

Category:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)

Description:

Extends the idea of serialization by value from local functions to global ones that were manually overwritten.

Fixes an issue with a new ipython notebook env, where open (which should be just io.open) is replaced with a custom wrapper (ipython/ipython@93992a7). This makes serailization by value of any callback that use open call fail with the pickle check that the open is io.open.

Additional information:

Plain python pickler serializes functions by name, just remembering "a path" to the function. However, if the function is not statically available (because it is dynamically created closure or it is global function but in a notebook), this approach fails.
DALI uses sligtly cutomized pickler to handle serialization of local functions and global functions in a notebook mode. The pickler serializes local functions by value (marshalling their bytecode) and serializing recursivelly the dependencies. The same procudure can be applied to a global function if it is marked with dali.pickling.pikcle_by_value.

Now, ipython recently added a wrapper around global open, which makes it non-serializable with pickle. As a result, serialization by value fails on callbacks that use open, because it needs to be serialzied as a dependency. The change in this PR just tries to catch any such cases in advance and apply "by value" mode to them too.

Affected modules and functionalities:

Key points relevant for the review:

Tests:

• Existing tests apply
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: N/A

[stiepan]

!build

[dali-automaton]

CI MESSAGE: [6343003]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [6343003]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [6343003]: BUILD PASSED