Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python pickle fails for Regions and Network #732

Open
fcr opened this issue Oct 28, 2019 · 6 comments
Open

Python pickle fails for Regions and Network #732

fcr opened this issue Oct 28, 2019 · 6 comments
Labels

Comments

@fcr
Copy link

fcr commented Oct 28, 2019

I would like to multiprocess several parallel networks, in
particular the location framework, but this fails on pickle errors for networks and regions.
I added pickle tests to the Python bindings unit tests for region and network to check if the problem is me, or is the bindings.
It seems it is the bindings unless I am missing something.
I have attached the modified unit tests for your convenience.

test.zip

@dkeeney
Copy link

dkeeney commented Oct 28, 2019

Good catch.

Serialization of the network is performed by calling network.saveToFile(filename) or save(stream).
However, from the Python side a pickle of a network should result in save( ) being called. The stream would need to be captured and copied into the pickle stream. All of this would need to be performed by the bindings. PyBindRegion has code to perform the pickle in the other direction (C++ serializing the python region), but not for a pickle starting on the python side.

For the region, we should have this as well... After a region has been deserialized it would need to be reconnected to the network before it could be used.

@fcr would you like to take on this project?

@fcr
Copy link
Author

fcr commented Oct 28, 2019

@dkeeney if I was comfortable enough with C++ I would have been happy to.
But it is way too far out of my comfort zone so I must decline.

@dkeeney
Copy link

dkeeney commented Oct 28, 2019

ok, let me see what I can do.

@fcr
Copy link
Author

fcr commented Oct 28, 2019

Thanks. It would be much appreciated.

@fcr
Copy link
Author

fcr commented Oct 28, 2019

BTW this is part of my efforts to get around the performance bottleneck of the ColumnPooler.

@dkeeney
Copy link

dkeeney commented Nov 6, 2019

I am able to reproduce your error:

======================================================================
ERROR: testAL246aCorticalColumnPickle (L46aNetwork_test.TestSimpleSPTMNetwork)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "J:\Projects\AI\htmA\py\tests\advanced\regions\L46aNetwork_test.py", line 117, in testAL246aCorticalColumnPickle
    network2 = pickle.loads(pickledColumn)
RuntimeError: CHECK FAILED: "srcOutput.get() != o" addLink -- link from region L4 output winnerCells to region L6a input anchorGrowthCandidates already exists

I don't know the cause yet but I will setup another PR to address this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants