Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add exclude modules argument to map #140

Merged
merged 4 commits into from Jul 7, 2017
Merged

add exclude modules argument to map #140

merged 4 commits into from Jul 7, 2017

Conversation

@Vaishaal
Copy link
Collaborator

@Vaishaal Vaishaal commented May 17, 2017

Exclude modules in map based on prefix

Fix to #136

@ooq
Copy link
Collaborator

@ooq ooq commented May 30, 2017

Let's maybe have an example to document this?

@Vaishaal
Copy link
Collaborator Author

@Vaishaal Vaishaal commented May 31, 2017

Here is an example

library.py:

import big_module # lets say this includes some stupid dependency tree that is like > 100 mb in size
def foo(x):
      return big_module.foo(x)

def bar(x):
       return x

driver.py:

import library 
pwex = pywren.default_executor()
pwex.map(library.bar, [0,1,2,3]) # will fail
pwex.map(library.bar, [0,1,2,3], exclude_modules=["big_module"]) # will work
@Vaishaal
Copy link
Collaborator Author

@Vaishaal Vaishaal commented Jun 1, 2017

I've also added a test. @ericmjonas could you see if this is sufficient?

fut = self.wrenexec.map(foo, [x], exclude_modules=["extmodule"])
try:
fut[0].result()
self.fail("shouldn't happen")

This comment has been minimized.

@ooq

ooq Jun 2, 2017
Collaborator

If exclusion does not work, there might still be some exception because extmodule does not really exist right?

This comment has been minimized.

@Vaishaal

Vaishaal Jun 2, 2017
Author Collaborator

What do you mean? In this case extmodule exists right?

This comment has been minimized.

@ooq

ooq Jun 2, 2017
Collaborator

Did you import it?

This comment has been minimized.

@Vaishaal

Vaishaal Jun 2, 2017
Author Collaborator

at the top. line 6.

This comment has been minimized.

@ooq

ooq Jun 2, 2017
Collaborator

Ah okay, I missed it.

Vaishaal added 2 commits Jul 5, 2017
…al/exclude_modules
@shivaram
Copy link
Collaborator

@shivaram shivaram commented Jul 7, 2017

I think this looks fine to me. @ooq do you want to take one final look ?

@ooq
Copy link
Collaborator

@ooq ooq commented Jul 7, 2017

Looks good. Will merge.

@ooq ooq merged commit 9363fe8 into master Jul 7, 2017
1 check passed
1 check passed
continuous-integration/travis-ci/push The Travis CI build passed
Details
@Vaishaal Vaishaal deleted the vaishaal/exclude_modules branch Nov 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.