New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda toobig seatbelts #172

Merged
merged 7 commits into from Aug 26, 2017

Conversation

Projects
None yet
3 participants
@ericmjonas
Collaborator

ericmjonas commented Aug 25, 2017

This is a series of fixes to handle:

  1. if the runtime is too big, communicate to the user and execute proper cleanup on the worker so the worker isn't forever broken. (issue #105)

  2. If downloading the func or the data is too large for the remaining space, error as well.

@ericmjonas

This comment has been minimized.

Show comment
Hide comment
@ericmjonas

ericmjonas Aug 25, 2017

Collaborator

This brings up the question: Should we download the runtime before the user data/func pickles?

Collaborator

ericmjonas commented Aug 25, 2017

This brings up the question: Should we download the runtime before the user data/func pickles?

Show outdated Hide outdated pywren/wrenhandler.py
Show outdated Hide outdated pywren/wrenhandler.py
func_key_size = get_key_size(s3_client, s3_bucket, func_key)
free_disk_bytes = free_disk_space("/tmp")

This comment has been minimized.

@shivaram

shivaram Aug 25, 2017

Collaborator

is /tmp hardcoded elsewhere as well or is there some variable we can use here cc @apengwin

@shivaram

shivaram Aug 25, 2017

Collaborator

is /tmp hardcoded elsewhere as well or is there some variable we can use here cc @apengwin

This comment has been minimized.

@apengwin

apengwin Aug 25, 2017

Contributor

/tmp is hardcoded in current wrenhandler. The portability PR abstracts this out to a generic TEMP variable that's set at the beginning, depending on what OS is running

@apengwin

apengwin Aug 25, 2017

Contributor

/tmp is hardcoded in current wrenhandler. The portability PR abstracts this out to a generic TEMP variable that's set at the beginning, depending on what OS is running

This comment has been minimized.

@ericmjonas

ericmjonas Aug 25, 2017

Collaborator

It is actually hardcoded in the runtimes as well. Unfortunately there are some absolute path dependencies that continue to be a problem in the way conda is packaged.

@ericmjonas

ericmjonas Aug 25, 2017

Collaborator

It is actually hardcoded in the runtimes as well. Unfortunately there are some absolute path dependencies that continue to be a problem in the way conda is packaged.

ericmjonas added some commits Aug 26, 2017

@shivaram

Thanks. One last question: is the catch all except at the end to handle python3 or is to handle other unforeseen exceptions ?

@ericmjonas

This comment has been minimized.

Show comment
Hide comment
@ericmjonas

ericmjonas Aug 26, 2017

Collaborator

So remember right now we only support running under the python2 runtime for the handler; it's for all the exceptions that I wouldn't otherwise catch, to make sure we have a way to clean up the extracted runtime.

Collaborator

ericmjonas commented Aug 26, 2017

So remember right now we only support running under the python2 runtime for the handler; it's for all the exceptions that I wouldn't otherwise catch, to make sure we have a way to clean up the extracted runtime.

@shivaram

This comment has been minimized.

Show comment
Hide comment
@shivaram

shivaram Aug 26, 2017

Collaborator

Ah got it. I forgot the handler was python2 specific. Sounds good.

Collaborator

shivaram commented Aug 26, 2017

Ah got it. I forgot the handler was python2 specific. Sounds good.

@ericmjonas ericmjonas merged commit d528f10 into master Aug 26, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@ericmjonas ericmjonas referenced this pull request Aug 26, 2017

Closed

Too big of runtime #105

@ericmjonas ericmjonas deleted the lambda-toobig-seatbelts branch Sep 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment