Lambda toobig seatbelts - #172
Conversation
|
This brings up the question: Should we download the runtime before the user data/func pickles? |
|
|
||
| PROCESS_STDOUT_SLEEP_SECS = 2 | ||
|
|
||
| TMP_MIN_FREE_SPACE_BYTES = 10000000 |
There was a problem hiding this comment.
nit - can we add a comment above this "Leave at least 10MB free space" or something like that
| condatar.extractall(runtime_etag_dir) | ||
| try: | ||
| condatar.extractall(runtime_etag_dir) | ||
| except: |
There was a problem hiding this comment.
Is there a particular exception that indicates the tarball was too big ? I guess there could be other errors like corrupt runtimes etc.
There was a problem hiding this comment.
Good question, I will dig further!
|
|
||
| func_key_size = get_key_size(s3_client, s3_bucket, func_key) | ||
|
|
||
| free_disk_bytes = free_disk_space("/tmp") |
There was a problem hiding this comment.
is /tmp hardcoded elsewhere as well or is there some variable we can use here cc @apengwin
There was a problem hiding this comment.
/tmp is hardcoded in current wrenhandler. The portability PR abstracts this out to a generic TEMP variable that's set at the beginning, depending on what OS is running
There was a problem hiding this comment.
It is actually hardcoded in the runtimes as well. Unfortunately there are some absolute path dependencies that continue to be a problem in the way conda is packaged.
shivaram
left a comment
shivaram
left a comment
There was a problem hiding this comment.
Thanks. One last question: is the catch all except at the end to handle python3 or is to handle other unforeseen exceptions ?
|
So remember right now we only support running under the python2 runtime for the handler; it's for all the exceptions that I wouldn't otherwise catch, to make sure we have a way to clean up the extracted runtime. |
|
Ah got it. I forgot the handler was python2 specific. Sounds good. |
3 participants
This is a series of fixes to handle:
if the runtime is too big, communicate to the user and execute proper cleanup on the worker so the worker isn't forever broken. (issue Too big of runtime #105)
If downloading the func or the data is too large for the remaining space, error as well.