Lambda now support Python 3.6

[zkanda]

Hi,

I will try it today, but just to track it, lambda now supports python 3.6.

Cheers!

Edit: Release notes - https://aws.amazon.com/releasenotes/5198208415517126

[zkanda]

Just an update, currently it doesn't work with 3.6:

START RequestId: d677fc2d-2661-11e7-9c3d-f55e39171089 Version: $LATEST
module initialization error: '/var/task/handler/__init__.py/__init__.pyc' is not a package

END RequestId: d677fc2d-2661-11e7-9c3d-f55e39171089
REPORT RequestId: d677fc2d-2661-11e7-9c3d-f55e39171089	Duration: 0.27 ms	Billed Duration: 100 ms 	Memory Size: 128 MB	Max Memory Used: 13 MB

I'm not sure if you have plan on supporting both, if not, should I close this? :)

[fsenart]

@zkanda as you've noticed THE change from Python 2 to 3 is the definition of a module in C. We will update the package asap to support Python 3.
We may have to perform other changes as well, but we think that these changes won't break the current interface.
Stay tuned 😉

[heri16]

Can't wait for Python 3 shim support...

[fsenart]

We have to produce 2 different binaries to support both Python 2 and 3. This comes with some friction because we have to provide a packaging process that allows the end user to choose between the two versions. That's why we haven't yet released a version which supports Python 3.

We are wondering:

• should we abandon Python 2 and force migration to Python 3 only
• should we support both
• should we stick with Python 2, as Python 3 seems to not have any positive impact on the shim.

The question is open, and it would be great if you guys can express your opinion about.

[heri16]

Isn't python wheels already supporting both targets at the same time. Try wheels on Traces CI.

[fsenart]

@heri16 it is not related to the python packaging process but the shim packaging process to produce the final Lambda zip package.

[heri16]

Sorry, I just realized I don't quite understand which binaries you mean. If you mean the docker image, the different versions is normally achieved via docker image tags. Look at docker php 7.

[fsenart]

Yeah I agree with you for docker tags and this is one of the possible solution to provide both binaries. But as I tried to explain above, this will come with some friction for end users as historically users integrated this project by using the latest tag of the docker image.
I really think that the good question to ask and to discuss about is, why should we support (no matter how) Python 3, regarding its advantages over Python 2 in the context of this shim. Does users gain any + if we support Python 3?

[heri16]

Good question. Python 2.7 is no longer being developed but we could forsee AWS supporting it for sometime to come. Not sure what is the cold start up performance of the new aws python engine also, since python 3.5+ is mainly adding better integration of async and coroutines for performance reasons. Could you point me to information on how this shim works again between the python side and the golang side so I can look into this?

[fsenart]

For the Python 2 support by AWS, we can imagine migrating to Python 3 as soon as AWS abandons Python 2.
For the cold start performance, we have to perform some new benchmarks to compare with current performances. I will come up with an alpha branch or so to test it.
Notice we do not use any advanced Python feature in the shim. The only purpose of the proxy.py file is to json marshal incoming events and the runtime.c have a set of pretty simple methods serving as a thin interface between python an go. So I can't imagine any performance gain here. But if the AWS Python 3 runtime outclasses the AWS Python 2 runtime, then we have to migrate to this one.

[heri16]

Regardless, https://github.com/lambci/docker-lambda uses tags to differentiate between python 3.6 and python 2.7. We might want to maintain 2 branches so contributors could work on optimizing the python 3 version with python 3 features. I don't think migrated code from python 2.7 would perform any better.

[heri16]

FYI: https://github.com/lambci/docker-lambda

[heri16]

FYI: https://speed.python.org/comparison/?exe=12%2BL%2Bmaster%2C12%2BL%2B3.5%2C12%2BL%2B3.6%2C12%2BL%2B2.7&ben=617%2C618%2C620%2C634%2C635&env=1&hor=false&bas=none&chart=normal+bars

[fsenart]

@heri16

optimizing the python 3 version with python 3 features.

The point is that this project does not use any particular python specific feature. This is why I tend to advocate on not migrating to Python 3.6.

I don't think migrated code from python 2.7 would perform any better.

I totally agree with you. Even more when seeing the benchmark you've posted. The difference in method calls is nearly null and Python 2.7 performs better for JSON dumps (which are a critical part in the shim).

These information shows that we should not migrate to Python 3.6 unless the whole AWS Python 3.6 runtime performs really better than the the Python 2.7 one.

[fsenart]

I will close this issue for the moment, until someone comes with a big argument in favor of Python 3.6 in the context of this shim.