1mb etcd limit

[sabrehagen]

From the docs:

--from-file test.py: This is the file containing the function code. It is supported to specify a zip file as far as it doesn't exceed the maximum size for an etcd entry (1 MB).

Is there a way to get around this limit? My functions are ~3MB compiled.

[sebgoa]

let's check with @andresmgot for the exact limit of the zip archive.

we are working on a build system, so your functions will be "built" within kubeless and the resulting image will be stored in a docker registry. This will remove any limitation of the current system.

[sabrehagen]

That's great to know, and sounds like the right way to go. How far off releasing do you think you are?

…

On Wed, 28 Mar 2018, 5:09 PM sebgoa ***@***.***> wrote: let's check with @andresmgot <https://github.com/andresmgot> for the exact limit of the zip archive. we are working on a build system, so your functions will be "built" within kubeless and the resulting image will be stored in a docker registry. This will remove any limitation of the current system. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#655 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABCzyCNG__MlqO-mFrgplum-ao-gcIvVks5tizcrgaJpZM4S-FlO> .

[sebgoa]

This was merged 5 days ago:
#621

And the new release should be cut really soon now (~ days)

[andresmgot]

Hi @sabrehagen,

I am afraid that the current build system still uses ConfigMaps and CRDs to obtain the function code so the limitation of 1 MB is still present. Note that --from-file should be used to specify just plain text code, the installation of dependencies and compilation will happen during the build process.

Also note that the limitation is per function, maybe we can help with your specific case with more info: What are you trying to deploy? Is it a zip file?

[sabrehagen]

Thank you for the clarification. It is a zip file, yes. A webpacked nodejs function including dependencies totalling 3mb.

…

On Wed, 28 Mar 2018, 5:23 PM Andres ***@***.***> wrote: Hi @sabrehagen <https://github.com/sabrehagen>, I am afraid that the current build system still uses ConfigMaps and CRDs to obtain the function code so the limitation of 1 MB is still present. Note that --from-file should be used to specify just plain text code, the installation of dependencies and compilation will happen during the build process. Also note that the limitation is per function, maybe we can help with your specific case with more info: What are you trying to deploy? Is it a zip file? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#655 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABCzyD-ao_-EhQI0ZgD8A_LkUx-Pv0k8ks5tizqBgaJpZM4S-FlO> .

[andresmgot]

In that case yes, you will need to include in the Zip file just your code files and then specify as --dependencies your package.json. That way the dependencies will be installed in the build process.

Also note that webpack is not supported yet: serverless/serverless-kubeless#83 (comment)

[sabrehagen]

Our build process is more complex and we use our own serverless webpack plugin that creates a zip with our custom build. The --dependencies method doesn't resolve this for us. Given that our per-function zips will exceed the 1mb limit and your comment that ConfigMaps and CRDs to obtain the function code so the limitation of 1 MB is still present, is there any possibility for us to use Kubeless?

[sebgoa]

Hi, is there a way to discuss your exact workflow in a google docs or via voice ? We would love to understand exactly your workflow and see how it could be enabled.

[alanjds]

Is this considered fixed, as #621 got merged?

[andresmgot]

Not really, the function is still obtained from a ConfigMap to generate the image in #621 so the issue is still there. We are working on #726 that should help though.

[andresmgot]

Now that #726 has been landed the 1MB limitation can be avoided using an URL with the flag --from-file.

[alanjds]

Thanks!

I had already switched tasks but will be back later.

[andresmgot]

closing this issue since this is already solved.