Nuclio (Serverless) Components
Nuclio is a native and high performance serverless platform over Kubernetes which automate the process of build, deployment, monitoring, and auto-scaling of micro-services. Nuclio support variety of data and data-science related features (e.g. stream processing, GPUs, volume/DB mounts, high concurrency, etc.)
Nuclio functions can be used in the following ML pipline tasks:
- Data collectors, ETL, stream processing
- Data preparation and analysis
- Hyper parameter model training
- Real-time model serving
- Feature vector assembly (real-time data preparation)
Read more on the use of Nuclio in data-science here. Nuclio functions can be generated automatically from 8 code languages, from Jupyter Notebooks, Zip, Git, Docker, etc. The nuclio-jupyter repo provide guidance and many examples.
There are currently 3 components in this package:
- deploy - Automatically build and deploy/re-deploy functions from code/zip/notebooks/git/.. and/or override various deployment configurations such as setting cpu/mem/gpu resources, scaling, environment variables, triggers, etc.
- delete - Delete a function
- invoker - invoke a function and return the results/logs
Additional components and examples will be added soon for parallel batch/stream processing
Deploy a function (from Github)
nuclio_dep = kfp.components.load_component_from_file('deploy/component.yaml') def my_pipeline(): new_func = nuclio_dep(url='git://github.com/nuclio/nuclio#master:/hack/examples/python/helloworld', name='myfunc', project='myproj', tag='0.11') ...