Custom init containers for vertex pods

[dseapy]

Summary

Add the ability for users to add init containers to the udf pods.

Use Cases

Currently the udf container is responsible for downloading the ML model. Instead it would be preferred for my use-case if there was an init-container (maintained by devops, not data-scientists developing the udf containers) that was responsible for acquiring the model and sharing it with the udf container through a volumemount. This provides a couple advantages:

• Ensures consistency for configuration/implementation of acquiring the model.
• Provides an easy way to migrate if/when storage implementation changes (i.e. custom s3 endpoint, move from s3 to another storage provider, etc).
• Simplifies debugging & any required fixes by devops in case there are network-related issues, since they own a single container.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[dseapy]

Alternatively, if numaflow provided a way to provide files from object storage to udf pods, that would be simpler than custom init-containers for the model-download use-case. Maybe it makes sense to make a ticket for that instead/in-addition? Maybe in-addition, as there may be other use-cases in the future for which specifying more custom init containers would be useful.

[vigith]

I would think giving custom init containers is better than our code abstracting out the download of objects. If we are to abstract, we will be strictly limiting our use cases.

[whynowy]

I would think giving custom init containers is better than our code abstracting out the download of objects. If we are to abstract, we will be strictly limiting our use cases.

Agreed.