-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
construct url-safe url when downloading file onto container #453
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks sensible to me. I have to admit, I don't fully understand the whole flow here but what you've done looks correct and like the fact that you've added tests to cover all the different permutations. I wasn't sure whether calling encode_url
which has a pydantic decorator on it would work outside of pydantic validation but it obviously must do :)
Thanks for fixing this 👍
Yeah no worries 👍. I wasn't sure either about calling the pydantic decorated method from outside either, but it worked. Considered moving the encoding logic to a utility function defined outside the pydantic model, but probably fine as is. |
When downloading a file onto the container, we safe-encode the url before downloading the file, e.g. white spaces and other special chars will be encoded etc..
Also, to ensure backward compatibility (for pipelines using older pipeline-ai versions) and to encode any file_urls at the API entry level, we update the RunInput schema to recursively encode any file_url nested within. Note also we ensure that urls are not encoded multiple times.