-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ Lazy image reading support (reading, converting, resizing) #1288
base: master
Are you sure you want to change the base?
Conversation
|
||
@vaex.register_function(scope='image') | ||
def as_image(arrays): | ||
images = [rgba_2_pil(image_array) for image_array in images] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
images = [rgba_2_pil(image_array) for image_array in **arrays**]?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, that's wrong, you can also see this is not tested yet in the unittests, feel free to fix.
I think the printing of the image in different sizes in an overkill, have the image always look as in the first "image" column is fine, but the resize is definitely important. |
this is not something vaex is doing, if you look at the code you see I created 3 image columns, just for a demo. |
We might want a helper function to find all files in a dir. Something like this orso
|
Hi @maartenbreddels @xdssio are there plans to continue/merge these efforts? We use Vaex for a lot of processing now and plan to use it for images in the near future :-] |
@JovanVeljanoski what do you think? |
Yeah, I think this is definitely worth looking into at some point soon. I think it would be quite cool do various types of pre-processing on all images instead of per-batch, especially for deep NN stuff. I wonder what impact that would have. We are working hard on the next major version, and the roadmap for that is pretty much fixed i believe. It revolves around stabilizing features like After that I do not know what the plan is yet, so we could look into this. We typically look at what is most in demand or has the highest impact. Or if someone is willing to fund/sponsor the development of certain features, it would get a priority. Do you agree @maartenbreddels ? |
I agree, i'd like to have a feature for displaying images, but lets focus on those things you mention first, and work on an example that uses/requires images, unless funding ups the priority. |
To continue the discussion.
|
369423b
to
6927b35
Compare
Example:
cc @xdssio