-
Notifications
You must be signed in to change notification settings - Fork 330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reorder channel layer for smooth native RGB - BGR #20
Comments
This seems only about reordering but more in general we have many colorspace conversions in: https://github.com/tensorflow/io/blob/v0.20.0/tensorflow_io/python/experimental/color_ops.py#L20 |
I see. The idea here is to enable BGR-RGB conversion on the model serving bundle, so we don't have to manipulate images "by hand" when serving it in any scenario. |
I understand that this is at Keras layer level implementation but about lowlevel reusable components Remember that we had already another color space overlapping other then with Tensorflow IO SIG also with TF Graphics: tensorflow/graphics#107 |
I see. In that case, that’s just me trying to export and deploy my models
without rechanneling my images out of the model.
Thank you for the info thou.
Em sex., 10 de dez. de 2021 às 11:10, bhack ***@***.***>
escreveu:
I understand that this is at Keras layer level implementation but about
lowlevel reusable components Remember that we had already another color
space overlapping other then with Tensorflow IO SIG also with TF Graphics:
tensorflow/graphics#107
<tensorflow/graphics#107>
https://www.tensorflow.org/graphics/api_docs/python/tfg/image/color_space
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#20 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALLYRXXT4GBHZKEAGDPPAH3UQIC4RANCNFSM5JTDHT6Q>
.
--
--
*Pi Esposito | piesposito.github.io <http://piesposito.github.io>*
|
I understand and it could be useful here. The only problem that I want to highlight is that I prefer to not going again to have a proliferation/duplication of low level ops/API in every SIG. |
I agree with you. What do you think would be the best solution for that? I think that we should have a keras layer for that, but avoiding redundancy of implementations :D. |
I'd be fond of the API:
I think we could include this if someone wanted to contribute it. |
If the parsing of the einsum notation is too verbose/unreadable, we can just have it work like a transpose op where you pass the channel ordering in a list. |
Hey folks, The RGB-BGR operation is very easily possible with the |
@ariG23498, could you post an example of RGB->BGR w/ permute? Thanks! I believe Francois thought this layer would be useful, maybe we can just have some examples showing how to use permute though if it’s possible. |
Hey @LukeWood The layer that I am thinking of should be structured like this. rgb_bgr_layer = keras.layers.Lambda(function=lambda x: x[..., -1::-1]) WDYT? |
We have already this in SIG: https://github.com/tensorflow/io/blob/master/tensorflow_io/python/experimental/color_ops.py#L36 |
If we don't want to depend on ecosystem we could just try to copy and adopt these ops chain in layers or in our private API. |
No, we don't want to rely on the ecosystem. We want first party keras extensions to cover common use cases. |
It is what I have suggested. I think copy these in private api in a colorspace utils file it could be useful. |
I don't think we need to copy these utils as I think we want to do something distinct. This is a generic ReorderChannels layer. It will take an arbitrary ordering and return a new one. The linked function only does RGB->BGR. |
I think that they do more as If you see the whole file they are covering also many color space conversions that It is more than just channel reordering (unstack, stack):
But probably we are not interested to have other colorspace transformations. |
Regarding the Proposal: Reorder channel layer for smooth native RGB - BGR, Feels like a mutual issue, keras-team/keras-io#467 |
Instead of Einsum notation, let's just use:
This is simpler to parse. |
@LukeWood @bhack |
It's an arbitrary transformation regarding the ordering of channels. If we want to include RGBtoGrayscale that would probably need to be it's own layer, same with yuv_to_rgb. These are more complex transformations than a simple permutation shift. Feel free to open an issue for RGBtoGrayscale layer. Not so sure about yuv_to_rgb, but if the community shows need for it we can include it. |
Ramesh will most likely take this task. |
I think reorder channels would have a more general purpose, so we could build that and on top of that the rgb to bgr to rgb transformations. |
Yes, it should be general-purpose. It should support the basic color transformations listed here https://github.com/tensorflow/io/blob/master/tensorflow_io/python/experimental/color_ops.py (shouldn't be limited to rgb/bgr, IMO) channel_reorder = ReorderChannels(format='rgb_to_bgr')( tensor )
channel_reorder = ReorderChannels(format='rgb_to_rgba')( tensor )
channel_reorder = ReorderChannels(format='bgr_to_rgb')( tensor )
channel_reorder = ReorderChannels(format='rgb_to_ycbcr')( tensor )
... |
sure, as far as implementation goes maybe a
makes more sense. In this case, the layer is more of a |
Does not seem like this is a priority on the immediate roadmap - might as well close until theres a strong need. |
* add nn ops * fix style * small * conv * Testing * add docstring * fix typo * add more testing
This is a commonly performed operation in image processing making it a good fit for keras-cv. While this is not a super complicated operation (it's really just a lambda into a tf.gather) the resulting code is more readable. Let's include this in keras-cv.
Originally discussed here: keras-team/keras#15705
Re the design of the layer signature...
The current proposal is to implement a layer that looks like this:
Perhaps we actually may want to use a einsum inspired syntax, such as:
This would be really readable to anyone stumbling upon a new codebase, and generalizes quite well to any number of channels.
Please feel free to comment below with any additional thoughts
The text was updated successfully, but these errors were encountered: