How to zip some features in tf.data.Dataset ?

[thopv133802]

I have a dataset with structure: (x, y_in, y_length, y_out) for my encoder-decoder network.
But my encoder-decoder network only accept ((x, y_in, y_length), y_out).
How to convert from (x, y_in, y_length, y_out) to ((x, y_in, y_length), y_out) ?

[ageron]

Hi @thopv133802 ,
Thanks for your question. I'm not sure whether these are tuples or shapes, and whether x represents features and y represent labels, and I don't know whether x, y_in, y_length and y_out are integers or NumPy arrays, or Tensors, or arbitrary objects.

Could you please clarify? Ideally, please provide a full example, up to the error that you are getting.

If you're talking about tuples, then it's super easy:

x = 1
y_in = 2
y_length = 3
y_out = 4
dataset = (x, y_in, y_length, y_out)
new_data=(dataset[:3], dataset[-1])
# or just:
new_data=((x, y_in, y_length), y_out)

If you are talking about shapes, then consider a simpler example:

import numpy as np
a = np.array([[[1,2,3],[4,5,6]], [[7,8,9],[10,11,12]]])
print(a.shape) # prints (2, 2, 3)

That's a 3D array. Now suppose that I want to have instead two arrays, one with shape (2, 2) and the other with shape (3,). You would end up with a 2D array and a 1D array. There's really no way to do that, at least not without losing information. Indeed, the 3D array contains 223 elements, which is 12. But after the change, we would have a 2D array with just 2*2=4 elements and a 1D array with just 3 elements, so that's a total of 12 elements.