How to pad data in Tensorflow Transform so that sequences are of equal lenght ?

[anantvir]

I am building a data pipeline using Tensorflow Transform using Apache Beam. Inside the preprocessing function I am generating vocabulary which is converting my input sentences to list of integers.
My data has 3 features:
Column 1 = Context (dtype = String) (Sentences of varying Length)
Column 2 = Utterance (dtype = String) (Sentences of varying Length)
Column 3 = Label (0/1)
I want to right pad my list of integers as soon as they are generated in the preprocessing function below to a max length of 160 words.
Example:
"I love Pizza" --> [34, 67, 78] --> Max length I want = 10
then I want [34, 67, 78,0,0,0,0,0,0,0] and if the length of my sentence is already greater than 10, then I want to trim the extra portion to make it length =10

Now to use tf.keras.preprocessing.sequence.pad_sequences you need as input a list of sequences but as shown below, my mapped_context and mapped_utterance are tf.int64 tensors. So I am not able to use the padding functionality of Keras.
Can someone please help me achieve this ?

The reference code I am following is Tensorflow Sentiment Analysis Example:
https://github.com/tensorflow/transform/blob/599691c8b94bbd6ee7f67c11542e7fef1792a566/examples/sentiment_example.py

My code's preprocessing function is below:

[anantvir]

can someone please update me on this ? It would be a great help !

[gowthamkpr]

@anantvir You can use tf.pad to pad the tensors in this case.

[anantvir]

I have sentences of different length which are tensors. And I want each sentence to be of length 200. So if they are larger, i want them trimmed to 200 and if smaller I want them padded to 200. Do you have any idea how can I do that ?

Below is my dataset

[zoyahav]

To trim the data since is should be formatted as SparseTensors you could do something like this:

_MAX_VALUES = 3
x = tf.compat.v1.string_split(['a,b,c,d,e,f', '1,2,3'], sep=',')
batch_size = x.dense_shape[0]
trimmed = tf.sparse.slice(x, [0, 0], [batch_size, _MAX_VALUES])

And to pad, again since the data is Sparse you could just use tf.sparse.to_dense after the slice with default_value=0.

here is how we did it:

feature = _fill_in_missing(inputs['request_url'])
    feature = create_ml_feature_string(feature)
    feature = tf.strings.strip(feature)
    feature = tf.strings.split(feature,
                                      name='stripped_to_ragged_feature',
                                      sep=' ', )
    feature = feature.to_tensor(shape=(None, _MAX_REQUEST_LEN),
                                             default_value='pad',
                                             name='ragged_to_dense_feeaturee', )

[skeptisq]

@zoyahav I could not get the solution you proposed for padding to work:

_MAX_VALUES = 8
x = tf.compat.v1.string_split(['a,b,c,d,e,f', '1,2,3'], sep=',')
batch_size = x.dense_shape[0]
trimmed = tf.sparse.slice(x, [0, 0], [batch_size, _MAX_VALUES])
padded = tf.sparse.to_dense(trimmed, default_value='')
tf.print(padded)

The final result in tensorflow==2.1.0:

[["a" "b" "c" "d" "e" "f"]
 ["1" "2" "3" "" "" ""]]

Note it is of length 6 not _MAX_VALUES = 8.

I think the reason is tf.sparse.to_dense does not accept a dense_shape argument to enforce the shape (it is easy to add it).

[zoyahav]

@habibfarr note that tf.sparse.to_dense depends on the sparse tensor's dense_shape. So if we modify the code to be:

_MAX_VALUES = 8 
x = tf.compat.v1.string_split(['a,b,c,d,e,f', '1,2,3'], sep=',')
x = tf.SparseTensor(values=x.values, indices=x.indices, dense_shape=[batch_size, _MAX_VALUES])
batch_size = x.dense_shape[0] 
trimmed = tf.sparse.slice(x, [0, 0], [batch_size, _MAX_VALUES]) 
padded = tf.sparse.to_dense(trimmed, default_value='') 
tf.print(padded, summarize=100)

The result is then:

[["a" "b" "c" "d" "e" "f" "" ""]
 ["1" "2" "3" "" "" "" "" ""]]

[arghyaganguly]

Closing this based on above comment trace.Thanks.