Customize continuous string tensor type for feature hashing on GPU.

[Lifann]

System information

• TensorFlow version (you are using): 1.15 & 2.4
• Are you willing to contribute it (Yes/No): Yes

Background
In a recommender system, there are large number of features are expressed in the form of text, like the name of a product, or behavior of user, in advertisement.

Describe the feature and the current behavior/state.
Currently, Tensorflow use discontinuous host memory to store the text. When hashing the text features, it will lead to lack of performance, by some reasons:

1. Extract string tensor from TFRecord will introduce lots of memcpy for moving string elements one by one.
2. When using GPU to hash the text features, is hard to do memcpy from discontinuous memory to continuous memory, both [H2D] or [H2H + single H2D], since we cannot use discontinuous data on GPU.

Who will benefit with this feature?
I implemented a Murmur hashing with CUDA, wrapped by Tensorflow OP. I found that string copying one by one consumes 90% of the time. So it will make a huge improvement if a continuous string Tensor is enabled.

I'm new to Tensorflow framework type system. How could I create a new tensor type? And what kinds of problems should I consider?

[sanjoy]

I'm not a TFRecord expert, but I can't easily see how this will work reading the TFRecord spec.

Can you more explicitly state what you're proposing? Are you suggesting to create a new format that is like TFRecord, but is formatted differently?

[Lifann]

I'm not a TFRecord expert, but I can't easily see how this will work reading the TFRecord spec.

Can you more explicitly state what you're proposing? Are you suggesting to create a new format that is like TFRecord, but is formatted differently?

Hi. I take quite a while to figure out this problem. And sorry for the delay.

Here is the background: In advertisements, news, videos features, extracted by humans or algorithms, are described as strings or numbers, mostly.
In common case, for string features, we can use tf.strings.to_hash_bucket, tf.strings. to_hash_bucket_fast, tf.strings. to_hash_bucket_strong to encode the features and get embeddings to them.
As to the number feature, we could convert it to string at first. And then use the string hash methods.
The numbers features are widely used in productions mentioned above.

But the such methods are using string as input to hash function, which is inconvenient for GPU to process. And currently, there are CPU OPs only.

I was trying to use a new tensor type to reserve the feature values, in continuous address. When parsing the examples in TFRecord, string features could be parsed directly into the continuous address. And GPU can run on them without bunch of Fragmented copies.

As for number features, I was tending to make the AsString output to the new type of string tensor. But it seems necessary.
I have implemented a set of GPU kernels directly hashing the number features, following using string as intermediate state. About 20 times performance improvement has been achieved, compared to the CPU version.

I'm wondering would it be possible for TensorFlow to accept a GPU implementation instead of tf.as_string + tf.strings. to_hash_bucket ? or just a GPU impl for the AsString OP?

[sanjoy]

I'm wondering would it be possible for TensorFlow to accept a GPU implementation instead of tf.as_string + tf.strings. to_hash_bucket ? or just a GPU impl for the AsString OP?

@Lifann I think this is already done, see #47936.