Skip to content

Conversation

@API92
Copy link
Contributor

@API92 API92 commented Jun 21, 2022

If I save tf.keras.layers.LSTM layer with _could_use_gpu_kernel=True into the SavedModel format with tf.saved_model.save, then cuDNN kernel is used when I load this model with tf.saved_model.load and it works fast. But if I load this model from C++ with tensorflow::LoadSavedModel function, then cuDNN kernel isn't used and it is slow.

Here is colab demonstrating the issue https://colab.research.google.com/drive/16WN0sqOoL37M7-5XMhGb-irkRX7fh503?usp=sharing . If model is loaded with tf.saved_model.load, then tf.keras.layers.LSTM and tf.raw_ops.CudnnLSTM layers both takes about 100 ms. But if model is loaded with LoadSavedModel from C++, then tf.keras.layers.LSTM takes about 275 ms, while tf.raw_ops.CudnnLSTM takes 100 ms.

There were some problems in FunctionOptimizer, ImplementationSelector and MetaOptimizer optimizers:

  1. When FunctionOptimizer specialized some function, it copied api_implements from origin function to specialization. It caused an error "Function ... and ... both implment 'interface_name' but their signatures do not match" in FunctionLibraryApiInfo::Init
  2. MetaOptimizer didn't pass whole FunctionDefLibrary along with a body of single function, which is needed for implementation selection.
  3. ImplementationSelector couldn't parse empty device name if it isn't specified on a node.

@google-ml-butler google-ml-butler bot added the size:L CL Change Size: Large label Jun 21, 2022
@tilakrayal tilakrayal added the comp:core issues related to core part of tensorflow label Jun 22, 2022
@tilakrayal tilakrayal requested a review from penpornk June 22, 2022 10:02
@google-ml-butler google-ml-butler bot added the awaiting review Pull request awaiting review label Jun 22, 2022
@gbaned
Copy link
Contributor

gbaned commented Aug 3, 2022

Hi @penpornk Can you please review this PR ? Thank you!

@penpornk penpornk requested review from ezhulenev and removed request for penpornk August 3, 2022 17:52
@gbaned gbaned added comp:grappler Grappler related issues and removed comp:core issues related to core part of tensorflow labels Aug 9, 2022
@gbaned
Copy link
Contributor

gbaned commented Dec 15, 2022

Hi @ezhulenev Can you please review this PR ? Thank you!

@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Dec 15, 2022
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Dec 15, 2022
@gbaned gbaned removed awaiting review Pull request awaiting review ready to pull PR ready for merge process labels Dec 16, 2022
@gbaned
Copy link
Contributor

gbaned commented Dec 16, 2022

@API92 Can you please check build failures. Thank you!

@gbaned gbaned added the stat:awaiting response Status - Awaiting response from author label Dec 16, 2022
@API92
Copy link
Contributor Author

API92 commented Dec 16, 2022

@gbaned Fixed.

@google-ml-butler google-ml-butler bot removed the stat:awaiting response Status - Awaiting response from author label Dec 16, 2022
@gbaned gbaned added the kokoro:force-run Tests on submitted change label Dec 19, 2022
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Dec 19, 2022
@gbaned gbaned requested a review from ezhulenev December 19, 2022 11:25
@google-ml-butler google-ml-butler bot added the awaiting review Pull request awaiting review label Dec 19, 2022
@gbaned gbaned requested review from ezhulenev and removed request for ezhulenev December 29, 2022 09:18
@gbaned
Copy link
Contributor

gbaned commented Feb 14, 2023

Hi @ezhulenev Can you please review this PR ? Thank you!

@penpornk penpornk requested a review from reedwm February 14, 2023 13:10
@penpornk
Copy link
Member

Adding @reedwm since @ezhulenev is on vacation for a while.

@keerthanakadiri keerthanakadiri added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Oct 1, 2024
@keerthanakadiri keerthanakadiri added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Oct 14, 2024
@keerthanakadiri keerthanakadiri added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Oct 23, 2024
@keerthanakadiri keerthanakadiri added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Nov 4, 2024
@keerthanakadiri
Copy link
Contributor

Hi @API92 , Can you please resolve the conflicts? Thank you!

@API92 API92 force-pushed the lstm_cudnn_impl_selection branch from 95bdb1d to e87a7c7 Compare November 17, 2024 02:04
@google-ml-butler google-ml-butler bot removed the ready to pull PR ready for merge process label Nov 17, 2024
@google-ml-butler google-ml-butler bot added the awaiting review Pull request awaiting review label Nov 18, 2024
@keerthanakadiri keerthanakadiri requested review from ezhulenev and removed request for ezhulenev December 4, 2024 07:26
@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jan 9, 2025
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Jan 9, 2025
@google-ml-butler google-ml-butler bot removed the ready to pull PR ready for merge process label Jan 9, 2025
@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jan 9, 2025
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Jan 9, 2025
@copybara-service copybara-service bot merged commit a72d9bf into tensorflow:master Jan 9, 2025
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review Pull request awaiting review comp:grappler Grappler related issues ready to pull PR ready for merge process size:L CL Change Size: Large

Projects

Status: Merged

Development

Successfully merging this pull request may close these issues.

9 participants