Skip to content

We are going to use CPU for Extract , Transform and Load, and GPU for training model parallelly

License

Notifications You must be signed in to change notification settings

novasush/Parallelization-with-TFDS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 

Repository files navigation

ETL data on CPU and simultaneously Training model on GPU using TFDS

We are going to use CPU for Extract , Transform and Load data from Tensorflow Dataset, and parallelly GPU for training model parallelly.

The code is available for the same in notebook file present in this notebook.

  • This code will not work on Google Colab as you cannot use both cpu and gpu in colab.
  • Make sure you are running this code on Machine containing both CPU and GPU and tensorflow with GPU support. You can find tensorflow gpu installation documentation here.
  • If you are getting
ValueError: No data provided for "input_1". Need data for each key in: ['input_1']

Then you are either running code on google colab or on a machine which has either cpu or gpu only.

You can also refer my blog from here.