This extension adds functions to the WarpScript™ language to manipulate TensorFlow TFRecords
.
TFRecord
is the recommended format for storing and feeding data into models for both training and inference. It is a compact binary format which can be used in TensorFlow without needing to convert back and forth to Python Numpy arrays.
The use of TFRecords
therefore brings improved performance and efficiency to your Machine Learning pipeline.
Once you have downloaded the .jar
file of the extension via the WarpFleet wf
tool you need to enable the extension by adding the following line to your Warp 10™ configuration:
warpscript.extension.tensorflow = io.warp10.script.ext.tensorflow.TensorFlowWarpScriptExtension
You can then restart your Warp 10™ instance.
The extension adds functions to create tf.train.Example
and tf.train.SequenceExample
instances and pack them in TFRecord
.
It also adds functions to extract information from those formats.