WarpScript™ TensorFlow extension

This extension adds functions to the WarpScript™ language to manipulate TensorFlow TFRecords.

TFRecord is the recommended format for storing and feeding data into models for both training and inference. It is a compact binary format which can be used in TensorFlow without needing to convert back and forth to Python Numpy arrays.

The use of TFRecords therefore brings improved performance and efficiency to your Machine Learning pipeline.

Installation

Once you have downloaded the .jar file of the extension via the WarpFleet wf tool you need to enable the extension by adding the following line to your Warp 10™ configuration:

  warpscript.extension.tensorflow = io.warp10.script.ext.tensorflow.TensorFlowWarpScriptExtension

You can then restart your Warp 10™ instance.

New functions

The extension adds functions to create tf.train.Example and tf.train.SequenceExample instances and pack them in TFRecord.

It also adds functions to extract information from those formats.