public final class ObjectDetector

Performs object detection on images.

The API expects a TFLite model with TFLite Model Metadata..

The API supports models with one image input tensor and four output tensors. To be more specific, here are the requirements.

  • Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
    • image input of size [batch x height x width x channels].
    • batch inference is not supported (batch is required to be 1).
    • only RGB inputs are supported (channels is required to be 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
  • Output tensors must be the 4 outputs of a DetectionPostProcess op, i.e:
    • Location tensor (kTfLiteFloat32):
      • tensor of size [1 x num_results x 4], the inner array representing bounding boxes in the form [top, left, right, bottom].
      • BoundingBoxProperties are required to be attached to the metadata and must specify type=BOUNDARIES and coordinate_type=RATIO.
    • Classes tensor (kTfLiteFloat32):
      • tensor of size [1 x num_results], each value representing the integer index of a class.
      • if label maps are attached to the metadata as TENSOR_VALUE_LABELS associated files, they are used to convert the tensor values into labels.
    • scores tensor (kTfLiteFloat32):
      • tensor of size [1 x num_results], each value representing the score of the detected object.
    • Number of detection tensor (kTfLiteFloat32):
      • integer num_results as a tensor of size [1].

An example of such model can be found on TensorFlow Hub..

Nested Classes

class ObjectDetector.ObjectDetectorOptions Options for setting up an ObjectDetector. 

Public Methods

static ObjectDetector
createFromBuffer(ByteBuffer modelBuffer)
Creates an ObjectDetector instance with a model buffer and the default ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromBufferAndOptions(ByteBuffer modelBuffer, ObjectDetector.ObjectDetectorOptions options)
Creates an ObjectDetector instance with a model buffer and ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromFile(Context context, String modelPath)
Creates an ObjectDetector instance from the default ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromFile(File modelFile)
Creates an ObjectDetector instance from the default ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromFileAndOptions(Context context, String modelPath, ObjectDetector.ObjectDetectorOptions options)
Creates an ObjectDetector instance from ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromFileAndOptions(File modelFile, ObjectDetector.ObjectDetectorOptions options)
Creates an ObjectDetector instance from ObjectDetector.ObjectDetectorOptions.
List<Detection>
detect(MlImage image)
Performs actual detection on the provided MlImage.
List<Detection>
detect(TensorImage image, ImageProcessingOptions options)
Performs actual detection on the provided image.
List<Detection>
detect(MlImage image, ImageProcessingOptions options)
Performs actual detection on the provided MlImage with ImageProcessingOptions.
List<Detection>
detect(TensorImage image)
Performs actual detection on the provided image.

Inherited Methods

Public Methods

public static ObjectDetector createFromBuffer (ByteBuffer modelBuffer)

Creates an ObjectDetector instance with a model buffer and the default ObjectDetector.ObjectDetectorOptions.

Parameters
modelBuffer a direct ByteBuffer or a MappedByteBuffer of the detection model
Throws
IllegalArgumentException if the model buffer is not a direct ByteBuffer or a MappedByteBuffer * @throws IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ObjectDetector createFromBufferAndOptions (ByteBuffer modelBuffer, ObjectDetector.ObjectDetectorOptions options)

Creates an ObjectDetector instance with a model buffer and ObjectDetector.ObjectDetectorOptions.

Parameters
modelBuffer a direct ByteBuffer or a MappedByteBuffer of the detection model
options
Throws
IllegalArgumentException if the model buffer is not a direct ByteBuffer or a MappedByteBuffer
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ObjectDetector createFromFile (Context context, String modelPath)

Creates an ObjectDetector instance from the default ObjectDetector.ObjectDetectorOptions.

Parameters
context
modelPath path to the detection model with metadata in the assets
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ObjectDetector createFromFile (File modelFile)

Creates an ObjectDetector instance from the default ObjectDetector.ObjectDetectorOptions.

Parameters
modelFile the detection model File instance
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ObjectDetector createFromFileAndOptions (Context context, String modelPath, ObjectDetector.ObjectDetectorOptions options)

Creates an ObjectDetector instance from ObjectDetector.ObjectDetectorOptions.

Parameters
context
modelPath path to the detection model with metadata in the assets
options
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ObjectDetector createFromFileAndOptions (File modelFile, ObjectDetector.ObjectDetectorOptions options)

Creates an ObjectDetector instance from ObjectDetector.ObjectDetectorOptions.

Parameters
modelFile the detection model File instance
options
Throws
IOException if an I/O error occurs when loading the tflite model
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public List<Detection> detect (MlImage image)

Performs actual detection on the provided MlImage.

Parameters
image an MlImage object that represents an image
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the storage type or format of the image is unsupported

public List<Detection> detect (TensorImage image, ImageProcessingOptions options)

Performs actual detection on the provided image.

ObjectDetector supports the following TensorImage color space types:

ObjectDetector supports the following options:

Parameters
image a UINT8 TensorImage object that represents an RGB or YUV image
options the options to configure how to preprocess the image
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the color space type of image is unsupported

public List<Detection> detect (MlImage image, ImageProcessingOptions options)

Performs actual detection on the provided MlImage with ImageProcessingOptions.

ObjectDetector supports the following options:

Parameters
image an MlImage object that represents an image
options the options to configure how to preprocess the image
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the storage type or format of the image is unsupported

public List<Detection> detect (TensorImage image)

Performs actual detection on the provided image.

ObjectDetector supports the following TensorImage color space types:

Parameters
image a UINT8 TensorImage object that represents an RGB or YUV image
Throws
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error
IllegalArgumentException if the color space type of image is unsupported