public final class ImageSearcher

Performs similarity search on images.

The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..

  • Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
    • image input of size [batch x height x width x channels].
    • batch inference is not supported (batch is required to be 1).
    • only RGB inputs are supported (channels is required to be 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
  • Output tensor (kTfLiteUInt8/kTfLiteFloat32)
    • N components corresponding to the N dimensions of the returned feature vector for this output layer.
    • Either 2 or 4 dimensions, i.e. [1 x N] or [1 x 1 x 1 x N].

TODO(b/180502532): add pointer to example model.

TODO(b/222671076): add factory create methods without options, such as `createFromFile`, once the single file format (index file packed in the model) is supported.

Nested Classes

class ImageSearcher.ImageSearcherOptions Options for setting up an ImageSearcher. 

Public Methods

static ImageSearcher
createFromBufferAndOptions(ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options)
Creates an ImageSearcher instance with a model buffer and ImageSearcher.ImageSearcherOptions.
static ImageSearcher
createFromBufferAndOptionsImpl(ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options, int indexFd)
static ImageSearcher
createFromFileAndOptions(Context context, String modelPath, ImageSearcher.ImageSearcherOptions options)
Creates an ImageSearcher instance from ImageSearcher.ImageSearcherOptions.
static ImageSearcher
createFromFileAndOptions(File modelFile, ImageSearcher.ImageSearcherOptions options)
Creates an ImageSearcher instance.
List<NearestNeighbor>
search(TensorImage image, ImageProcessingOptions options)
Performs embedding extraction on the provided TensorImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.
List<NearestNeighbor>
search(MlImage image)
Performs embedding extraction on the provided MlImage, followed by nearest-neighbor search in the index.
List<NearestNeighbor>
search(MlImage image, ImageProcessingOptions options)
Performs embedding extraction on the provided MlImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.
List<NearestNeighbor>
search(TensorImage image)
Performs embedding extraction on the provided TensorImage, followed by nearest-neighbor search in the index.

Inherited Methods

Public Methods

public static ImageSearcher createFromBufferAndOptions (ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options)

Creates an ImageSearcher instance with a model buffer and ImageSearcher.ImageSearcherOptions.

Parameters
modelBuffer a direct ByteBuffer or a MappedByteBuffer of the search model
options
Throws
IllegalArgumentException if the model buffer is not a direct ByteBuffer or a MappedByteBuffer
IOException if an I/O error occurs when loading the index file
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ImageSearcher createFromBufferAndOptionsImpl (ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options, int indexFd)

Parameters
modelBuffer
options
indexFd

public static ImageSearcher createFromFileAndOptions (Context context, String modelPath, ImageSearcher.ImageSearcherOptions options)

Creates an ImageSearcher instance from ImageSearcher.ImageSearcherOptions.

Parameters
context
modelPath path of the search model with metadata in the assets
options
Throws
IOException if an I/O error occurs when loading the tflite model or the index file
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public static ImageSearcher createFromFileAndOptions (File modelFile, ImageSearcher.ImageSearcherOptions options)

Creates an ImageSearcher instance.

Parameters
modelFile the search model File instance
options
Throws
IOException if an I/O error occurs when loading the tflite model or the index file
IllegalArgumentException if an argument is invalid
IllegalStateException if there is an internal error
RuntimeException if there is an otherwise unspecified error

public List<NearestNeighbor> search (TensorImage image, ImageProcessingOptions options)

Performs embedding extraction on the provided TensorImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.

ImageSearcher supports the following options:

ImageSearcher supports the following TensorImage color space types:

Parameters
image a UINT8 TensorImage object that represents an RGB or YUV image
options
Throws
IllegalArgumentException if the color space type of image is unsupported

public List<NearestNeighbor> search (MlImage image)

Performs embedding extraction on the provided MlImage, followed by nearest-neighbor search in the index.

Parameters
image an MlImage object that represents an image
Throws
IllegalArgumentException if the storage type or format of the image is unsupported

public List<NearestNeighbor> search (MlImage image, ImageProcessingOptions options)

Performs embedding extraction on the provided MlImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.

ImageSearcher supports the following options:

Parameters
image a MlImage object that represents an image
options configures options including ROI and rotation
Throws
IllegalArgumentException if the storage type or format of the image is unsupported

public List<NearestNeighbor> search (TensorImage image)

Performs embedding extraction on the provided TensorImage, followed by nearest-neighbor search in the index.

ImageSearcher supports the following TensorImage color space types:

Parameters
image a UINT8 TensorImage object that represents an RGB or YUV image
Throws
IllegalArgumentException if the color space type of image is unsupported