Skip to content

kolena.fr#

Legacy Warning

The kolena.fr module is considered legacy and should not be used for new projects.

Please see kolena.workflow for customizable and extensible definitions to use for all new projects.

Example Face Recognition (1:1) image pair from the Labeled Faces in the Wild dataset.

Face Recognition (1:1) workflow is built to test models answering the question: do these two images depict the same person? The terminology and methodology are adapted from the NIST FRVT 1:1 challenge.

The Face Recognition (1:1) workflow is also referred to as Face Verification.

Terminology#

Term Definition
Embedding The embedding extracted by your model representing the face depicted in an image
Similarity The similarity score computed from two embeddings extracted by your model, where higher = more similar, lower = less similar
Threshold A threshold applied to computed similarity scores, above which two faces are considered the same
Genuine Pair Two images depicting the same person
Imposter Pair Two images depicting different people
False Match (FM) An incorrect model classification of an imposter pair as a genuine pair
False Match Rate (FMR) The percentage of imposter pairs that are incorrectly classified as genuine pairs (i.e. similarity is above threshold)
False Non-Match (FMR) An incorrect model classification of a genuine pair as an imposter pair
False Non-Match Rate (FNMR) The percentage of genuine pairs that are incorrectly classified as imposter pairs (i.e. similarity is below threshold)
Baseline The test case(s) against which similarity score thresholds are computed (see below)

Test Suite Baseline#

In the Face Recognition (1:1) workflow, similarity score thresholds are computed based on target false match rates over a special section of your test suite known as the baseline.

In the simple standard case, you typically want the baseline to be the entire test suite. However, having control over the baseline allows you to define test suites that answer questions like:

  • How does my model perform on a certain population when the thresholds are computed using a different population (i.e. FMR/FNMR shift)?
  • How does my model perform in different deployment conditions (e.g. low lighting, infrared) when using thresholds computed from standard conditions? (i.e. can my model generalize to unseen scenarios?)

Model Pipeline#

The Face Recognition (1:1) workflow is built to accommodate standard face recognition model pipelines with the following steps:

  1. Face detection (using object detection model)
  2. Face alignment (using landmark detection model)
  3. Feature extraction (using embedding extraction model)
  4. Feature comparison (using embeddings comparison algorithm)

Multiple Faces in a Single Image#

The Face Recognition (1:1) workflow supports extraction of any number of faces from a given image during testing, following the methodology of the NIST FRVT 1:1 Verification specification section Accuracy Consequences of Recognizing Multiple Faces in a Single Image.

During testing, any number between zero (i.e. failure to enroll) and N embeddings may be uploaded for a given image. When computing similarity scores between embeddings from two images in an image pair, all combinations of embeddings extracted from the left and the right image in the pair are computed. When computing metrics, only the highest similarity score between different embeddings in the image pair is considered.

InferenceModel #

Bases: Model

A Model capable of running tests via test.

Currently supports extracting a single embedding per image. To extract multiple embeddings per image, see TestRun.

create(name, extract, compare, metadata) classmethod #

Create a new model with the provided name and metadata.

Parameters:

Name Type Description Default
name str

Unique name of the new model to create.

 required
extract Callable[[str], Optional[np.ndarray]]

A function implementing embeddings extraction for this model.

 required
compare Callable[[np.ndarray, np.ndarray], float]

A function implementing embeddings similarity comparison for this model.

 required
metadata Dict[str, Any]

Unstructured metadata to associate with the model.

 required

Returns:

Type Description
InferenceModel

The newly created model.

Raises:

Type Description
ValueError

A model by the provided name already exists.

load_by_name(name, extract, compare) classmethod #

Load an existing model.

Parameters:

Name Type Description Default
name str

The name of the model to load.

 required
extract Callable[[str], Optional[np.ndarray]]

A function implementing embeddings extraction for this model.

 required
compare Callable[[np.ndarray, np.ndarray], float]

A function implementing embeddings similarity comparison for this model.

 required

Returns:

Type Description
InferenceModel

The loaded model.

Model #

Bases: Uninstantiable['Model.Data']

The descriptor for your model within the Kolena platform.

create(name, metadata) classmethod #

Create a new model with the provided name and metadata.

Parameters:

Name Type Description Default
name str

Unique name of the new model to create.

 required
metadata Dict[str, Any]

Unstructured metadata to associate with the model.

 required

Returns:

Type Description
Model

The newly created model.

Raises:

Type Description
ValueError

A model by the provided name already exists.

iter_pair_results(test_object, batch_size=10000000) #

Iterator over DataFrames of previously stored pair results for this model on the provided test case or test suite, grouped in batches. If this model has not been run on the provided test object, a zero-length response is returned. Partial results are returned when testing on the requested test case or test suite is incomplete.

See Model.load_pair_results for details on the returned DataFrame.

Parameters:

Name Type Description Default
test_object Union[TestSuite, TestSuite.Data, TestCase, TestCase.Data]

The TestSuite or TestCase to load pair results from.

 required
batch_size int

Optionally specify maximum number of rows to be returned in a single DataFrame.

 10000000

Raises:

Type Description
ValueError

An invalid test object was provided.
RemoteError

The pair results could not be loaded for any reason.

load_by_name(name) classmethod #

Retrieve the existing model with the provided name.

Parameters:

Name Type Description Default
name str

Name of the model to retrieve.

 required

Returns:

Type Description
Model

The retrieved model.

Raises:

Type Description
KeyError

If no model with the provided name exists.

load_pair_results(test_object) #

Load previously stored pair results for this model on the provided test case or test suite. If this model has not been run on the provided test object, a zero-length response is returned. Partial results are returned when testing on the requested test case or test suite is incomplete.

The returned DataFrame has the following relevant fields:

  • locator_a: the locator pointing to the left image in the pair
  • locator_b: the locator pointing to the right image in the pair
  • is_same: boolean indicating if the two images depict the same person or a different person
  • image_a_fte: boolean indicating that the left image failed to enroll (FTE)
  • image_b_fte: boolean indicating that the right image failed to enroll (FTE)
  • similarity: float similarity score between the left and right images. NaN if either image failed to enroll. When multiple similarity scores were provided for a given image pair, only the highest similarity score is returned

Parameters:

Name Type Description Default
test_object Union[TestSuite, TestSuite.Data, TestCase, TestCase.Data]

The TestSuite or TestCase to load pair results from.

 required

Raises:

Type Description
ValueError

An invalid test object was provided.
RemoteError

The pair results could not be loaded for any reason.

TestCase(name, version=None, description=None, test_samples=None, reset=False) #

Bases: ABC, Frozen, WithTelemetry

A group of image pairs that can be added to a TestSuite.

The test case is the base unit of results computation in the Kolena platform. Metrics are computed by test case.

name: str instance-attribute #

The unique name of this test case. Cannot be changed after creation.

version: int instance-attribute #

The version of this test case. A test case's version is automatically incremented whenever it is edited via TestCase.edit.

description: str instance-attribute #

Free-form, human-readable description of this test case. Can be edited at any time via TestCase.edit.

image_count: int instance-attribute #

The number of images included in this test case.

pair_count_genuine: int instance-attribute #

The number of genuine image pairs included in this test case.

pair_count_imposter: int instance-attribute #

The number of imposter image pairs included in this test case.

data: API.EntityData property writable #

Deprecated: since 0.57.0

Access this data via instance attributes, e.g. self.image_count, directly.

The data associated with this test case.

Editor(description, reset=False) #

description(description) #

Update the description of this test case.

Parameters:

Name Type Description Default
description str

The new test case description.

 required
add(locator_a, locator_b, is_same) #

Add the provided image pair to the test case.

Note that if the image pair with locator_a and locator_b is already defined within the platform, the value for is_same must match the value already defined.

Parameters:

Name Type Description Default
locator_a str

The left locator for the image pair.

 required
locator_b str

The right locator for the image pair.

 required
is_same bool

Whether to treat this image pair as a a genuine pair (True) or an imposter pair (False).

 required
remove(locator_a, locator_b) #

Remove the provided pair from the test case.

Parameters:

Name Type Description Default
locator_a str

The left locator for the image pair.

 required
locator_b str

The right locator for the image pair.

 required

Raises:

Type Description
KeyError

If the provided locator pair is not in the test case.

create(name, description=None, test_samples=None) classmethod #

Create a new test case with the provided name.

Parameters:

Name Type Description Default
name str

The name of the new test case to create.

 required
description Optional[str]

Optional free-form description of the test case to create.

 None
test_samples Optional[List[TestCaseRecord]]

Optionally specify a set of test samples to populate the test case.

 None

Returns:

Type Description
TestCase

The newly created test case.

load(name, version=None) classmethod #

Load an existing test case with the provided name.

Parameters:

Name Type Description Default
name str

The name of the test case to load.

 required
version Optional[int]

Optionally specify a particular version of the test case to load. Defaults to the latest version when unset.

 None

Returns:

Type Description
TestCase

The loaded test case.

load_by_name(name, version=None) classmethod #

Deprecated: since 0.57.0

Use TestCase.load instead.

Load an existing test case with the provided name.

Parameters:

Name Type Description Default
name str

The name of the test case to load.

 required
version Optional[int]

optionally specify the target version of the test case to load. When absent, the highest version of the test case with the provided name is returned

 None

Returns:

Type Description
TestCase

the loaded test case

load_data() #

Load all image pairs for a test case.

Returns:

Type Description
TestCaseDataFrame

DataFrame containing all pairs defined in this test case.

edit(reset=False) #

Edit this test case in a context:

with test_case.edit() as editor:
    # perform as many editing actions as desired
    editor.add(...)
    editor.remove(...)

Changes are committed to the Kolena platform when the context is exited.

Parameters:

Name Type Description Default
reset bool

Clear all existing test samples in the test case.

 False

iter_data(batch_size=10000000) #

Iterator of DataFrames describing all pairs data for a test case.

Parameters:

Name Type Description Default
batch_size int

Optionally specify maximum number of rows to be returned in a single DataFrame.

 10000000

TestImages #

Bases: Uninstantiable[None]

Registrar #

Bases: Uninstantiable[_Registrar]

add(locator, data_source, width, height, bounding_box=None, landmarks=None, tags=None) #

Add a new image to Kolena. If the provided locator is already registered with the platform, its metadata will be updated.

Parameters:

Name Type Description Default
locator str

Bucket locator for the provided image, e.g. s3://bucket-name/path/to/image.jpg.

 required
data_source str

Name of the source for the image being registered.

 required
width int

Width in pixels of the image being registered.

 required
height int

Height in pixels of the image being registered.

 required
bounding_box Optional[np.ndarray]

Optional 4-element array specifying the ground truth bounding box for this image, of the form [top_left_x, top_left_y, bottom_right_x, bottom_right_y].

 None
landmarks Optional[np.ndarray]

Optional 10-element array specifying (x, y) coordinates for five facial landmarks of the form [left_eye_{x,y}, right_eye_{x,y}, nose_{x,y}, left_mouth_{x,y}, right_mouth_{x,y}].

 None
tags Optional[Dict[str, str]]

Tags to associate with the image, of the form {category: value}.

 None
add_augmented(original_locator, augmented_locator, augmentation_spec, width=None, height=None, bounding_box=None, landmarks=None, tags=None) #

Add an augmented version of an existing image to Kolena.

Note that the original image must already be registered in a previous pass. Tags on the original image are not propagated forward to the augmented image.

Parameters:

Name Type Description Default
original_locator str

The bucket locator for the original version of this image within the platform.

 required
augmented_locator str

The bucket locator for the augmented image being registered.

 required
augmentation_spec Dict[str, Any]

Free-form JSON specification for the augmentation applied to this image.

 required
width Optional[int]

Optionally specify the width of the augmented image. When absent, the width of the corresponding original image is used.

 None
height Optional[int]

Optionally specify the height of the augmented image. When absent, the height of the corresponding original image is used.

 None
bounding_box Optional[np.ndarray]

Optionally specify a new bounding box for the augmented image. When absent, any bounding box corresponding to the original image is used.

 None
landmarks Optional[np.ndarray]

Optionally specify a new set of landmarks for the augmented image. When absent, any set of landmarks corresponding to the original image is used.

 None
tags Optional[Dict[str, str]]

Optionally specify a set of tags to associate with the augmented image.

 None

load(data_source=None, include_augmented=False) classmethod #

Load a DataFrame describing images registered in the Kolena platform.

Parameters:

Name Type Description Default
data_source Optional[Union[str, TestSuite, TestSuite.Data, TestCase, TestCase.Data]]

Optionally specify the single data source to be retrieved, e.g. "my-data-source". Alternatively, provide a TestSuite or TestCase as source. If no argument is provided, all images registered using `TestImages.register are returned.

 None
include_augmented bool

Optionally specify that augmented images should be returned. By default, only original images are returned. Ignored when test case or test suite is provided as data_source.

 False

register() classmethod #

Context-managed interface to register new images with Kolena. Images with locators that already exist in the platform will have their metadata updated. All changes are committed when the context is exited.

Raises:

Type Description
RemoteError

The registered images were unable to be successfully committed for any reason.

iter(data_source=None, include_augmented=False, batch_size=10000000) classmethod #

Iterator of DataFrames describing images registered in the Kolena platform.

Parameters:

Name Type Description Default
data_source Optional[Union[str, TestSuite, TestSuite.Data, TestCase, TestCase.Data]]

Optionally specify the single data source to be retrieved, e.g. "my-data-source". Alternatively, provide a TestSuite or TestCase as source. If no argument is provided, all images registered using TestImages.register are returned.

 None
include_augmented bool

Optionally specify that augmented images should be returned. By default, only original images are returned. Ignored when test case or test suite is provided as data_source.

 False
batch_size int

Optionally specify maximum number of rows to be returned in a single DataFrame.

 10000000

TestRun(model, test_suite, reset=False) #

Bases: ABC, Frozen, WithTelemetry

Interface to test a Model on a TestSuite. Any in-progress tests for this model on this test suite is resumed.

For a streamlined interface, see test.

Parameters:

Name Type Description Default
model Model

The model being tested.

 required
test_suite TestSuite

The test suite on which to test the model.

 required
reset bool

Overwrites existing inferences if set.

 False

create_or_retrieve(model, test_suite, reset=False) classmethod #

Deprecated: since 0.57.0

Use the TestRun constructor instead.

Create a new test run for the provided Model on the provided TestSuite. If a test run for this model on this suite already exists, it is returned.

Parameters:

Name Type Description Default
model Model

The model being tested.

 required
test_suite TestSuite

The test suite on which to test the model.

 required
reset bool

Overwrites existing inferences if set.

 False

Returns:

Type Description
TestRun

The created or retrieved test run.

load_remaining_images(batch_size=10000000) #

Load a DataFrame containing records for each of the images in the configured test suite that does not yet have results from the configured model.

Parameters:

Name Type Description Default
batch_size int

Optionally specify the maximum number of image records to return.

 10000000

Returns:

Type Description
ImageDataFrame

DataFrame containing records for each of the images that must be processed.

Raises:

Type Description
InputValidationError

The requested batch_size failed validation.
RemoteError

Images could not be loaded for any reason.

upload_image_results(df_image_result) #

Upload inference results for a batch of images.

All columns except for image_id and embedding are optional. An empty embedding cell in a record indicates a failure to enroll. The failure_reason column can optionally be specified for failures to enroll.

To provide more than one embedding extracted from a given image, include multiple records with the same image_id in df_image_result (one for each embedding extracted). Records for a given image_id must be submitted in the same df_image_result DataFrame, and not across multiple calls to upload_image_results.

Parameters:

Name Type Description Default
df_image_result ImageResultDataFrame

DataFrame of any size containing records describing inference results for an image.

 required

Returns:

Type Description
int

Number of records successfully uploaded.

Raises:

Type Description
TypeValidationError

The DataFrame failed type validation.
RemoteError

The DataFrame was unable to be successfully ingested for any reason.

load_remaining_pairs(batch_size=10000000) #

Load DataFrames containing computed embeddings and records for each of the image pairs in the configured test suite that have not yet had similarity scores computed.

This method should not be called until all images in the TestRun have been processed.

Parameters:

Name Type Description Default
batch_size int

Optionally specify the maximum number of image pair records to return.

 10000000

Returns:

Type Description
Tuple[EmbeddingDataFrame, PairDataFrame]

Two DataFrames, one containing embeddings computed in the previous step (df_embedding) and one containing records for each of the image pairs that must be computed (df_pair). See documentation on EmbeddingDataFrameSchema for expected format when multiple embeddings were uploaded from a single image in TestRun.upload_image_results.

Raises:

Type Description
InputValidationError

The requested batch_size failed validation.
RemoteError

Pairs could not be loaded for any reason.

upload_pair_results(df_pair_result) #

Upload image pair similarity results for a batch of pairs.

This method should not be called until all images in the TestRun have been processed.

All columns except for image_pair_id and similarity are optional. An empty similarity cell in a record indicates a pair failure (i.e. one or more of the images in the pair failed to enroll).

For image pairs containing images with more than one embedding, a single record may be provided with the highest similarity score, or M x N records may be provided for each embeddings combination in the pair, when there are M embeddings from image_a and N embeddings from image_b.

When providing multiple records for a given image pair, use the embedding_a_index and embedding_b_index columns to indicate which embeddings were used to compute a given similarity score. Records for a given image pair must be submitted in the same df_pair_result DataFrame, and not across multiple calls to upload_pair_results.

Parameters:

Name Type Description Default
df_pair_result PairResultDataFrame

DataFrame containing records describing the similarity score of a pair of images.

 required

Returns:

Type Description
int

Number of records successfully uploaded.

Raises:

Type Description
TypeValidationError

The DataFrame failed type validation.
RemoteError

The DataFrame was unable to be successfully ingested for any reason.

test(model, test_suite, reset=False) #

Test the provided InferenceModel on the provided TestSuite. Any tests already in progress for this model on these suites are resumed.

Parameters:

Name Type Description Default
model InferenceModel

The model being tested, implementing both extract and compare methods.

 required
test_suite TestSuite

The test suite on which to test the model.

 required
reset bool

Overwrites existing inferences if set.

 False

TestSuite(name, version=None, description=None, baseline_test_cases=None, non_baseline_test_cases=None, reset=False) #

Bases: ABC, Frozen, WithTelemetry

A test suite groups together one or more test cases.

name: str instance-attribute #

The unique name of this test suite.

version: int instance-attribute #

The version of this test suite. A test suite's version is automatically incremented whenever it is edited via TestSuite.edit.

description: str instance-attribute #

Free-form, human-readable description of this test suite. Can be edited at any time via TestSuite.edit.

baseline_test_cases: List[TestCase] instance-attribute #

The baseline TestCase object(s) for this test suite.

non_baseline_test_cases: List[TestCase] instance-attribute #

The non-baseline TestCase object(s) for this test suite.

baseline_image_count: int instance-attribute #

The number of images attached to the baseline test case(s).

baseline_pair_count_genuine: int instance-attribute #

The number of genuine pairs attached to the baseline test case(s).

baseline_pair_count_imposter: int instance-attribute #

The count of imposter pairs attached to the baseline test case(s).

data: API.EntityData property writable #

Deprecated: since 0.57.0

Access this data via instance attributes, e.g. self.baseline_test_cases, directly.

The data associated with this test suite.

Editor(description, reset) #

Interface to edit a test suite. Create with TestSuite.edit.

description(description) #

Update the description of the test suite.

Parameters:

Name Type Description Default
description str

The new description of the test suite.

 required
add(test_case, is_baseline=None) #

Add a test case to this test suite. If a different version of the test case already exists in this test suite, it is replaced and its baseline status will be propagated when is_baseline is unset.

Parameters:

Name Type Description Default
test_case TestCase

The test case to add to the test suite.

 required
is_baseline Optional[bool]

Specify that this test case is a part of the "baseline," i.e. if the samples in this test case should contribute to the computation of thresholds within this test suite.

 None
remove(test_case) #

Remove the provided TestCase from the test suite. Any version of this test case in the suite will be removed; the version does not need to match exactly.

Parameters:

Name Type Description Default
test_case TestCase

The test case to remove.

 required
merge(test_case, is_baseline=None) #

Deprecated: since 0.57.0

Replaced by idempotent behavior in TestSuite.Editor.add.

Add the TestCase to the suite. If a test case by this name already exists in the suite, replace the previous version of that test case with the newly provided version.

Parameters:

Name Type Description Default
test_case TestCase

The test case to be merged into the test suite.

 required
is_baseline Optional[bool]

Optionally specify whether or not this test case should be considered as a part of the baseline for this test suite. When not specified, the previous value for is_baseline for this test case in this test suite is propagated forward. Defaults to false if the test case does not already exist in this suite.

 None

create(name, description=None, baseline_test_cases=None, non_baseline_test_cases=None) classmethod #

Create a new test suite with the provided name.

Parameters:

Name Type Description Default
name str

The name of the new test suite to create.

 required
description Optional[str]

Optional free-form description of the test suite to create.

 None
baseline_test_cases Optional[List[TestCase]]

Optionally specify a list of test cases to use as baseline for the test suite.

 None
non_baseline_test_cases Optional[List[TestCase]]

Optionally specify a list of test cases to populate the test suite.

 None

Returns:

Type Description
TestSuite

The newly created test suite.

load(name, version=None) classmethod #

Load an existing test suite with the provided name.

Parameters:

Name Type Description Default
name str

the name of the test suite to load.

 required
version Optional[int]

optionally specify a particular version of the test suite to load. Defaults to the latest version when unset.

 None

Returns:

Type Description
TestSuite

the loaded test suite.

load_by_name(name, version=None) classmethod #

Deprecated: since 0.57.0

Use TestSuite.load instead.

Retrieve the existing test suite with the provided name.

Parameters:

Name Type Description Default
name str

Name of the test suite to retrieve.

 required
version Optional[int]

Optionally specify the version of the named test suite to retrieve. When absent the latest version of the test suite is returned.

 None

Returns:

Type Description
TestSuite

The retrieved test suite.

Raises:

Type Description
NotFoundError

If the test suite with the provided name doesn't exist.

edit(reset=False) #

Context-managed way to perform many modification options on a test suite and commit the results when the context is exited, resulting in a single version bump.

with TestSuite.load("my-test-suite").edit() as editor:
    editor.add(TestCase.load("my-test-case"))

Data Types#

Schema declarations for the Pandas DataFrames used in kolena.fr.

TestImageDataFrameSchema #

Bases: pa.SchemaModel

image_id: Series[pa.typing.Int64] = pa.Field(coerce=True) class-attribute instance-attribute #

Internal ID corresponding to this image.

locator: Series[pa.typing.String] = pa.Field(coerce=True, _validate_image_locator=()) class-attribute instance-attribute #

External locator pointing to image in bucket.

data_source: Series[pa.typing.String] = pa.Field(coerce=True, nullable=True) class-attribute instance-attribute #

Specify source dataset, e.g. "CIFAR-10".

width: Series[pa.typing.Int64] = pa.Field(coerce=True, _validate_optional_dimension=()) class-attribute instance-attribute #

Width of the image, in pixels. The value -1 is used to specify "unspecified".

height: Series[pa.typing.Int64] = pa.Field(coerce=True, _validate_optional_dimension=()) class-attribute instance-attribute #

Height of the image, in pixels. The value -1 is used to specify "unspecified".

original_locator: Series[pa.typing.String] = pa.Field(coerce=True, nullable=True, _validate_image_locator=()) class-attribute instance-attribute #

Specify that this image is an augmented version of another (registered) image.

augmentation_spec: Series[JSONObject] = pa.Field(coerce=True, nullable=True, _validate_json_object=()) class-attribute instance-attribute #

Free-form specification describing the augmentation applied to the image, e.g. {"rotate": 90}. Should not be specified unless original_locator is present.

bounding_box: Series[BoundingBox] = pa.Field(coerce=True, nullable=True, _validate_bounding_box=()) class-attribute instance-attribute #

Ground truth bounding box. If absent, no ground truth will be available for display.

landmarks: Series[Landmarks] = pa.Field(coerce=True, nullable=True, _validate_landmarks=()) class-attribute instance-attribute #

Ground truth landmarks. If absent, no ground truth will be available for display.

tags: Series[JSONObject] = pa.Field(coerce=True, _validate_tags=()) class-attribute instance-attribute #

Specify a set of tags to apply to this object in the form {category: value}. Note that this format intentionally restricts tags to a single value per category.

ImageResultDataFrameSchema #

Bases: pa.SchemaModel

image_id: Series[pa.typing.Int64] = pa.Field(coerce=True) class-attribute instance-attribute #

[Required] The ID of the image corresponding to this record.

bounding_box: Optional[Series[BoundingBox]] = pa.Field(coerce=True, nullable=True, _validate_bounding_box=()) class-attribute instance-attribute #

[Optional] A bounding box around the face detected in this image.

landmarks_input_image: Optional[Series[RGBImage]] = pa.Field(coerce=True, nullable=True, _validate_rgb_image=()) class-attribute instance-attribute #

[Optional] RGB image (np.ndarray with cells of type np.uint8) corresponding to the input to the "landmarks" model in the face recognition pipeline.

landmarks: Optional[Series[Landmarks]] = pa.Field(coerce=True, nullable=True, _validate_landmarks=()) class-attribute instance-attribute #

[Optional] A 10-element array with (x, y) coordinates corresponding to the left eye, right eye, nose tip, left mouth corner, right mouth corner of the detected face.

quality_input_image: Optional[Series[RGBImage]] = pa.Field(coerce=True, nullable=True, _validate_rgb_image=()) class-attribute instance-attribute #

[Optional] RGB image (np.ndarray with cells of type np.uint8) corresponding to the input to the "quality" model in the face recognition pipeline.

quality: Optional[Series[pa.typing.Float64]] = pa.Field(coerce=True, nullable=True) class-attribute instance-attribute #

[Optional] Score produced by the "quality" model in the face recognition pipeline.

acceptability: Optional[Series[pa.typing.Float64]] = pa.Field(coerce=True, nullable=True) class-attribute instance-attribute #

[Optional] Score produced by the "acceptability" model in the face recognition pipeline.

fr_input_image: Optional[Series[RGBImage]] = pa.Field(coerce=True, nullable=True, _validate_rgb_image=()) class-attribute instance-attribute #

[Optional] RGB image (np.ndarray with cells of type np.uint8) corresponding to the input to the facial embeddings extraction model in the face recognition pipeline.

embedding: Series[EmbeddingVector] = pa.Field(coerce=True, nullable=True, _validate_embedding_vector=()) class-attribute instance-attribute #

[Optional] Embedding vector (np.ndarray) extracted representing the face detected in the image. An empty cell (no array is provided) indicates a failure to enroll for the image, i.e. no face detected.

failure_reason: Optional[Series[pa.typing.String]] = pa.Field(coerce=True, nullable=True) class-attribute instance-attribute #

[Optional] The reason why the image was a failure to enroll.

EmbeddingDataFrameSchema #

Bases: pa.SchemaModel

image_id: Series[pa.typing.Int64] = pa.Field(coerce=True) class-attribute instance-attribute #

The provided ID of the image.

embedding: Series[EmbeddingVector] = pa.Field(nullable=True, coerce=True, _validate_embedding_vector=()) class-attribute instance-attribute #

The extracted embedding(s) corresponding to the image_id. A missing embedding indicates a failure to enroll for the image.

For images with only one extracted embedding, the embedding is a one-dimensional np.ndarray with length matching the length of the extracted embedding. When multiple embeddings were extracted from a single image, the first dimension represents the index of the extracted embedding. For example, for an image with 3 extracted embeddings and embeddings of length 256, the embedding is an array of shape (3, 256).

PairResultDataFrameSchema #

Bases: pa.SchemaModel

image_pair_id: Series[pa.typing.Int64] = pa.Field(coerce=True) class-attribute instance-attribute #

[Required] The ID of the image corresponding to this record.

similarity: Series[pa.typing.Float64] = pa.Field(nullable=True, coerce=True) class-attribute instance-attribute #

[Optional] The similarity score computed between the two embeddings in this image pair. Should be left empty when either image in the pair is a failure to enroll.

embedding_a_index: Optional[Series[pa.typing.Int64]] = pa.Field(nullable=True, coerce=True) class-attribute instance-attribute #

[Optional] Index of the embedding in image_a corresponding to this similarity score. Required when multiple embeddings are extracted per image and multiple similarity scores are computed per image pair.

embedding_b_index: Optional[Series[pa.typing.Int64]] = pa.Field(nullable=True, coerce=True) class-attribute instance-attribute #

[Optional] Index of the embedding in image_b corresponding to this similarity score. Required when multiple embeddings are extracted per image and multiple similarity scores are computed per image pair.