A model is a deterministic transformation turning an input into a set of predictions.
Object Detection / Instance Segmentation
Face Recognition
Keypoints Detection
# specific types are dependent on your workflow implementation and are entirely customizable
from my_workflow import MyTestSample, MyInference
def infer(test_sample: MyTestSample) -> MyInference:
"""Transform your test sample into inferences"""
from kolena.detection import TestImage
from kolena.detection.inference import BoundingBox
# to test Instance Segmentation models, use the SegmentationMask inference type
# from kolena.detection.inference import SegmentationMask
def infer(test_image: TestImage) -> List[BoundingBox]:
"""Transform a TestImage into a list of BoundingBox inferences"""
# Step 1: load image at `test_image.locator`
# Step 2: perform inference
# Step 3: transform inferences into BoundingBox objects and return
from kolena.classification import TestImage
Classification = Tuple[str, float] # (label, confidence) pairs
def infer(test_image: TestImage) -> List[Classification]:
"""Transform a TestImage into a list of (label, confidence) inferences"""
# Step 1: load image at `test_image.locator`
# Step 2: perform inference and return
import numpy as np
def extract(locator: str) -> np.ndarray:
"""Extract an embedding representing the face in the image"""
# Step 1: load image at `locator`
# Step 2: run model pipleine -- detect, align, and extract
# Step 3: return extracted embedding, or None if no face was detected
def compare(embedding_a: np.ndarray, embedding_b: np.ndarray) -> float:
"""Compare two embeddings and generate similarity score"""
from kolena.keypoints import TestImage
from kolena.keypoints.inference import Keypoints
def infer(test_image: TestImage) -> Keypoints:
"""Transforms a TestImage into a list of Keypoints inferences"""
# Step 1: load image at `test_image.locator`
# Step 2: perform inference
# Step 3: transform inferences into Keypoints objects and return
In Kolena, a "model" is simply a unique descriptor that identifies a model you've trained. When you test your models, you run inference on your hardware and upload results via kolena-client. Your model is never uploaded to Kolena and you maintain complete control over the testing process.

Model Registry

Models tested with Kolena are tracked in the Model Registry. This registry can be used to track models you've released as well as new experiments being evaluated.
The Model Registry. All models tested with Kolena are displayed.
Clicking on a model brings up its associated metadata and other information such as the test suites it's been run on. Here let's look at YOLOR-D6 (modified CSP, 1280x1280, coco17, pytorch-1.7):
Detailed view of a model in the Model Registry.
When this model was created, relevant metadata such as the framework, training dataset, source code, and responsible engineer was entered. This information can be used to track your model development process and provide easy visibility into things such as:
  • Models trained by a given team member
  • Training dataset and methodology
  • Exact code used to test that model
  • Location in e.g. an S3 bucket where the model weights are stored

Model Best Practices & FAQ

What should I name my model?
What metadata should I include with my model?
Copy link
On this page
Model Registry
Model Best Practices & FAQ