The ground truth associated with a
TestSample. Typically, a ground truth will represent
the expected output of a model when given a test sample and will be manually annotated by a human.
from dataclasses import dataclass from typing import List from kolena.workflow import GroundTruth from kolena.workflow.annotation import Polyline, SegmentationMask @dataclass(frozen=True) class AvGroundTruth(GroundTruth): road_area: SegmentationMask lane_boundaries: List[Polyline] visibility_score: int
TestCase holds a list of test samples (model inputs) paired with ground truths
The ground truth against which a model is evaluated.
A test case contains one or more
TestSample objects each paired with a ground truth
object. During evaluation, these test samples, ground truths, and your model's inferences are provided to the
This object may contain any combination of scalars (e.g.
Annotation objects, or lists of these objects.
Composite, each object can contain multiple basic test sample elements. To
associate a set of attributes and/or annotations as the ground truth to a target test sample element, declare
annotations by extending
DataObject and use the same attribute name as used in the
Composite test sample.
Continue with the example given in
Composite, where the
FacePairSample test sample
type is defined using a pair of images under the
target members, we can design a corresponding ground
truth type with image-level annotations defined in the
from dataclasses import dataclass from kolena.workflow import DataObject, GroundTruth from kolena.workflow.annotation import BoundingBox, Keypoints @dataclass(frozen=True) class FaceRegion(DataObject): bounding_box: BoundingBox keypoints: Keypoints @dataclass(frozen=True) class FacePair(GroundTruth): source: FaceRegion target: FaceRegion is_same_person: bool
This way, it is clear which bounding boxes and keypoints are associated to which image in the test sample.