DetectionEngine.evaluate_images

DetectionEngine.evaluate_images(
    model_name=None,
    split='validation',
    score_threshold=None,
)

One row per image instead of one row per model or per class.

The question a table of averages cannot answer: which images it fails on. The counterpart of the segmentation engine’s per-case scores, and the second question anyone asks after seeing a mean.

No average precision per image. AP is the area under a precision-recall curve and therefore a property of a ranking over a dataset; on one image with three boxes it swings on a single box’s rank and says nothing. The columns are counts and overlap, which do mean something for one picture.

Counts are read at the specification’s operating_point unless another score_threshold is given, because “how many were missed” is undefined over the whole ranking - at a threshold of zero every box the model dimly considered is a prediction, and spurious would count the tail that average precision exists to integrate over rather than anything a user would see.

Sort by missed to find the frames it cannot see, or by mean_matched_iou to find the ones where it sees everything and places it badly. Those are different problems: the first is usually the data, the second is usually the anchors.

Parameters

Name	Type	Description	Default
model_name	str | None	Which detector. Defaults to the first.	None
split	str	Which data to score on.	'validation'

Returns

Name	Type	Description
	list[dict[str, Any]]	One row per image: key, n_truth, n_predicted, matched, missed,
	list[dict[str, Any]]	spurious and mean_matched_iou. Order by missed for the frames it cannot
	list[dict[str, Any]]	see and by mean_matched_iou for the ones where it finds everything and places
	list[dict[str, Any]]	it badly - usually the data and the anchors respectively.

mean_matched_iou is None when nothing matched, not 0: found nothing and found badly are different failures, and that distinction crosses the bridge into R, so a test pins it rather than trusting it.

There is no average precision per image, deliberately. It is the area under a precision-recall curve, so it is a property of a ranking over a dataset; on one picture with three boxes it moves on a single box’s rank.