DetectionEngine
DetectionEngine(
spec,
*,
device=None,
num_workers=0,
strict_data=True,
accumulate=1,
)Methods
| Name | Description |
|---|---|
| anchor_coverage | How well the anchors in use cover this split’s boxes. |
| best_model | Which model scored highest on one column. |
| box_shapes | The cloud of box shapes in a split, with the anchors in use beside it. |
| crops | Every detection cut out of the image it was found in. |
| dataset | The dataset for one split: images letterboxed to input_shape, boxes encoded. |
| evaluate | One row per model: the comparison table, with detection’s columns. |
| evaluate_classes | One row per class instead of one row per model. |
| evaluate_images | One row per image instead of one row per model or per class. |
| export_weights | Write one detector’s weights, with the anchors in the sidecar. |
| fit | Train every model in the specification, in turn, against the same data. |
| fit_anchors | k-means with an IoU distance over the training boxes, or nothing to do. |
| loader | A torch DataLoader over one split, wrapping dataset. |
| model_names | The models that have been fitted or loaded, in the order they were. |
| predict | Boxes, scores and labels for every image in a split, in its own pixels. |
| report | Everything a detection table is read off, from one pass over the split. |
| split_sizes | How many images each split holds, and in which order they were found. |
anchor_coverage
DetectionEngine.anchor_coverage(model_name=None, split='train')How well the anchors in use cover this split’s boxes.
Worth asking of a split the anchors were not fitted on: anchors that cover the training boxes at 0.92 and the validation boxes at 0.70 say the two splits hold different objects, which no training curve shows.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | None | Which detector. Defaults to the first. | None |
| split | str | Which split’s boxes to measure against. | 'train' |
Returns
| Name | Type | Description |
|---|---|---|
| dict[str, Any] | The mean IoU between each box and its best anchor. Measured on a split the | |
| dict[str, Any] | anchors were not fitted on, this says whether they generalise; it was 0.877 | |
| dict[str, Any] | in every one of six BCCD runs, which is also why GIoU could not help there - | |
| dict[str, Any] | a prediction never starts disjoint from the truth it answers. |
best_model
DetectionEngine.best_model(key='map_50', split='validation')Which model scored highest on one column.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| key | str | The column to rank on. map_50 by default - but on a small dataset read mean_matched_iou first: measured over five seeds on BCCD, average precision has a standard deviation of 0.0159 while matched overlap has 0.0026, six times steadier. How well the boxes fit is reproducible there and how many are found is not, because a borderline cell changing rank moves average precision and leaves the overlap alone. |
'map_50' |
| split | str | Which data to rank on. | 'validation' |
Returns
| Name | Type | Description |
|---|---|---|
| str | The winning model’s name. |
Raises
| Name | Type | Description |
|---|---|---|
| EngineError | If no such column exists - the message lists the ones that do - or if the column is undefined for any model, which happens when a class has no truth boxes in that split and so has no average precision to average. |
box_shapes
DetectionEngine.box_shapes(model_name=None, split='train')The cloud of box shapes in a split, with the anchors in use beside it.
Everything a picture of the anchor fit needs, in one call and in one set of coordinates: both are fractions of the model’s input, computed by the same function that fitted the anchors. The alternative - widths and heights from one place and anchors from another - is how a plot comes to show boxes in different places from where the anchors were fitted, which looks like a bad fit and is a bug.
Worth drawing for a split the anchors were not fitted on, for the same reason as anchor_coverage: it is the only way to see a class the anchors have nothing near.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | None | Which detector. Defaults to the first. | None |
| split | str | Which split’s boxes to return. | 'train' |
Returns
| Name | Type | Description |
|---|---|---|
| dict[str, Any] | Every box’s width and height, normalised by the letterboxed input rather | |
| dict[str, Any] | than by the source image - which is the convention the anchors use, and getting | |
| dict[str, Any] | it wrong is how a coverage figure of 0.67 was quoted for two years against a | |
| dict[str, Any] | true 0.65. |
crops
DetectionEngine.crops(
model_name=None,
split='test',
*,
score_threshold=None,
context=0.0,
size=None,
fit='letterbox',
fill=0.5,
)Every detection cut out of the image it was found in.
The pipeline this is for is a detector followed by a classifier it does not contain: find the objects with three classes, crop them, hand the crops to something that knows eighty. It is the second thing people ask for after boxes, and doing it by hand means re-reading every image and getting the rounding right.
Filtered at operating_point, not at score_threshold. The specification’s score_threshold is deliberately near zero so that average precision integrates the whole ranking - on BCCD that is hundreds of boxes an image, almost all of them the tail that AP exists to measure over. Cropping them would hand a classifier mostly noise. operating_point is where someone stands when they act on a prediction, which is what cropping is.
One record per image rather than one flat list, because a crop without its source is not traceable: key says which image, and boxes are the coordinates it came from, in that image’s own pixels.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | None | which trained detector. | None |
| split | str | which split’s images to read. | 'test' |
| score_threshold | float | None | override the confidence to crop at. | None |
| context | float | expand each box by this fraction of its size per side, 0 by default. | 0.0 |
| size | tuple[int, int] | None | bring every crop to (height, width), or None to keep each as cut. |
None |
| fit | str | "letterbox" to preserve aspect, "stretch" to resize both axes. |
'letterbox' |
| fill | float | padding value for "letterbox". |
0.5 |
Returns
| Name | Type | Description |
|---|---|---|
| list[dict[str, Any]] | One dict per image with key, crops, boxes, scores, labels, names. |
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| split | str | Which data to cut from. | 'test' |
| model_name | str | None | Which detector. Defaults to the first. | None |
| context | float | Extra margin, as a fraction of the box rather than in pixels, so one value suits a platelet and a white cell. | 0.0 |
| size | tuple[int, int] | None | Resize every crop to this. Letterboxed rather than stretched, because a stretch changes the aspect of every non-square object and has to be asked for. | None |
Returns
| Name | Type | Description |
|---|---|---|
| list[dict[str, Any]] | One record per image, each with the crops and the boxes they came from, plus | |
| list[dict[str, Any]] | dropped - boxes the inverse letterbox mapped to no extent at all, which a |
|
| list[dict[str, Any]] | model may legitimately produce by saying an object is off the frame. |
Read at operating_point, not at score_threshold: that one sits near zero so average precision can integrate the whole ranking, and cropping it would hand a classifier the tail.
dataset
DetectionEngine.dataset(
model,
split,
*,
anchors=None,
only_images=False,
augmented=False,
)The dataset for one split: images letterboxed to input_shape, boxes encoded.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model | DetectionModel | Which model’s settings to read. | required |
| split | str | "train", "validation" or "test". |
required |
| anchors | Anchors to encode the targets against. Unset means the model’s own - either the ones it declares or the ones fitted for it, and the encoding is meaningless against any others. | None |
|
| only_images | bool | Read images alone, with no boxes. What a split with no annotations gets, so predict works on it while evaluate refuses it by name. |
False |
| augmented | bool | Apply the model’s augmentation. Every geometric transform in albumentations moves the boxes with the pixels, measured over 24 configurations, worst disagreement one pixel from anti-aliasing. | False |
Returns
| Name | Type | Description |
|---|---|---|
| DetectionDataset | A DetectionDataset, whose source_image(index) also reads an image at its |
|
| DetectionDataset | native size through the same reader - which is what crops cuts from, since |
|
| DetectionDataset | cropping the letterboxed frame would cut a downscaled object. |
evaluate
DetectionEngine.evaluate(split='validation')One row per model: the comparison table, with detection’s columns.
Different columns from the segmentation table, because they answer a different question. mean_matched_iou is the one worth keeping beside the averages: the gap between mAP@0.5 and mAP@[.50:.95] is localisation, and this is that gap as a single number - how well the boxes that matched actually fit, rather than merely that they cleared a threshold.
No precision or recall here, although operating_point says where to read them. Averaging them over classes needs a weighting and every choice of weighting is a different claim: summed over BCCD’s 4155 red cells, 372 white and 361 platelets, a single precision is a statement about red cells wearing the costume of a statement about the model. They are in evaluate_classes, per class, which is the only form in which they mean anything. DetectionMetrics.as_rows made the same decision - its “all” row leaves both as None.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| split | str | Which data to score on. | 'validation' |
Returns
| Name | Type | Description |
|---|---|---|
| list[dict[str, Any]] | One row per model: map_50, map_50_95, mean_matched_iou and the counts. |
|
| list[dict[str, Any]] | No overall precision or recall, deliberately: averaging them over classes | |
| list[dict[str, Any]] | needs a weighting, and over BCCD’s 4,155 red cells against 372 white one | |
| list[dict[str, Any]] | precision is a statement about red cells wearing the costume of a statement | |
| list[dict[str, Any]] | about the model. |
evaluate_classes
DetectionEngine.evaluate_classes(model_name=None, split='validation')One row per class instead of one row per model.
The row that matters on an unbalanced dataset, which is most of them: BCCD has 4155 red cells against 372 white and 361 platelets, so a single number is a number about red cells.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | None | Which detector. Defaults to the first. | None |
| split | str | Which data to score on. | 'validation' |
Returns
| Name | Type | Description |
|---|---|---|
| list[dict[str, Any]] | One row per class: average precision at IoU 0.5, the mean overlap of the boxes | |
| list[dict[str, Any]] | that matched, how many truth boxes there were, and precision and recall at the | |
| list[dict[str, Any]] | model’s operating_point. The row that matters on unbalanced data, which is |
|
| list[dict[str, Any]] | most detection data. |
evaluate_images
DetectionEngine.evaluate_images(
model_name=None,
split='validation',
score_threshold=None,
)One row per image instead of one row per model or per class.
The question a table of averages cannot answer: which images it fails on. The counterpart of the segmentation engine’s per-case scores, and the second question anyone asks after seeing a mean.
No average precision per image. AP is the area under a precision-recall curve and therefore a property of a ranking over a dataset; on one image with three boxes it swings on a single box’s rank and says nothing. The columns are counts and overlap, which do mean something for one picture.
Counts are read at the specification’s operating_point unless another score_threshold is given, because “how many were missed” is undefined over the whole ranking - at a threshold of zero every box the model dimly considered is a prediction, and spurious would count the tail that average precision exists to integrate over rather than anything a user would see.
Sort by missed to find the frames it cannot see, or by mean_matched_iou to find the ones where it sees everything and places it badly. Those are different problems: the first is usually the data, the second is usually the anchors.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | None | Which detector. Defaults to the first. | None |
| split | str | Which data to score on. | 'validation' |
Returns
| Name | Type | Description |
|---|---|---|
| list[dict[str, Any]] | One row per image: key, n_truth, n_predicted, matched, missed, |
|
| list[dict[str, Any]] | spurious and mean_matched_iou. Order by missed for the frames it cannot |
|
| list[dict[str, Any]] | see and by mean_matched_iou for the ones where it finds everything and places |
|
| list[dict[str, Any]] | it badly - usually the data and the anchors respectively. |
mean_matched_iou is None when nothing matched, not 0: found nothing and found badly are different failures, and that distinction crosses the bridge into R, so a test pins it rather than trusting it.
There is no average precision per image, deliberately. It is the area under a precision-recall curve, so it is a property of a ranking over a dataset; on one picture with three boxes it moves on a single box’s rank.
export_weights
DetectionEngine.export_weights(model_name, path, **extra)Write one detector’s weights, with the anchors in the sidecar.
The anchors are not optional metadata. The same weights read with different anchors decode every box scaled by a fixed factor, and nothing about the output says so - the boxes are plausible, the scores are plausible, and they are in the wrong places.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | Which detector’s weights to write. | required |
| path | str | Path | Where to write. The sidecar goes beside it. | required |
| **extra | Anything else to record. | {} |
Returns
| Name | Type | Description |
|---|---|---|
| Path | The path written. |
fit
DetectionEngine.fit(verbose=False)Train every model in the specification, in turn, against the same data.
Anchors are fitted first for any model that does not declare them, because the targets cannot be encoded without them. A model naming weights adopts the architecture, the class names and the anchors from the file: weights mean nothing without the anchors they were trained with, so a specification naming both is refused rather than warned about.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| verbose | bool | Print each model’s name and per-epoch numbers as they arrive. The learning rate is printed with enough digits to see a schedule moving - at four decimal places a cosine decaying to 1e-8 read learning_rate=0.0000 from epoch 90 and a scheduled run looked identical to an unscheduled one. |
False |
Returns
| Name | Type | Description |
|---|---|---|
| dict[str, History] | One History per model, by name, and the same objects are kept on the engine |
|
| dict[str, History] | so evaluate, predict and export_weights can be called afterwards. |
Raises
| Name | Type | Description |
|---|---|---|
| EngineError | If a split a model needs is missing or carries no annotations. |
fit_anchors
DetectionEngine.fit_anchors(model)k-means with an IoU distance over the training boxes, or nothing to do.
Euclidean distance on (width, height) is scale-blind - (0.02, 0.02) and (0.04, 0.04) sit the same distance apart as (0.50, 0.50) and (0.52, 0.52), while their overlaps are 0.25 and 0.93 - so the distance is 1 - IoU, which is what the anchors are for.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model | DetectionModel | Which model’s boxes to fit against. | required |
Returns
| Name | Type | Description |
|---|---|---|
| AnchorFit | None | An AnchorFit, or None when the model already declares anchors - the |
|
| AnchorFit | None | difference between “fitted these” and “nothing to do”. |
Fitting is worth about 0.23 of mean overlap on BCCD: COCO’s nine anchors cover its boxes at 0.6473, fitted ones at 0.8751 to 0.8802 across three seeds.
loader
DetectionEngine.loader(
model,
split,
*,
anchors=None,
shuffle=False,
augmented=False,
)A torch DataLoader over one split, wrapping dataset.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model | DetectionModel | Which model’s settings to read. | required |
| split | str | "train", "validation" or "test". |
required |
| anchors | As for dataset. |
None |
|
| shuffle | bool | Shuffle between epochs. Training only. | False |
| augmented | bool | Apply the model’s augmentation. Training only. | False |
Returns
| Name | Type | Description |
|---|---|---|
A torch.utils.data.DataLoader yielding images and encoded targets. |
model_names
DetectionEngine.model_names()The models that have been fitted or loaded, in the order they were.
Returns
| Name | Type | Description |
|---|---|---|
| list[str] | Their names - the keys every other method takes as model_name. Empty before |
|
| list[str] | fit has run, which is the difference between a specification and a run. |
predict
DetectionEngine.predict(model_name=None, split='test')Boxes, scores and labels for every image in a split, in its own pixels.
A list and not an array, and in source pixels rather than the network’s frame - both for the same reason the segmentation side returns a list for space="source". Images differ in size, the number of boxes found differs per image, and a box in a letterboxed 416x416 frame cannot be drawn on the photograph it came from without undoing the letterbox, which is a step nobody should have to remember.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| split | str | Which data to predict on. | 'test' |
| model_name | str | None | Which detector. Defaults to the first. | None |
Returns
| Name | Type | Description |
|---|---|---|
| list[dict[str, Any]] | A list, one entry per image, each holding boxes in that image’s own pixels | |
| list[dict[str, Any]] | with the inverse letterbox already applied. No array form: images differ in | |
| list[dict[str, Any]] | size, so does the number of boxes, and a box in a letterboxed frame cannot be | |
| list[dict[str, Any]] | drawn on the photograph it came from. |
report
DetectionEngine.report(model_name=None, split='validation')Everything a detection table is read off, from one pass over the split.
Public because evaluate and evaluate_classes are both views over it, and a caller that wants both - which the example does - would otherwise run the model over the split twice. No caching: a cache would be stale the moment the model trained another epoch, and the honest alternative is to hand back the thing and let the caller hold it.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| model_name | str | None | Which detector. Defaults to the first. | None |
Returns
| Name | Type | Description |
|---|---|---|
| DetectionReport | A DetectionReport: the anchors, their coverage, the target survey - how many |
|
| DetectionReport | boxes were placed, how many could not be, how many were dropped - and the split | |
| DetectionReport | sizes. What to read before trusting a score, because a box that cannot be | |
| DetectionReport | placed is a box the model is never asked to find. |
BCCD carries one annotation of zero extent in each split, which comes out one pixel across under Pascal VOC’s inclusive convention, so dropped is 1 and has been all along.
split_sizes
DetectionEngine.split_sizes()How many images each split holds, and in which order they were found.
Returns
| Name | Type | Description |
|---|---|---|
| dict[str, int] | How many samples each split holds, and whether each carries annotations - which | |
| dict[str, int] | is what decides whether it can be scored at all. A split of images alone is | |
| dict[str, int] | accepted and predict works on it; evaluate refuses it by name. |