Loss functions

Description

The objective the model is trained against. Every one of these works unchanged on volumes as well as images.

Usage

loss_iou(smooth = 1)

loss_dice(smooth = 1)

loss_generalized_dice(smooth = 1)

loss_cce(label_smoothing = 0)

loss_cce_dice(cce_weight = 0.5, smooth = 1)

loss_focal(gamma = 2, alpha = NULL)

loss_tversky(alpha = 0.5, smooth = 1)

loss_focal_tversky(alpha = 0.5, gamma = 1, smooth = 1)

loss_combo(alpha = 0.5, ce_ratio = 0.5)

loss_mse(where = NULL)

loss_l1(where = NULL)

loss_smooth_l1(beta = NULL, where = NULL)

loss_lovasz(per_image = FALSE)

Arguments

smooth Added to numerator and denominator to keep an empty class finite.
label_smoothing For cross-entropy, train towards 1 - label_smoothing instead of towards 1, so the model is not pushed to be certain. Worth having where the labels themselves are uncertain, which on a hand-drawn boundary they are.
cce_weight For loss_cce_dice(), how much of the total the cross-entropy half provides. At 0 or 1 it is one of the two alone, which is worth knowing before tuning it as though it were a third thing.
gamma For focal losses, how hard to discount the pixels already classified well.
alpha For Tversky, the weight on false negatives; raising it buys recall. Note that alpha = 0.5 is Dice exactly only when smooth = 0.
ce_ratio For loss_combo(), how its cross-entropy half is tilted between punishing missed foreground (towards 1) and false alarms (towards 0), while alpha sets how much of the total each half provides.
where For the three regression losses, where the target means anything and so where the error is averaged. A flow field is defined on the objects and undefined off them - there is no direction towards a centre for a pixel in no object - so the builder writes zero on the background, and “everywhere” would spend most of the objective learning that zero. The foreground is 13.97% of a Data Science Bowl image, so “everywhere” gives 86% of the gradient to a constant; “foreground” is the default for that reason rather than by taste.
beta For loss_smooth_l1(), where the quadratic part ends and the linear one begins, in the target’s own units. A flow field is scaled to roughly [-1, 1], so the default of 1 makes the whole range quadratic - which is loss_mse() with extra steps. Lower it to get any of the robustness this loss is for.
per_image For loss_lovasz(), sort the errors of each image separately rather than over the whole batch. The harsher reading, and on mostly empty images the noisier one.

Details

There is one documented separately because it needs a page of its own: loss_boundary() adds the signed distance to the truth’s boundary on top of any of these, which removes the systematic volume bias at some cost in per-case accuracy.

Three of these score numbers rather than classes - loss_mse(), loss_l1() and loss_smooth_l1() - and they are not alternatives to the others. A class target has nothing for them to measure and a continuous one has nothing for an overlap to measure, so they belong in exactly one place: the loss of separation_flows(), where the target is a vector field. Measured against the pinned engine rather than assumed: giving one as any segmentation model’s own loss is refused by name - semantic or instance - and giving a class loss to separation_flows() is refused by name the other way round.

One of these weights the classes rather than treating them alike - loss_generalized_dice(), by the inverse square of how much of the picture each class occupies - and the argument usually made for it is not the argument for it here. It is introduced in the literature as a fix for Dice being dominated by the background, and loss_dice() in this package is a mean over the class channels: a lesion in 1% of the frame already carries the same weight as the 99% around it. Measured on a 1% target, with one prediction that finds the lesion and leaks a false positive of equal size and one that misses it and gets the background perfect:

loss finds it misses it prefers finding by
loss_dice() 0.1658 0.4731 0.3073
loss_cce() 0.1600 0.1600 nothing at all
loss_cce_dice() 0.1629 0.3166 0.1536
loss_generalized_dice() 0.3278 0.9511 0.6233

So what it adds is weight beyond an equal share - on that target the classes come out weighted 1 to 9801 - and the objective that is genuinely dominated is the per-pixel half of loss_cce_dice(): to cross-entropy a false positive and a missed lesion of the same size are the same number of wrong pixels, which is the indifference in that row. Two things follow. With every class the same size this is exactly loss_dice() pooled over the classes, so on a balanced problem it costs nothing and buys nothing. And it is not a third loss_tversky(): Tversky moves the balance between false positives and false negatives by a number you choose, loss_focal_tversky() reweights the hard pixels, and this reweights classes by how much of the image they occupy, computed from each batch rather than set by hand - so a batch holding no lesion at all is scored on flat weights, which is an argument for a larger batch_size on a sparse dataset rather than against the loss.

This page said an instance model’s own until the pin moved to 0.8.0a22, and it was true rather than wrong: the refusal lived on the instance class alone, and a semantic model accepted loss_mse() and trained with it, reporting a validation loss of 0.0014 for a mask that was half missing where loss_cce() reaches a Dice of 1.0000. The default where = “foreground” is what made it look as though it had worked - it evaluates the objective only inside the labelled objects and leaves the background unconstrained. The engine refuses it at either task now (pyplatypus#223).

Value

A loss specification, to be passed to a model constructor.

See Also

separation_flows(), whose loss is the one place the regression losses belong

Examples

$name
[1] "focal"

$gamma
[1] 2

$alpha
NULL
loss_tversky(alpha = 0.7)
$name
[1] "tversky"

$alpha
[1] 0.7

$smooth
[1] 1
loss_smooth_l1(beta = 0.1, where = "foreground")
$name
[1] "smooth_l1"

$beta
[1] 0.1

$where
[1] "foreground"