Where the images and masks are

Description

Two layouts. nested_dirs expects one directory per sample containing an images and a masks subdirectory - the Data Science Bowl arrangement, where a sample may hold dozens of separate mask files that are combined into one. config_file expects a CSV with images and masks columns, several paths per cell if needed, resolved relative to the file itself so a configuration travels with its data.

Usage

segmentation_data(
  train,
  validation = NULL,
  colormap = NULL,
  labels = NULL,
  regions = NULL,
  classes_from = NULL,
  test = NULL,
  split = NULL,
  mode = c("nested_dirs", "config_file"),
  window = NULL,
  dicom_window = NULL,
  target_spacing = NULL,
  channels_from = NULL,
  subdirs = c("images", "masks"),
  column_sep = ";",
  shuffle = TRUE
)

Arguments

train, validation	Paths to the training and validation data. A split_dataset() may be given as train on its own: it carries all three paths and selects config_file mode, so a split needs no unpacking. validation = FALSE says this run has no validation set, which is the third and last answer to the question validation and split both answer - a final fit on every case you have, once the hyperparameters are settled. It goes in this argument rather than a new one because it is the same question, and a second argument could contradict the first. Leaving validation and split both unset is still an error, and that is deliberate: "I have no validation set" and "I forgot" look identical, and only one of them is a decision.
colormap	A list of RGB triples, one per class, background first. For masks stored as pictures. Give this or labels, not both.
labels	The voxel value of each class, in class order, for masks stored as label maps - c(0, 1) for a binary segmentation, c(0, 1, 2) where 1 is liver and 2 is tumour. This is how NIfTI and every other volume format label anything. Give this or colormap, not both. Include 0 unless the masks really do fill every voxel. A value matching no entry falls back to the first class, so a list without 0 trains the background as whatever that first class is. The engine refuses that combination rather than let it look plausible, which means the error arrives before the run and not in the scores.
regions	Overlapping classes, named as unions of a label-map value set: a list of list(name = , of = ), where of holds the voxel values belonging to that region. list(list(name = “whole tumour”, of = c(1, 2, 4)), list(name = “enhancing”, of = 4)) is the BraTS shape, and the second entry shows that a single value is ordinary rather than an edge case. Give this, colormap or labels, not more than one. Two things this field changes that its name does not say. There is no background entry, so n_class here is not the same number it would be for that many labels - with labels the background is a class and with regions it is not. And each channel is an independent binary question, a sigmoid per channel rather than one softmax across them, so a voxel may be in several regions at once - which is the whole point, and the reason save_masks() cannot write such a prediction as one picture. Two regions are the minimum: one region is ordinary binary segmentation, which labels = c(0, 1) already says. Disjoint regions are allowed and mean something - grouping several values into one class and dropping the background channel - so they are not refused.
classes_from	For masks kept as one binary file per class: one pattern per class, in class order. The fourth way of saying what a mask’s classes are, and the only one that can express an overlap that was drawn independently - a vessel crossing an organ, or two annotators’ masks of the same structure. Give this, colormap, labels or regions, not more than one. It is regions’ sibling, not its replacement, and the difference is the file on disk. regions derives overlapping channels from the values of one label map, which is how BraTS stores its ground truth; this reads channels that were separate files to begin with, because no single voxel value can hold two things drawn independently. Everything the two share is everything after the target: no background entry, a sigmoid per channel rather than one softmax across them, a voxel that may be in several classes at once, and a prediction save_masks() cannot write as one picture - use save_class_masks() or save_class_volumes(). Patterns are Python regular expressions matched against a sample’s mask files, with the same two rules channels_from has for images and for the same reason: each must match exactly one file, and every mask file must be claimed by some pattern. Both halves are refused, because each is invisible from the other side - a pattern matching nothing is a class the model is never shown, and a file matching no pattern is a class the data has and nobody asked for. R’s own quoting applies: c(“_vessel\.png$“,”_organ\.png$“). The order is written rather than inferred, and here it matters more than for channels: a model trained with the vessel in the first channel and used where the first channel is the organ answers plausibly, in range, and nothing further down can notice.
test	Optional test data. Only images are read from it.
split	Divide train instead of naming a validation set: a list with fractions (two numbers, or three to cut a test set as well) and group_by. Exactly one of split, validation, and validation = FALSE. group_by has to be given, even as NULL. It is a regular expression read against each sample’s name, and everything sharing a group lands in a single split - usually a patient, sometimes a study or a scanner. NULL divides by file instead, and is a perfectly good answer when the images are independent. It is required rather than optional because the mistake it prevents leaves no trace: slices of one patient in training and validation at once make validation measure memory rather than generalisation, and the score comes out several points too high with nothing in the output to say so. Nothing is written. split_dataset() is still the way when the three CSV files are the point - to keep, to hand to a colleague, to cite - and its result can be passed straight to train.
mode	“nested_dirs” or “config_file”.
window	How values in real units reach the model. A named window - “lung”, “soft_tissue”, “bone”, “brain”, “abdomen”, “liver”, “mediastinum”, “subdural”, “stroke” - or an explicit c(centre, width) in Hounsfield units, or “auto” for the window the file recorded, which only DICOM has, or “full” for the whole range present. Applies to DICOM and to NIfTI; ignored for ordinary pictures, which are already 0-255. Worth choosing rather than leaving: a fixed window is what makes two scans comparable. Scaling each scan by its own darkest and brightest voxel lets a single metal implant or marker rescale everything else in it. NULL, the default, sends nothing and lets the engine decide - which also keeps ordinary image work running against an engine too old to know the setting exists.
dicom_window	The former name of window, still accepted. It was accurate while DICOM was the only format that needed it.
target_spacing	For volumes: resample every scan to this many millimetres per voxel, then centre-crop or pad to the model’s input_shape. Without it a volume is simply resized into that shape, which is only harmless when every scan covers the same amount of the patient - and clinical scans do not. Forty slices of 1 mm is 40 mm of patient and forty of 2.5 mm is 100 mm, so resizing alone leaves the same organ a different size in each, and nothing in the data says so. c(1, 1, 1) is the usual starting point. Ignored for 2D.
channels_from	For datasets that keep one channel per file: one pattern per channel, in channel order. BraTS ships four MRI sequences per patient - T1, T1 after contrast, T2 and FLAIR - because different tumour structures are visible in different sequences; satellite sets keep their bands apart the same way. The order is stated rather than inferred, and that is the point. Sorted, the BraTS names come out flair, t1, t1ce, t2 - reproducible and anatomically meaningless. A model trained with FLAIR in channel one and used on data whose channel one is T1 returns a plausible answer with the right shape and the right range, and nothing further down can notice. Patterns are Python regular expressions, like group_by in split_dataset(), and R’s own quoting applies: c(“_t1\.nii”, “_t1ce\.nii”, “_t2\.nii”, “_flair\.nii”). Each must match exactly one of a sample’s files - “_t1” would match both _t1.nii.gz and _t1ce.nii.gz, which is an error rather than a race - and every file must be claimed by some pattern, because one left out is data the model never sees.
subdirs	For nested_dirs, the names of the image and mask subdirectories.
column_sep	For config_file, what separates several paths in one cell.
shuffle	Shuffle the training data between epochs.

Details

Classes are named one of two ways, and exactly one of them. colormap is for masks stored as pictures: a list of RGB triples where position is the class index. labels is for masks stored as label maps - a vector of the voxel values, in class order - which is how every volume format stores them, and how a single-channel PNG can be read as well.

A sample that is missing its masks is an error, not a warning - a warning in a loop over five hundred directories is a warning nobody reads.

Value

A data specification, to be passed to platypus_spec().

Examples

library("rplatypus")

segmentation_data("train/", "valid/", colormap = binary_colormap)

$train_path
[1] "train/"

$validation_path
[1] "valid/"

$validation
NULL

$test_path
NULL

$split
NULL

$mode
NULL

$colormap
$colormap[[1]]
[1] 0 0 0

$colormap[[2]]
[1] 255 255 255


$labels
NULL

$regions
NULL

$window
NULL

$target_spacing
NULL

$channels_from
NULL

$classes_from
NULL

$subdirs
NULL

$column_sep
NULL

$shuffle
NULL