vis4d.eval.shift.multitask_writer

SHIFT result writer.

Classes

SHIFTMultitaskWriter(output_dir[, ...])

SHIFT result writer for online evaluation.
class SHIFTMultitaskWriter(output_dir, submission_file='submission.zip')[source]

SHIFT result writer for online evaluation.

__init__(output_dir, submission_file='submission.zip')[source]

Creates a new writer.

Parameters:

  • output_dir (str) – Output directory.

  • submission_file (str) – Submission file name. Defaults to “submission.zip”.

process_batch(frame_ids, sample_names, sequence_names, pred_sem_mask=None, pred_depth=None, pred_flow=None, pred_boxes2d=None, pred_boxes2d_classes=None, pred_boxes2d_scores=None, pred_boxes2d_track_ids=None, pred_instance_masks=None)[source]

Process SHIFT results.

You can omit some of the predictions if they are not used.

Parameters:

  • frame_ids (list[int]) – Frame IDs.

  • sample_names (list[str]) – Sample names.

  • sequence_names (list[str]) – Sequence names.

  • pred_sem_mask (list[ArrayLike], optional) – Predicted semantic masks, each in shape (C, H, W) or (H, W). Defaults to None.

  • pred_depth (list[ArrayLike], optional) – Predicted depth maps, each in shape (H, W), with meter unit. Defaults to None.

  • pred_flow (list[ArrayLike], optional) – Predicted optical flows, each in shape (H, W, 2). Defaults to None.

  • pred_boxes2d (list[ArrayLike], optional) – Predicted 2D boxes, each in shape (N, 4). Defaults to None.

  • pred_boxes2d_classes (list[ArrayLike], optional) – Predicted 2D box classes, each in shape (N,). Defaults to None.

  • pred_boxes2d_scores (list[ArrayLike], optional) – Predicted 2D box scores, each in shape (N,). Defaults to None.

  • pred_boxes2d_track_ids (list[ArrayLike], optional) – Predicted 2D box track IDs, each in shape (N,). Defaults to None.

  • pred_instance_masks (list[ArrayLike], optional) – Predicted instance masks, each in shape (N, H, W). Defaults to None.

Return type:

None

gather(gather_func)[source]

Gather variables in case of distributed setting (if needed).

Parameters:

gather_func (Callable[[Any], Any]) – Gather function.

Return type:

None

evaluate(metric)[source]

No evaluation locally.

Return type:

tuple[Dict[str, Union[float, int, Tensor]], str]

save(metric, output_dir)[source]

Save scalabel output to zip file.

Raises:

ValueError – If the number of samples in each category is not the same.

Return type:

None