Anomaly segmentation is a task that aims to find objects that are present only at inference time and unknown during training. A typical anomaly is a deer or a cardboard box in the middle of the road. Current benchmarks use semantic segmentation to evaluate the performance of anomaly segmentation methods. However, this approach is not sufficient for complex driving cases with multiple anomalies. Semantic segmentation does not provide enough information for downstream tasks such as planning or object tracking. The more challenging problem of instance segmentation remains under-researched and lacks accessible benchmarks. This benchmark addresses the lack of test evaluation protocols available to the community. In the benchmark, we extend the labels of well-known benchmarks such as SegmentMeIfYouCan and FishyScapes Lost and Found to instance segmentation. We combine two benchmarks into a unified benchmark and evaluate the most common metrics instance metrics of Average Precision.
The OoDIS bencmark is online at Codalab
AP (Average Precision) measures the average precision values across multiple Intersection Over Union (IoU) thresholds. It is a popular metric in object detection and segmentation tasks for evaluating the precision of predictions.
AP50 refers to the Average Precision at 50% IoU, providing insight into how well the model can detect objects with a moderate overlap requirement.