What is Panoptic Tracking (PAT)?
Panoptic Tracking (PAT) is an instance-centric panoptic tracking metric that combines frame-level Panoptic Quality with Tracking Quality through a harmonic mean.
Quick Facts
| Full Name | Panoptic Tracking Metric |
|---|---|
| Specification | Official Specification |
How It Works
Compute frame-level PQ over thing and stuff classes
The PQ component matches same-class predicted and Ground Truth segments above the protocol's IoU threshold. For each class, PQ = sum IoU / (TP + 0.5*FP + 0.5*FN), then the evaluator averages class scores. Thing instances and Stuff regions both influence this component.
The Panoptic nuScenes paper combines this segmentation view with a separate instance-centric tracking term. A good TQ cannot compensate for a semantic or mask failure without limit because the final harmonic mean is pulled toward the weaker component.
Build TQ from association and identity transitions
For Ground Truth track g of length |g| and each predicted identity p, let TPA be accepted frame matches, FNA = |g| - TPA, and FPA be occurrences of p outside matches to g. AQ_g = sum_p TPA^2 / (TPA + FNA + FPA) / |g|. Let IS_g = 1 - IDS_g / (|g| - 1), with IS_g = 1 for a one-frame track.
The track score is sqrt(AQ_g * IS_g), and TQ averages it equally over Ground Truth thing instances. Fragmenting one truth identity across predictions lowers AQ; changed or missing adjacent assignments lower the transition term under the reference evaluator.
Combine components and preserve evaluator semantics
PAT = 2 * PQ * TQ / (PQ + TQ), bounded by zero and one. PQ is class-macro averaged while TQ is Ground-Truth-instance-macro averaged, so neither point count nor frame count directly determines top-level weight. Report PQ and TQ beside PAT rather than presenting the harmonic mean alone.
The official nuScenes implementation uses class-agnostic prediction IDs for TQ matching, an IoU threshold, minimum-point filtering, scene-bounded identity histories, and a designated unmatched value in transition counting. Pin those choices. Use LSTQ for point-centric sequence association and sPTQ for switch-IoU-adjusted frame-level PQ.
Key Characteristics
- Balances panoptic segmentation and tracking with a harmonic mean
- Uses standard frame-level PQ as its segmentation component
- Scores association and identity transitions per ground-truth track
- Averages TQ equally over thing instances
- Penalizes fragmentation, ID transfer, and missed continuity
- Depends on IoU, minimum-point, class, and scene-boundary rules
Common Use Cases
- Ranking Panoptic nuScenes tracking submissions
- Evaluating LiDAR panoptic segmentation and tracking jointly
- Separating frame segmentation from instance continuity
- Diagnosing fragmentation and ID-transfer behavior
- Comparing instance-centric panoptic tracking systems
Example
Loading code...Frequently Asked Questions
How is PAT calculated?
PAT is the harmonic mean of frame-level Panoptic Quality and instance-centric Tracking Quality. TQ averages each ground-truth thing track's geometric mean of association quality and identity-transition consistency.
What does Tracking Quality in PAT measure?
TQ measures how completely and exclusively each ground-truth track maps to predicted identities, then combines that association score with a normalized identity-transition score. Each ground-truth thing instance has equal top-level weight.
How is PAT different from LSTQ?
PAT combines class-averaged frame PQ with instance-averaged TQ and uses thresholded segment matches. LSTQ combines semantic mIoU with point-centric full-sequence Association Quality and does not first threshold whole segments for association credit.
How is PAT different from sPTQ?
sPTQ modifies the frame-level PQ numerator by removing switched-match IoU. PAT constructs a separate instance-centric TQ from full track associations and identity transitions, then harmonically balances TQ with ordinary PQ.
What must be fixed before comparing PAT scores?
Fix the dataset split, class mapping, Thing/Stuff definitions, segment-IoU threshold, minimum points, ignored labels, predicted-ID namespace, sequence boundaries, unmatched transition handling, and evaluator commit.