What is Identity F1 Score?
Identity F1 Score is the harmonic mean of identification precision and identification recall after a sequence-level one-to-one assignment between ground-truth and predicted identities.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
Find one global identity assignment for the sequence
After frame-level eligibility matching establishes which detections can correspond, an overlap matrix counts compatible detections for every truth-identity and predicted-identity pair. A bipartite assignment maximizes the number of correctly identified detections while allowing unmatched identities through dummy assignments.
The original identification-metrics paper introduced this sequence-level view for multi-target, multi-camera tracking. Detections covered by selected identity pairs are IDTP; remaining computed detections are IDFP, and remaining ground-truth detections are IDFN. The Go example enumerates that assignment for a small matrix.
Interpret ID precision, recall, and their harmonic mean
IDP = IDTP / (IDTP + IDFP) asks what fraction of computed detections has the correct identity. IDR = IDTP / (IDTP + IDFN) asks what fraction of ground-truth detections is correctly identified. IDF1 = 2 * IDTP / (2 * IDTP + IDFP + IDFN) balances the two.
A long correct fragment can receive more credit than several short fragments, even if the latter briefly recover the right object more often. Because one global mapping is used, IDF1 emphasizes strict sequence-wide identity consistency and can hide the timing or number of individual switches.
Report detection and localization evidence alongside IDF1
TrackEval implements IDF1, IDP, and IDR alongside HOTA and CLEAR MOT for widely used tracking benchmarks. Its preprocessing and matching logic are part of the metric contract, not incidental code. MOTA counts frame-level errors, while Track mAP evaluates confidence-ranked trajectory discovery.
IDF1 can improve when association improves, but it can also change with detection thresholds, ignored regions, class filters, and localization eligibility. ATA gives tracks equal weight, and LIDF1 and ALTA expose the horizon at which identity fails. HOTA gives balanced detection/association diagnostics, while OSPA(2) and T-GOSPA are trajectory-set distances.
Key Characteristics
- Uses one sequence-level one-to-one identity assignment
- Counts correctly identified detections as IDTP
- Balances identity precision and identity recall
- Works for single-camera and multi-camera identity evaluation
- Emphasizes global identity consistency over switch timing
- Depends on detection eligibility and benchmark preprocessing
Common Use Cases
- Evaluating identity preservation in multi-object tracking
- Comparing multi-camera person or vehicle tracking systems
- Measuring the effect of re-identification and data association
- Diagnosing track fragmentation and identity merge behavior
- Reporting MOTChallenge Identity metrics with HOTA and MOTA
Example
Loading code...Frequently Asked Questions
How is IDF1 calculated?
After a sequence-level one-to-one assignment between truth and predicted identities, IDF1 is `2*IDTP / (2*IDTP + IDFP + IDFN)`. It is equivalently the harmonic mean of identification precision and identification recall.
What are IDTP, IDFP, and IDFN?
IDTP counts detections covered by selected correct identity pairs. IDFP counts computed detections not correctly identified under that mapping, and IDFN counts ground-truth detections not correctly identified.
How is IDF1 different from counting ID switches?
An ID-switch count records discrete changes under a frame-to-frame matching convention. IDF1 instead optimizes one global identity mapping and measures how many detections are correctly identified, so it emphasizes duration rather than the number or timing of switch events.
How is IDF1 different from HOTA?
IDF1 uses a strict global track assignment and focuses on identification precision and recall. HOTA gives partial association credit for each matched detection and combines Association Accuracy with Detection Accuracy across localization thresholds.
Can a high IDF1 prove that localization is accurate?
No. IDF1 relies on a localization threshold to determine eligible matches but does not report the quality of localization among accepted pairs. Report localization metrics, HOTA components, detection errors, and task-specific tolerances separately.