1. Haptera
- F1
- 40.58%
- Precision
- 33.94%
- Recall
- 50.44%
- Median timing error, ms
- 80.39
Onset Benchmark
How reliably can a system find physical events using only standard video?
Maintained by Haptera
−280 ms
Approach−120 ms
Visible touch0 ms
Loading onsetLoading onset
Find the start of a force change across 83 REASSEMBLE recordings and 3,898 sensor-timed events. Predictions within 200 ms of the sensor time count as matches.
Swipe to see all columns.
| Rank / Model | F1 | Precision | Recall | Median timing error, ms |
|---|---|---|---|---|
| 1. Haptera | 40.58% | 33.94% | 50.44% | 80.39 |
| 2. Gemini 3.1 Pro Preview | 21.30% | 15.36% | 34.71% | 56.97 |
| 3. Open V-JEPA2 | 12.09% | 42.24% | 7.05% | 87.31 |
| 4. Open DINOv2 | 4.15% | 23.00% | 2.28% | 119.98 |
2. Gemini 3.1 Pro Preview
- F1
- 21.30%
- Precision
- 15.36%
- Recall
- 34.71%
- Median timing error, ms
- 56.97
3. Open V-JEPA2
- F1
- 12.09%
- Precision
- 42.24%
- Recall
- 7.05%
- Median timing error, ms
- 87.31
4. Open DINOv2
- F1
- 4.15%
- Precision
- 23.00%
- Recall
- 2.28%
- Median timing error, ms
- 119.98
Contact state
Identify contact that produces force in each video window. The same 83 REASSEMBLE recordings.
Swipe to see all columns.
| Model | Contact precision | No-contact precision | Contact recall | Coverage |
|---|---|---|---|---|
| Haptera | 92.74% | 99.72% | 51.82% | 76.25% |
| Gemini 3.1 Pro Preview | 24.62% | 79.55% | 22.93% | 99.88% |
Haptera
- Contact precision
- 92.74%
- No-contact precision
- 99.72%
- Contact recall
- 51.82%
- Coverage
- 76.25%
Gemini 3.1 Pro Preview
- Contact precision
- 24.62%
- No-contact precision
- 79.55%
- Contact recall
- 22.93%
- Coverage
- 99.88%
RH20T timing
Find sensor changes from video across three robot setups. F1 at ±200 ms, averaged equally across setups.
Swipe to see all columns.
| Model | F1 (mean) | Setup 2 | Setup 6 | Setup 7 |
|---|---|---|---|---|
| Haptera | 2.26% | 1.12% | 3.24% | 2.42% |
| Open DINOv2 | 0.85% | 0.67% | 0.90% | 0.98% |
| Open V-JEPA2 | 0.27% | 0.20% | 0.23% | 0.39% |
Haptera
- F1 (mean)
- 2.26%
- Setup 2
- 1.12%
- Setup 6
- 3.24%
- Setup 7
- 2.42%
Open DINOv2
- F1 (mean)
- 0.85%
- Setup 2
- 0.67%
- Setup 6
- 0.90%
- Setup 7
- 0.98%
Open V-JEPA2
- F1 (mean)
- 0.27%
- Setup 2
- 0.20%
- Setup 6
- 0.23%
- Setup 7
- 0.39%
Make physical experience usable
A new machine should be able to learn from interactions recorded before it existed, somewhere else, on different hardware.
Run the benchmark
Test your model with the prepared videos, labels and evaluator.
Get the benchmark ↗