Introducing Perceptron Egocentric 1.0

Today we're releasing Perceptron Egocentric 1.0, a self-service API that turns first-person human and robot video into structured training data for robotics. It delivers frontier-quality subtask timelines, continuous activity labels for each hand, and state-of-the-art 3D hand reconstruction at $29 per video-hour, less than half of what GPT-6 Astra costs for subtask timelines alone (~$76 per video-hour), all accessible through a single API.

Explore what Perceptron Egocentric produces on household, robot and manufacturing video: pick a clip, play it, click any step to jump to that moment, and switch between 2D keypoints and the 3D hand mesh. Robot clips are from the Galaxea Open-World dataset [1]; manufacturing clips from Build AI's Egocentric-10K [2].

Despite recent advances in egocentric video understanding, high-quality annotation at scale is still hard. Perceptron Egocentric closes three gaps:

Subtask labels at a fraction of the cost. Frontier models like GPT-6 Astra produce strong subtask annotations but cost too much to run on large datasets. Standard benchmarks don't capture segments less than a second. On , we score within 0.01 of GPT-6 Astra at one-eighth the cost ($9 vs $76 per video-hour). We also achieve the highest reported segment F1 on WGO-Bench [3].
Structured, per-hand annotations. Few pipelines combine continuous activity labels for each hand, a consistent action taxonomy, and precise 2D keypoints and 3D hand reconstruction. We provide all of these, including a new 3D hand reconstruction model with state-of-the-art hand-shape accuracy on HOT3D [4] and ARCTIC [5].
Simple, self-service access. Open-source tools leave compute, scaling and deployment to your engineers, and managed providers often require custom engagements. Our API lets you submit videos, track jobs and download structured annotations, with transparent pricing and no manual onboarding.

Together, these capabilities mark a major upgrade from our early-access launch in July. Behind the API, our custom egocentric annotation harness combines specialized models for subtask segmentation, per-hand activity labeling, and 2D/3D hand tracking to deliver high-quality annotations cost-effectively at scale.

Labeling subtasks at a fraction of the cost

In first-person video, grasping, releasing or turning an object can take a fraction of a second. Annotations at whole-second resolution lump these actions together, making it harder for robots to learn the timing of each step. Perceptron Egocentric segments every video into timestamped subtasks, including these short actions.

Frontier-quality timelines at an eighth of the cost

At its default 4 fps, Perceptron Egocentric scores 0.787 on EgoHome, our new benchmark of sub-second household steps (described below), within 0.010 of GPT-6 Astra at 4 fps (0.797), for $9 per video-hour against ~$76. At 8 fps it scores 0.798, level with GPT-6 Astra, at an estimated $15. Both settings outperform NVIDIA's FLORA (0.747).

Figure 1: EgoHome annotation quality versus cost per video-hour, on a logarithmic cost scale. The dashed line is the Pareto frontier, the system that no other system beats on both score and cost; the orange area above and to the left of it, cheaper and higher scoring, is reached by no system we measured. Both Perceptron Egocentric settings lie on the frontier. Pantheon Argus (0.32 to 0.37) is below the chart's range and not shown.

On the public WGO-Bench benchmark, Perceptron Egocentric achieves the highest segment F1 (0.583), which measures segment timing, and the second-highest semantic F1 (0.474), which measures timing and labels together.

Table 1: WGO-Bench, 100 videos (higher is better). Semantic F1 counts a step as correct when both its timing and its label match; segment F1 checks timing only. Both are pooled: matches are counted across all 100 videos before computing F1, rather than averaging a per-video F1, to match how other teams report WGO-Bench. Best in bold.

System	Semantic F1 ↑	Segment F1 ↑
Midcentury Optimized Argus † [6]	0.502	0.579
Perceptron Egocentric	0.474	0.583
Glimpse (Elorian) †	0.427	–
GPT-6 Astra †	0.394	–
FLORA (NVIDIA) † [7]	‡	0.503 ‡

† Self-reported, not reproduced by us. ‡ FLORA reports WGO-Bench results under its own evaluation protocol, so they are not directly comparable: we omit its semantic F1 and show its reported segment F1.

Capturing steps shorter than a second

WGO-Bench has no reference segments shorter than one second, so it can't tell whether short actions are captured. To measure that, we built EgoHome, a new benchmark of 137 manually verified clips covering roughly two hours of first-person household chores. In 24 of them, at least one step is shorter than half a second.

2× speed

0.0 s / 61.0 s

Gold labelHolding the maroon pillowcase on the bed

Perceptron EgocentricFold bottom edge of maroon cloth up on mattress

GPT-6 Astra (4 fps)Fold and smooth the burgundy patterned cloth

Figure 2: Subtask timelines for one EgoHome clip of a person folding laundry, at double speed: the reference labels (gold), Perceptron Egocentric with default settings, and GPT-6 Astra at 4 frames per second. Gold and Perceptron Egocentric both split the clip into 19 steps; GPT-6 Astra returns 8, labeling 30 seconds of towel folding as a single step. On this clip Perceptron Egocentric scores 0.77 and GPT-6 Astra 0.59; across all of EgoHome the two score about the same (Table 2). Full video.

We also introduce a new metric, permissible subsegment Jaccard, that accounts for differences in annotation granularity. The same 45-second sequence might be labeled “doing dishes” or split into “scrubbing,” “rinsing” and “drying”; both are valid. Rather than penalizing either, the metric uses an LLM judge to decide when coarse and fine descriptions cover the same activity, then measures agreement between the predicted and reference timelines. We will release EgoHome, along with the metric's details and code, in the coming days.

Table 2. EgoHome results on all 137 clips. Subtask score (higher is better) is the share of video time on which predicted and reference steps agree (permissible subsegment Jaccard), shown overall and by length of the reference step. Cost is per hour of video (lower is better). Best in bold, second best underlined.

System	All ↑	< 0.5 s ↑	0.5–1 s ↑	> 1 s ↑	Timeline $ / video-hour ↓
Perceptron Egocentric, 8 fps (option)	0.798	0.555	0.672	0.824	≈ $15
GPT-6 Astra, 4 fps frames	0.797	0.503	0.649	0.831	≈ $76
Perceptron Egocentric, 4 fps (default)	0.787	0.559	0.615	0.814	$9
GPT-6 Astra, 1 fps frames	0.775	0.595	0.500	0.807	≈ $20
FLORA (NVIDIA)	0.747	0.579	0.615	0.777	$35.39
Glimpse (Elorian), 4 fps video	0.689	0.380	0.410	0.720	≈ $2.8
Gemini 3.8 Flash, 4 fps video	0.663	0.375	0.392	0.695	≈ $1.3
Argus (Pantheon), GPT-6 Astra [8]	0.367	0.135	0.133	0.381	$24.17
Argus (Pantheon), Midcentury's settings: GPT-6.1 Sol [6]	0.315	0.098	0.106	0.332	$7.76
Clips with a step of this length	137	24	60	137

All systems were scored on all 137 clips with the same scorer. Perceptron results average three runs at 4 fps and two at 8 fps; other systems ran once. GPT-6 Astra received timestamped frames; Gemini and Glimpse received native video. Argus labels describe object states (for example, "mug loaded into the upper rack") rather than actions, which EgoHome's judge counts as mismatches.

Cost is what each system charges to produce timelines for one hour of video: Perceptron's customer price, API charges for GPT-6 Astra, Gemini and Glimpse, and GPU runtime for FLORA. Argus costs also include hand labels. Costs marked ≈ are estimates. GPT-6 Astra, Gemini and Glimpse were billed once for all tasks, so we counted only the timeline's share: one pass of the video as input plus the timeline output. For 8 fps, we scaled our $9 price by its 1.7× higher model spend.

On steps under half a second, GPT-6 Astra at 1 fps (0.595) and FLORA (0.579) score slightly higher than ours (0.559); on steps of half a second to a second, our 8 fps setting scores highest (0.672).

Tracking each hand in 3D and with activity labels

Perceptron Egocentric treats each hand separately: it reconstructs both hands in 3D and labels what each one is doing over time.

Reconstructing hands in 3D turns human demonstrations into supervision for robot manipulation. Recent work such as EgoMimic [9] has shown that co-training manipulation policies on egocentric video paired with 3D hand tracking improves performance on long-horizon single-arm and bimanual tasks. VideoDex [10] mines internet videos, retargeting estimated 3D hand motions to a dexterous robot hand as a prior for policy learning.

State-of-the-art hand shape, no calibration needed

Perceptron Egocentric reconstructs 3D hands directly from egocentric video. Our model, built on Mk1.5, estimates each hand's 3D shape, finger articulation and position relative to the camera. It reconstructs hand shape 12–25% more accurately than ACE-Ego-Hand [11], the strongest prior method in Table 3, in all four settings, and needs no camera calibration: its hand-shape error on HOT3D is 5.21 mm with or without it.

Figure 3: Hand tracking on two first-person dish-washing clips filmed on a phone, with no camera calibration: (a) bare hands, (b) rubber gloves. Each hand gets its own 3D mesh, labeled with its side, detection confidence and distance from the camera. Left of each row: 2D keypoints; right: the 3D mesh on the frame and the same hands seen from another angle. Full 10-second videos: bare hands 2D, bare hands 3D, rubber gloves 2D, rubber gloves 3D.

Table 3: Hand shape and pose accuracy are measured on HOT3D [4] and ARCTIC [5], two standard benchmarks of first-person videos showing hands manipulating objects. Errors are measured in millimeters (lower is better). Hand shape (PA-MPJPE) aligns translation, rotation, and scale to isolate finger articulation errors. Hand pose (wrist-aligned MPJPE) aligns only wrist position, penalizing global orientation and scale errors. Calibrated means the model is given the camera's parameters, such as focal length and image center. Uncalibrated means it must estimate them, as our API does automatically. Best results are in bold and second-best results are underlined.

	Hand shape (PA-MPJPE)				Hand pose (wrist-aligned MPJPE)
Method	HOT3D cal. ↓	HOT3D uncal. ↓	ARCTIC cal. ↓	ARCTIC uncal. ↓	HOT3D cal. ↓	HOT3D uncal. ↓	ARCTIC cal. ↓	ARCTIC uncal. ↓
Perceptron (ours)	5.21	5.21	6.61	6.61	15.62	15.63	15.65	15.29
ACE-Ego-Hand †	6.44	6.77	7.47	8.76	12.89	13.54	15.26	16.62
ViDiHand †	11.38	–	9.82	–	–	–	–	–
WiLoR †	19.98	–	11.87	–	–	–	–	–
HaMeR †	21.46	–	14.60	–	–	–	–	–

† From the ACE-Ego-Hand paper [11] (Table 1), not reproduced by us. That paper reports HaMeR [12], WiLoR [13] and ViDiHand [14] for calibrated hand shape only. Its segment list is not public, so our segments differ.

A separate activity timeline for every hand

Each hand also gets its own activity labels: a separate timeline per hand, with a short description of what that hand is doing at every step. Robot grippers get one timeline each in the same way. In Figure 4, the person's left hand holds the pan and tilts it under the water while the right hand turns on the faucet and rinses the sponge, and each gets its own labels. On the two-armed robot, the left gripper moves a cup and a pump bottle into one basket while the right gripper moves a peach into the other.

Figure 4: Per-hand activity labels from Perceptron Egocentric on (a) the bare-hands dish-washing clip from Figure 3, at half speed, and (b) a two-armed Galaxea robot [1] sorting objects into baskets, at normal speed. Each hand or gripper has its own timeline; the orange step is the one happening now, and its label is shown above the bar. A dash means that hand or gripper is out of view. Labels are the model's output, unedited. Full video.

Start annotating today

Perceptron Egocentric provides subtask segmentation, per-hand activity labels, and 2D/3D hand tracking out of the box. It runs as an asynchronous batch API. Send up to 1,000 video links in one request, choose the outputs you want, then poll each job and download its results when it finishes. Subtask segments come with every job; 2D or 3D hand tracking and per-hand activity labels are add-ons chosen per batch. The Perceptron dashboard lists every batch and job with its status, the hours of video processed and the credits charged, next to your held and available balance.

A completed batch in the Perceptron dashboard: 240 jobs, 172.8 billed video hours, a cost breakdown by output and the list of jobs. — **Figure 5:** A batch in the Perceptron dashboard shows your annotation cost broken down by output (subtask timelines, 3D hand tracking and activity labels) and the status and cost of every job.

The API has seven endpoints:

Submit Batch (POST /v1/egocentric/batches) — send up to 1,000 video links in one request with an idempotency key; the batch is accepted or refused as a whole.
List Batches (GET /v1/egocentric/batches) — page through your organization's batches.
Retrieve Batch (GET /v1/egocentric/batches/{batch_id}) — a batch's status and timing.
List Batch Jobs (GET /v1/egocentric/batches/{batch_id}/jobs) — the jobs in a batch, in input order.
List Jobs (GET /v1/egocentric/jobs) — all of your organization's jobs, in creation order.
Retrieve Job (GET /v1/egocentric/jobs/{job_id}) — a job's status, input metadata, errors and execution timing.
Retrieve Job Result (GET /v1/egocentric/jobs/{job_id}/result) — download the job's result JSON once it succeeds.

See the egocentric guide for authentication, limits and error handling.

Submit videos with POST /v1/egocentric/batches:

curl https://api.perceptron.inc/v1/egocentric/batches \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "inputs": [
      {
        "type": "video_url",
        "video_url": { "url": "https://example.com/videos/kitchen.mp4" }
      }
    ],
    "hand_tracking": { "output": "3d" },
    "hand_activity": true
  }'

Check a job's status with GET /v1/egocentric/jobs/{job_id}:

curl https://api.perceptron.inc/v1/egocentric/jobs/$JOB_ID \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY"

Download its results with GET /v1/egocentric/jobs/{job_id}/result:

curl https://api.perceptron.inc/v1/egocentric/jobs/$JOB_ID/result \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -o result.json

Outputs: Subtask timelines, always included. 2D or 3D hand tracking and per-hand activity labels are optional add-ons, selected per batch.

Pricing is per hour of video, by output:

Egocentric list prices per hour of video, by output
Output	Per video-hour
Subtask timeline (always included)	$9
2D hand tracking (add-on)	$12
3D hand tracking (add-on)	$14
Per-hand activity labels (add-on)	$6

Videos shorter than a minute are billed as one minute. Volume discounts available; contact sales@perceptron.inc.

Billing: Credits for a job's video length are held when it starts and charged only if it succeeds. The usage dashboard lists every job with its hours and cost.

The standard API is fully self-service. For additional capabilities, contact sales@perceptron.inc; for support, support@perceptron.inc.

Create an API key

Read the docs

Notes on evaluation

Hand pose: We follow the ACE-Ego-Hand evaluation protocol on HOT3D (participant P0001, 244 segments) and ARCTIC (subject s05, 302 segments). All model and evaluation choices were finalized on separate validation data before a single test evaluation.
EgoHome: We use a Gemini 3.5 Flash judge. All systems in Table 2 were evaluated in a single pass, with judge verdicts cached for each label pair to ensure consistent scoring. Perceptron settings were run two or three times; competitors were run once. The model evaluated on EgoHome was trained without EgoHome clips.
WGO-Bench: We follow the benchmark's published evaluation rubric. A predicted step matches a reference step if its temporal IoU is at least 0.75, after aligning the outer boundaries to the reference. Semantic F1 additionally requires a Gemini 3.5 Flash judge to verify that the predicted label matches the task instruction. F1 is pooled across all 100 videos rather than averaged per video, which would inflate scores by 0.14–0.19.
Self-reported results: Scores marked as self-reported were taken from external sources and were not independently reproduced.

Citation

If you reference this work, please cite it as:

@article{perceptron2026egocentric,
  author = {PerceptronAI},
  title = {Introducing Perceptron Egocentric 1.0},
  journal = {PerceptronAI Research Blog},
  year = {2026},
  note = {https://perceptron.inc/blog/introducing-perceptron-egocentric-1-0},
}

References

T. Jiang, T. Yuan, Y. Liu, et al. Galaxea Open-World Dataset and G0 Dual-System VLA Model. 2025. arXiv:2509.00576
Build AI. Egocentric-10K. Hugging Face dataset. huggingface.co/datasets/builddotai/Egocentric-10K
Macrodata Labs. Segmenting Robot Video into Actionable Subtasks (WGO-Bench). June 2026. macrodata.co/blog/annotating-robot-video-subtasks
P. Banerjee, S. Shkodrani, P. Moulon, et al. HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos. CVPR 2025. arXiv:2411.19167
Z. Fan, O. Taheri, D. Tzionas, et al. ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation. 2022. arXiv:2204.13662
Midcentury. Optimizing Argus for Action Labelling. October 2026. midcentury.xyz/blog/optimizing-argus
S. Leng, Y. Man, Z. Li, et al. FLORA: An Open Model Suite for Scaling Robotics Data Engines. 2026. arXiv:2610.31779
Pantheon Industries. Argus: an open source harness for annotating robot learning data. GitHub repository. github.com/Pantheon-Industries-Inc/argus
S. Kareer, D. Patel, R. Punamiya, et al. EgoMimic: Scaling Imitation Learning via Egocentric Video. 2024. arXiv:2410.24221
K. Shaw, S. Bahl, D. Pathak. VideoDex: Learning Dexterity from Internet Videos. CoRL 2022. arXiv:2212.04498
Y. Liu, X. Wang, H. Li, et al. ACE-Ego-Hand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery. 2026. arXiv:2608.20308
G. Pavlakos, D. Shan, I. Radosavovic, et al. Reconstructing Hands in 3D with Transformers (HaMeR). CVPR 2024. arXiv:2312.05251
R. A. Potamias, J. Zhang, J. Deng, et al. WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild. CVPR 2025. arXiv:2409.12259
Y. Wang, C. Jin, Y. Liu, et al. The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (ViDiHand). 2026. arXiv:2606.30308